WO2010069878A1

WO2010069878A1 - Method for acquiring images using a matrix detector

Info

Publication number: WO2010069878A1
Application number: PCT/EP2009/066957
Authority: WO
Inventors: Olivier Cocle; Frédéric Bellot
Original assignee: Thales
Priority date: 2008-12-16
Filing date: 2009-12-11
Publication date: 2010-06-24
Also published as: FR2939938A1; FR2939938B1

Abstract

The invention relates to a method for acquiring images of a scene (4), said images comprising previously indexed faulty pixels (101). The method includes a step of acquiring, using a matrix detector (10), an image of the scene, referred to as an image to be corrected (1), having a line of sight that is offset by a predetermined offset D relative to the line of sight of a reference image (2') of the scene including valid pixels (200), and a step of processing the acquired images that comprises the following sub-steps: resetting (B) the reference image (2) with the image to be corrected (1) from the offset D; replacing (C1, C2) the indexed faulty pixels (101) of the image to be corrected with the valid pixels (200) of the reference image (2') representing the same information of the scene (4) in order to obtain a corrected image (3); if a predetermined rejuvenation criterion is respected, rejuvenating (D) the reference image (2') by replacing the pixels of the reference image with the non-faulty pixels (101) of said image to be corrected; repeating the above sub-steps while acquiring (A), using the matrix detector (10), a new image to be corrected having a line of sight that is offset relative to the previous image to be corrected by a new offset D.

Description

METHOD FOR ACQUIRING IMAGES USING A MATRIX DETECTOR

The field of the invention is that of the acquisition of images by means of a matrix detector, in particular an IR detector.

One of the major problems that must be solved to provide a good quality infrared image is the correction of the non-uniformities of infrared detectors regardless of the technology.

The cooled sensors based on quantum detectors using the materials InSb, HgCdTe or AsGa (QWIP technology), uncooled pyroelectric or bolometric sensors (VOx or αSi) present on the same matrix of individual detectors, different responses to each other; some of these individual detectors are not usable because of a too weak, non-linear, too noisy or saturated response, etc., and generate in the image defective pixels. They can prevent the detection of objects of interest and in particular targets in the directions corresponding to these detectors.

Indeed, for wide-field monitoring systems for example, using detector arrays, the target whose image is smaller than the size of a pixel, can be projected on the same pixel for a long time (tangential velocity null, or displacement of small amplitude compared to the resolution of the sensor). In this case, the detection of this target can not take place if the target is projected on a dead detector, which reduces the overall performance of the system and can make it unacceptable in the case in particular of airborne, naval or earthly. One of the quality criteria of the matrix detector is the operability rate, ie the percentage of usable individual detectors. This criterion defines ranges of matrix detectors whose cost price varies according to the desired level of operability. In order to have low-cost detectors, a number of defective individual detectors that can be large and sometimes contiguous are tolerated, thus forming aggregates designated "clusters" in English.

Several techniques are currently used to replace the defective pixels: they consist of making spatial interpolations more or less complex. Whatever the subtlety and complexity of solutions based on interpolations and spatial filtering, information that has not been acquired can not be reconstructed with a guarantee of reality, especially in the case of large aggregates. Complex solutions allow to present an image pleasing to the eye but do not regenerate the missing information.

Consequently, there remains to this day a need for the correction of defective pixels of a matrix detector simultaneously satisfying all the aforementioned requirements, namely a correction by a value close to that which they should have had. they had been valid.

According to the invention, the correction of defective pixels is obtained by time extrapolation. The invention consists in reconstructing the image from a detector generating defective pixels, by using the valid pixels of the preceding images which are spatially at the same location. The information from the visualized scene, which could be hidden behind an aggregate of defective pixels, is thus unmasked by moving the line of sight, that is to say by modifying the position of the defective pixels relative to the scene. This solution makes it possible to replace the defective pixels with pixels having original information of the scene and not with a pixel reconstructed from other information already present in the image at the same time.

More specifically, the subject of the invention is a method for correcting images of a scene by means of a matrix detector, these images comprising defective pixels previously listed. It comprises a step of acquiring an image of the scene, called the image to be corrected, whose line of sight is shifted by a determined offset D with respect to the line of sight of a reference image of the scene comprising valid pixels, and a step of processing the acquired images which comprises the following substeps:

- readjust the reference image with this image to be corrected from the offset D,

replace the defective pixels listed in the image to be corrected by the valid pixels of the reference image, representing the same scene information, in order to obtain a corrected image,

refreshing the reference image by replacing the pixels of the reference image by the non-defective pixels of said image to be corrected,

repeating the preceding substeps by acquiring a new image to be corrected, the line of sight of which is shifted with respect to the previous image to be corrected, of a new determined shift D. According to one characteristic of the invention, the image reference is initially constructed as follows:

- all the pixels of the reference image are declared invalid,

- acquisition and storage of an image of the scene, - replacement of the pixels of the reference image by the non-defective pixels of the image of the scene.

According to another characteristic of the invention, a pixel of the reference image is declared invalid if its age counted in number of replacement absences is greater than a maximum age. Preferably, when a pixel of the reference image is replaced, it is declared valid and its age is set to 0 otherwise for each other pixel, if its age is less than a maximum age, its age is incremented by 1 otherwise the pixel is declared invalid.

According to a preferred variant, when the defective pixels of said image to be corrected are not replaced, these pixels being designated residual defective pixels, it comprises a step of correcting the residual defective pixels of said image, by spatial extrapolation with pixels of the same picture.

The residual defective pixels are possibly listed in a table.

The invention also relates to an imaging system equipped with an image acquisition device of a scene which comprises a matrix detector, and a unit for processing acquired images, characterized in that the unit of treatment includes means of implementation of the described method. It possibly includes a device for orienting the line of sight of the detector.

According to a characteristic of the invention, the processing unit comprises means for determining the offset D, means for storing the offset D, the table of defective pixels and a reference image and possibly a table of residual defective pixels.

Other characteristics and advantages of the invention will become apparent on reading the detailed description which follows, given by way of nonlimiting example and with reference to the appended drawings in which: FIG. 1 represents a flowchart describing the main steps of the method according to the invention, FIGS. 2 diagrammatically illustrate the different steps of registration of the reference image with the image to be corrected, FIG. 3 diagrammatically represents an imaging system according to the invention.

From one figure to another, the same elements are identified by the same references.

The defective pixels of the images come from defective individual detectors; these defective pixels are known and listed for example in a table.

A pixel is declared defective if it is unusable. This is the case when, for example, its value is outside a predetermined range of values, or possibly lower than a threshold value.

The defective pixels are corrected in the images from a reference image. The pixels of the reference image are associated with a validity status which will be defined later: this reference image therefore comprises valid or invalid pixels.

The size of the reference image is at least equal to that of the images to be corrected. It is preferably larger, typically 1, 5 times larger.

For example, the reference image is initially constructed as follows, as shown in FIG. 2a. At first, all the pixels of the reference image are declared invalid: we have a reference image 2 'empty. Then an image of the scene 4 is acquired and stored (it is designated image to be corrected 1 and corresponds to the scene 4-ι) and the pixels of the reference image 2 'are replaced (or rejuvenated) by the valid pixels of this image 1 of scene 4: we obtain a refreshed reference image 2. This amounts to applying the process described later with D = O. An initial reference image 2 is thus obtained that includes invalid pixels 201, which correspond to the defective pixels 101 listed and the acquired image size pixels. As this reference image is obtained, the method according to the invention comprises a step of acquiring an image of scene 4, called the image to be corrected 1 whose line of sight is shifted with respect to the line of sight of the previous image (of the scene), of a determined shift D; it corresponds to the scene A ₂ . In this way, from one image to another, the position of the pixels relative to the scene is modified and in particular that of the defective pixels: the information from the displayed scene, which is hidden behind the defective pixels of the image to correct can be unmasked when they are on valid pixels of the reference image. The process of replacing the defective pixels in the image to be corrected will be described in relation to the flowchart of FIG. 1 and FIGS. 2b and 2c; they are replaced by the valid pixels of the reference image.

Each pixel of the reference image 2 'is of course associated with its value (its gray level or color level) as well as a validity status (step G). The pixel is declared valid if a validity criterion is fulfilled. This criterion is for example based on the age of the pixel counted in number of images without replacement. A pixel of the reference image is declared valid if: - it has already been replaced by a pixel of an acquired image - and if the pixel is not too old, that is to say if its age n ' has not reached a predetermined maximum age. This maximum age can be set according to different criteria such as the speed of movement of the line of sight, or the image acquisition rate. If one of these criteria is not satisfied, the pixel is declared invalid and listed as such in the status associated with the pixel. For each pixel listed in the table 31 of the defective pixels, it is examined whether the spatially corresponding pixel in the reference image is valid. As illustrated in FIG. 2, this spatial correspondence is obtained by recalibrating the two images with respect to each other from the determined offset D (step B), in order to be able to compare the pixels representing the same information of the displayed scene: gets a dummy reference image 2 '. The locations of the pixels are generally located from an orthonormal reference frame Ox, Oy. The registration of the images then consists, for example, in translating the repository.

If the pixel of the reference image 2 'is valid, it is used in the image to be corrected 1 to replace the defective pixel (steps C1 and C2), otherwise it is not used and preferably we index the pixel remaining defective of the image to be corrected in the table of residual defective pixels 32 (step F). This table can be used for a complementary correction (step E) based on a spatial extrapolation of the residual defective pixels with other pixels of the same image.

For the other pixels of the image to be corrected (which are therefore not defective), they are copied into the reference image, that is to say that their value is assigned to the corresponding pixels of the reference image; in addition, these copied pixels are declared valid and their age is 0 which results in the status: V, 0. The reference image is thus refreshed (step

D) as and when corrections are made: we get a refreshed reference image 2. In fact it is partially refreshed since only some pixels are refreshed.

For pixels in the reference image that have not been refreshed and are valid, their age is incremented by 1. This also applies when the reference image is larger than the image to be corrected, to the pixels of the reference image located outside those of the image to be corrected after registration: this is represented on the image Refreshed reference numerals of FIGS. 2B and 2C.

More generally, this refresh can be performed:

at each iteration, that is to say for each image to be corrected, ie at the image frequency, or at a periodicity to be determined, or according to the offset applied, - or any other criterion.

These steps are repeated by acquiring a new image to be corrected, whose line of sight is shifted with respect to the previous image (and therefore with respect to the reference image) of a new shift D (step A). The refreshed reference image becomes the reference image to be recalibrated.

To implement this invention in an imaging system such as a camera (or binocular), it is necessary for the system to have access to the measurement of the movement of the line of sight, i.e. D offsets in order to spatially recalive the successive images. This mechanism can be external or internal. It can come from information of the system carrier or the system itself. Portable cameras that have an image stabilization system must have information on the movement of the line of sight. In other cases, the shift D of the line of sight is imposed by the imaging system.

Offsets (undergoing or imposed) are stored for use when resetting images to each other.

An example of an imaging system according to the invention is shown in FIG. 3. It comprises an image acquisition device of a scene which comprises a matrix detector 10, a device 20 for orienting the line of sight of the detector. , and a processing unit 30 acquired images.

This processing unit comprises means for implementing the method described above. It is connected to storage means 31 of the table of defective pixels that have been previously listed.

It includes means for determining and storing 32 offsets

D, the reference image, and preferably a table of residual defective pixels.

The means for determining the offset D may be means for measuring the offset D when it is for example a shift undergone; this is illustrated by the arrow that goes from the detector 10 to the device 20 which calculates this offset. The shift can also be imposed; this is illustrated by the arrow that goes from the device 20 to the detector 10.

For images to be corrected with, for example, 384 x 288 pixels, a reference image of 576 × 432 pixels can be considered. ie 1, 5 times larger than the image to be corrected. When the reference image and the image to be corrected are recalibrated with each other, only part of the reference image is taken into account during the pixel replacement process: it is of course the part corresponding spatially to that of the image to correct.

The successive offsets typically vary from 1 to 10 pixels without this value being limiting (it is reasonable not to exceed a quarter of the size of the image). These offsets may or may not be identical from one image to another.

Claims

1. Method for correcting images of a scene (4), these images comprising defective pixels (101) previously listed, characterized in that it comprises an acquisition step by means of a matrix detector (10) of an image of the scene, called the image to be corrected (1), whose line of sight is shifted by a determined offset D with respect to the line of sight of a reference image (2 ') of the pixel scene valid (200), and a step of processing acquired images that includes the following substeps:

- Recalibrate (B) the reference image (2) with the image to be corrected (1) from the shift D, - Replace (C1, C2) the defective pixels (101) listed in the image to be corrected (1). ) by the valid pixels (200) of the reference image (2 '), representing the same information of the scene (4), in order to obtain a corrected image (3),

- refreshing (D) the reference image (2 ') by replacing the pixels of the reference image with the non-defective pixels (101) of said image to be corrected,

repeating the preceding substeps by acquiring (A) by means of the matrix detector (10) a new image to be corrected, the line of sight of which is shifted relative to the previous image to be corrected, of a new shift D.

2. An image correction method according to the preceding claim, characterized in that the reference image is refreshed according to a predetermined refresh criterion.

An image correction method according to one of the preceding claims, characterized in that the reference image is initially constructed as follows:

- all the pixels of the reference image are declared invalid,

- acquisition and storage of an image of the scene, replacing the pixels of the reference image with the non-defective pixels of the scene image.

4. image correction method according to one of the preceding claims, characterized in that the defective pixels are listed in a table (31).

5. An image correction method according to one of the preceding claims, characterized in that a pixel of the reference image (2) is declared invalid if its age counted in number of replacement absences is greater than at a maximum age.

6. A method of image correction according to the preceding claim, characterized in that when a pixel of the reference image (2) is replaced, it is declared valid (F) and its age is set to 0 otherwise for every other pixel, if its age is below a maximum age, its age is incremented by 1, otherwise it is declared invalid.

7. A method of image correction according to one of the preceding claims, characterized in that when the defective pixels of said image to be corrected (1) are not replaced, these pixels being designated residual defective pixels, it comprises a step of correction (E) of the residual defective pixels of said image, by spatial extrapolation with pixels of the same image.

8. An image correction method according to the preceding claim, characterized in that the residual defective pixels are listed in a table (32).

9. image correction method according to one of the preceding claims, characterized in that the reference image is larger than the image to be corrected.

An imaging system equipped with a scene acquisition device (4) which comprises a matrix detector (10), and a display unit processing (30) acquired images, characterized in that the processing unit comprises means for implementing the method according to one of the preceding claims.

11. Imaging system according to the preceding claim, characterized in that it comprises a device (20) for orienting the line of sight of the detector.

12. Imaging system according to one of claims 10 or 1 1, characterized in that it comprises means (31) for storing the table of defective pixels and in that the processing unit (30) comprises means for storing (32) the offset D, and the reference image (2).

13. Imaging system according to one of claims 10 to 12, characterized in that the processing unit (30) comprises means (32) for storing a table of residual defective pixels.