WO1996031978A1

WO1996031978A1 - Method and device for eliminating defects in a ccd camera

Info

Publication number: WO1996031978A1
Application number: PCT/FR1996/000489
Authority: WO
Inventors: Jean-Claude Robin; Yvon Masse; Christian Burgos; Patrice Menard
Original assignee: Lhesa Electronique
Priority date: 1995-04-07
Filing date: 1996-04-01
Publication date: 1996-10-10
Also published as: FR2732850B1; FR2732850A1

Abstract

A CCD camera comprising means for eliminating defects that reduce sensed image quality, and a method therefor, are disclosed. The defect elimination means are portable and built into the camera housing, and include electronic filtering means for filtering in real time each pixel of an image sensed at time Ti, by averaging it with the pixel at the same address in the image sensed at time Ti-1, as well as electronic means for comparing the image averaged in real time with a reference image taken at a predetermined time.

Description

METHOD AND DEVICE FOR ELIMINATING DEFECTS IN A LOAD COUPLING CAMERA.

The present invention relates to a charge coupled camera comprising means for eliminating faults altering the quality of the captured image.

It also relates to a method for eliminating defects in the image obtained with a charge coupled camera.

It finds a particularly important, although not exclusive, application in the field of real-time or quasi-real imaging with very low light flux using television cameras.

These are particularly and not exclusively the fields of optical microscopy in fluorescence at very low light levels, electron microscopy, industrial and medical radiology, astronomy, particle detection and / or of the nuclear industry for the detection of low radiation.

The present invention is also applicable to night surveillance or in the field of infrared vision or even more generally to fields which relate to imaging under difficult optical conditions requiring real-time information for automatic processing.

In such conditions, several factors alter the quality and the representation of details on images from a television camera, especially when they have a very high sensitivity.

The defects affecting the image quality are first and foremost due to the sources of quantum noises linked to the number of incident photons generating the image, sources of quantum noises linked to the fluctuation of the number of electrons in the case of an electron microscope, or sources of noise of electronic origin due to the factors of amplification necessary to optimize the sensitivity of the camera.

The noise also comes from the detector converting photons into electrons. It can also be created by the light intensifier when one is used to increase the sensitivity of the camera and / or be generated by the fluctuation of particles in the nuclear fields as well as in radiology. The defects can then be due to an inhomogeneity of structure of revolution, for example in the event of vignetting of the optics (for example in astronomy) in the event of inhomogeneity of the light sources, and this in particular when the absorption of light is not homogeneous as with optical fiber reducers where the differences in length of the optical paths between the center and the edges of the reducing networks generate this type of defect.

Other faults are linked to the inhomogeneity of the medium without, however, having a structure of revolution.

These defects are for example localized defects due to a loss of sensitivity, an abnormal absorption of light (notably optical spots), or even structural defects linked to the technology of optical fiber networks (known as "Chicken ire" effect in English terminology). , faults generated during the coupling of said networks, for example because of a problem of glue or due to a strong gain used in electronic circuits etc.

Finally, the defects altering the quality of the captured image can be of a point type. For example these are the black dots due to a point loss of sensitivity coming either from the intensifier of the charge coupled camera

("CCD" in English terminology) or black spots inherent in the assembly technology of optical fibers.

It can also be white dots, for example coming from defective pixels of the charge-coupled camera.

Devices are already known which make it possible to limit and / or eliminate some of the defects of the type described above, altering the quality of the images captured by a charge-coupled camera.

For example, these can be minimized by a drastic selection of the components of the camera in order to obtain an acceptable overall characteristic with regard to technological defects.

However, such devices have drawbacks, namely poor performance, a significant increase in costs and delivery times, the need to provide for large stocks with the obligation to create a market for products with reduced specification, which is not not acceptable in the context of high-end products and does not really remedy the problems originating in physical phenomena.

Another known means consists in introducing a corrective circuit called "shading correction" in English terminology.

But such a solution only makes it possible to compensate for defects having a character of revolution, as is the case for the vignetting of optics or the non-conformity of electronic lenses in intensifier tubes.

However, it does not resolve localized faults or those of physical origin.

Another known way of minimizing the defects altering the image quality consists in performing an image processing with a correction function corresponding to the formula SC = S - B / W - N with

SC: Video signal after Correction

S: Useful signal to detect

N: Black signal B: Signal delivered by the camera when it observes an intense homogeneous image (White signal).

This latter solution, only implemented with a complex image processing system, does not, however, by itself allow the faults of the charge-coupled camera originating from noise sources, and this in particular when it is used in very low light flux imaging.

In fact, the only solution known to date ensuring a truly effective correction of camera faults uses a device comprising an external image processing system with associated computer.

This solution again has drawbacks. In particular, it does not allow operation in real time, it is expensive and requires specialized operators. It is more complex and cumbersome to use.

The present invention aims to provide a charge coupled camera comprising means for eliminating faults altering the quality of the captured image, and a method using such a camera, which meets the requirements of practice better than those previously known, in particular in that it enables effective correction in real or almost real time of all the faults mentioned above, in an environment with very low light, and in that it is simple to use, portable, space-saving and inexpensive.

The camera according to the invention can also operate in two or three modes which can be used simultaneously, which makes it possible to increase its sensitivity performance by a factor of more than ten, while maintaining, thanks to the elimination of faults, a quality of exceptional image. The three functions which can be used simultaneously are those of integration on the "CCD", classic function allowing to increase the quantity of light obtained by image, of averaging of images after integration allowing to improve the characteristic of signal to noise ratio by erosion, and elimination of point type defects.

For this purpose, the present invention proposes in particular a charge coupled camera comprising means for eliminating faults altering the quality of the captured image, characterized in that said elimination means are portable, integrated into the camera housing and include: - electronic filtering means arranged to perform real-time filtering on each pixel of the image captured at time Ti, by averaging with the pixel located at the same address of the image captured at time Tι -ι. - And electronic means of comparison between the average image in real time and a reference image taken at a determined time.

In advantageous embodiments, one and / or the other of the following arrangements are also used:

- The camera comprises means for accumulating the video image for a determined sequence time t before restoring the image, said accumulation means being arranged to operate simultaneously with the filtering and comparison means.

This makes it possible to obtain, with a camera of given sensitivity, higher performance than that obtained with a camera equipped with an additional intensification stage, without however introducing the additional constraints inherent in such a device and in particular, in addition to the cost, decrease in resolution, non-uniformity and increase in volume of the camera head.

- The filtering means comprise: means for acquiring the digital brightness value ii__ι of each pixel of the image captured at an instant Tι-ι. means for storing these digital values X - _J. at determined addresses corresponding to each pixel, means for acquiring the digital brightness value i of the pixels of the following image located at the same address, at time li, means for calculating an average value of the numerical values of each of said pixels according to the formula:

with K: time constant of averaging d: division factor

Mi-i: average value at time Tι-ι of the video display means of the value Mi, for each of said pixels, and of means of storage of said value Mi, at the address of each pixel concerned, for constitution of the following image; the camera includes a control clock with a frequency greater than or equal to 14 MHz;

the means for acquiring the digital brightness value are arranged to digitize the information on eight bits;

the filtering and comparison means comprise two memory planes, namely a first memory plan

PI arranged to calculate and store the average image information in real time, called Signal information, for the elimination or reduction of the effect of the noise sources, and a second memory plane P2, of reference, arranged to memorize at a determined time, the image information also averaged, called reference or white signal information, delivered by the camera during the observation of a homogeneous scene, said filtering and comparison means being arranged to calculate the ratio, said Signal / White ratio, by comparison between the Signal information extracted from the memory PI and the White Signal information extracted from the memory P2; the comparison means comprise means for calculating the signal / white ratio arranged so that the value of the signal supplied by the plane PI, for a given address pixel, addresses the most significant bits of the address bus of a EPROM memory, and the value of the signal supplied by the P2 memory, for the pixel located at the same address of the reference plane, addresses the least significant bits of the address bus of said EPROM memory.

Such an arrangement allows particularly efficient operation.

The invention also provides a method for eliminating defects altering the quality of

1 image captured by a charge coupled camera, characterized in that each point of the image captured at time Xi is filtered in real time, by averaging with the image captured at time Tι-ι and on ^compares' .es averaged image information in real time with reference information to remove blackheads. In an advantageous embodiment, the video image is also integrated during a time sequence simultaneously with averaging.

Advantageously, the method comprises the following steps: - the digital value of brightness Xi__ι of each pixel of the image captured at an instant Tι-ι is acquired.

- we store each of these digital values Ii-i at determined addresses corresponding to each pixel, - we acquire the digital brightness value Xi of the pixels of the next image located at the same address, at time Xi, we calculate an average value of the numerical values of each of said pixels according to the formula:

with

K: time constant of averaging d: division factor

Mi-i: average value at time Ti-i - the value Mi is displayed on video for each of said pixels,

- And we store said value Mi, at the address of each pixel concerned, for constituting the following image.

Also advantageously:

- the average image in real time, called signal information, is continuously calculated and stored on a first PI memory plane to eliminate or reduce the effect of noise sources,

- the average image delivered by the camera is memorized during the observation of a homogeneous scene, called White Signal information on a second reference memory plane P2, - and the Signal / White ratio is calculated by comparison between the information of Signal extracted from the memory PI and the reference information known as of White Signal extracted from the memory P2. The comparison is made for example by dividing the digitalized digital value S of the signal extracted for a given pixel of the image, by the digital value B of the white signal extracted for a corresponding pixel of the reference image at saturation, this ratio S / B being multiplied by a constant, for example 255 for an eight-bit storage.

The invention will be better understood from the following description of a particular embodiment, given by way of non-limiting example. It refers to the accompanying drawings in which:

- Figure 1 shows a television device usable with a camera according to the invention. Figure 2 is a graphical representation illustrating the averaging used in the method 1 according to the invention.

- Figure 3 is a block diagram of the filtering and integration means of a camera according to the embodiment of the invention more particularly written here.

- Figure 4 is a time diagram showing normal operation and with integration function of a camera according to the invention.

Figure 1 shows a charge coupled camera 1 connected via a connection circuit 2 to a television monitor 3.

The camera 1 comprises a box 4 provided with means 5 for eliminating faults altering the quality of the captured image 6.

The means for eliminating faults 5 will be detailed with reference to FIG. 3.

They use a filter, the principle of which is detailed below with reference to FIG. 2.

This filter makes it possible to carry out an average weighted by an exponential law of the successive images.

This results, for still images, in an improvement in the signal to noise ratio. The calculation performed by the digital filter is as follows:

Each point of the image is the sum multiplied by the two constants of a fraction of the point or pixel provided by the video source and the point or pixel of the previous image, calculated by the same rule.

Let I _A brightness of a point provided by the video source, I _i _ ₁ that the same point in the previous image Ii _-2, etc.

Let the division factor of Ii be. Only the bits of the whole part are concerned with this.

Let] j_ be the multiplicative factor.

Let i- _! the brightness of the pixel contained in memory and calculated during the previous image.

This luminosity is therefore replaced by M _t after the calculation represented by the block diagram of FIG. 2 commented below.

In the embodiment more particularly described here, I _L and Mi are also digitized on eight bits.

The input 7 of the signal I _± attacks the divider 8 connected to the Sum block 9, which in turn attacks the operator 10 (multiplication by K) before being stored at 11.

The signal is taken up via 12 to re-attack the Sum block 9.

For each image, a value is thus stored (output 13)

either by posing

Mi = K xj + K ² xj - _! + K ³ x ji_ ₂ + ... The index response of the filter is therefore a discretized exponential, the time constant of the filtering being given (in number of images) by the formula τ = l / lK The index j corresponds to number of images used either: j = - In (255 / d) / In K The improvement of the signal / noise ratio is given by the formula

(1 + / 0

A = = 10 log (1 + K / 1-K) (dB) The values used in the invention are for example the following: d K τ j AA (dB)

Position 1 1 0.5 2 8 5

Position 2 2 0.666 3 12 7

Position 3 4 0.8 5 19 9.5

The elimination means 5 are shown more precisely in FIG. 3 and revolve around a double image memory 14.

The addressing of the memory 14 is generated by chains of digital counters 15 controlled by the HD, VD and pixel clock signals from the camera (not shown). More precisely, the horizontal addressing counters receive the pixel clock signal as a clock signal, the resetting of these counters being effected by the horizontal synchronization signal HD coming from the camera. The vertical addressing counters receive the HD signal as a clock signal, the resetting of these counters being effected by the frame parity signal generated from the signal VD coming from the camera. At the output of these counters a bus 16 of twenty bits simultaneously addresses each of the two memory planes PI and P2 by the same signals A0 to A19.

The video signal 17 is digitized by the Analog to Digital converter 18. In the embodiment more particularly described here, the video signal is digitized on eight bits.

This scanning depth does not constitute a limit, the embodiment being able for example to operate with a scanning on twelve, sixteen or twenty four bits. However, an eight-bit realization is particularly well suited to vision at very low light levels.

The signal, thus digitized, is structured into two identical data buses 19 called input buses

PI DE-0 to PI DE-7 for the one corresponding to the PI memory plane, and P2 DE-8 to P2 DE-15 for the P2 memory plane.

The data PI DE-0 to PI DE-7 are identical to those present on the bus P2 DE-8 to P2 DE-15.

The two memory plans PI and P2 are moreover organized in a similar manner, although having a different role.

The PI plane calculates and stores the average signal in real time for the elimination (or reduction) of the effect of noise sources.

The P2 plane stores the mean image, delivered by the camera when the latter observes a homogeneous scene. This sequence, called "reference travel", does not need to be repeated frequently but for example only after each cut of the power supply to the camera.

A battery can also be provided to save the data of the memory plane P2 to avoid regular taking of the reference plane.

The PI plane is therefore used for "real-time image" operation, that is to say that it operates permanently according to the following process: • Acquisition of the digital value of a pixel at a given time T0 via its input data bus (PI DE-0 to PI DE-7). • Search, in its own memory plane, for the value of this pixel memorized during the construction of the previous image (brightness Mi-i at the instant ^τ i-ι 'ie forty ms earlier with the CCIR standard). The corresponding value is then stored in a location characterized by its address obtained by reading the address bus (AO to A19)

^• Calculation of averaging

• Display of the result M thus obtained.

^• Storage of this new value M, at the address of the pixel concerned, so that it becomes the average value intended for carrying out the next averaging, which will take place during the production of the next image, ie here again as example for a CCIR standard camera, forty ms later.

The P2 plane is only used in acquisition when the reference image is taken. During normal operation of the device, only its output data bus is therefore active, the input data bus being inhibited.

The size of the memory planes PI and P2 is for example 756 x 576 x 8 bits, this configuration corresponding to the size of the CCD matrix used. The circuit portion 20 eliminating the other types of faults ensures the real-time calculation, pixel after pixel, at the rate of reading of these, of the Signal / White ratio.

To do this, the Signal information is extracted from the PI memory map at 21, while the White Signal information (corresponding to the white image taken as a reference) is extracted at 22 from the corresponding pixel, located at the same address on the memory map P2. A module 23 then calculates the ratio between the two values thus obtained and then directs it to the output stages of the device.

More precisely, concerning the comparison, it will be noted that before the memorization in the memorized plane P2 of the average image, when observing a homogeneous scene, the camera is adjusted so that the brightest regions of the image obtained are close to saturation. In other words, the digital values associated with these regions are therefore close to 255 for an 8-bit digitization.

For the other parts of the image, therefore corresponding to defects (black point, vignetting, etc.), the values are lower. This fact may be associated with mitigation.

The elimination of faults consists in locally providing a gain inversely proportional to this attenuation. Let B be the average value associated with a point or pixel of the camera and stored in the memory plane P2, following the previous adjustment procedure: the corresponding attenuation is B / 255, therefore the gain to be brought is 255 / B. Let S be the value provided by the camera for this same point, the comparison therefore consists in providing for the couple (S, B) the value: S x 255 / B.

The principle used, to carry out the calculation carried out by module 23, is for example the following:

The value of the signal provided by the PI memory plane

(for the record, this value corresponds to the Signal information), on its data bus, and for a given address pixel, is routed to the most significant bits of the address bus of a memory 24 275 kilogram EPROM 27512 Bits belonging to module 23.

The value of the signal supplied by the P2 memory plane

(for the record, this value corresponds to the White Signal information also called Reference), on its data bus, and for the pixel located at the same address on the "Reference" plane, is routed to the least significant bits of the bus. address of the same memory 24.

Thus the memory 24 is addressed by a pair of data corresponding to a single value which has been previously programmed as being the result of the Signal / White calculation.

This technique ensures the operation in real time of the whole. In order to obtain maximum performance, the white reference can be taken at a level close to saturation so as to make the best use of the dynamics of the Reference memory.

To do this, a circuit 25 is provided connected to the output of the memory plane P2.

The circuit is active during the Reference signal acquisition phase.

During this, the average image is displayed and available in real time in the P2 reference plane. A saturation detector is also provided, which has the role of presenting in black the pixels reaching saturation (supposed to be in white) in order to allow the operator to memorize this reference under the optimal conditions.

This module is inhibited during normal camera operation.

The output stages of the means 5 include a switch 26 for displaying the saturation as well as a converter Digital / Analog output 27, which delivers on its video output 28, a signal identical to that delivered by the camera, but free from all of the faults thanks to the invention. The output stages also include the stage 29 mixing synchronization signals 30 as well as an impedance adapter stage (not shown) making it possible to attack, at low impedance, the coaxial cable 31 intended to route the image to d '' other devices (monitor etc.)

In the embodiment described more particularly, the available signal is a composite video signal, according to the CCIR standard, compatible with direct viewing on a television monitor.

The assembly using digital techniques, it is also possible to provide a video signal in digital form, by means of an interface circuit suitable for the application. According to the embodiment of the invention more particularly described here, the camera also comprises means 32 for integrating the video signal, comprising a sequencer 33.

The sequencer 33 consists of a counting module 34 which generates a signal corresponding to a number of images and defined by the decoding of the value which is supplied for example by a group of encoder wheels 35. A monostable circuit could also have been be used to generate this signal which corresponds to the accumulation on the CCD.

With reference also to FIG. 4, it can be observed that during the time corresponding to the "accumulation" part (upper part of the figure) and the duration of which is a function of the available light flux, the CCD sensor (not shown) is blocked and therefore does not deliver a video signal 17 at the output.

This phase 35 in fact corresponds to the accumulation of photoelectrons in the image area of the CCD sensor.

The accumulation signal being the complement of the reading signal from the CCD sensor, this signal is delivered to the chain of counters 15 when the sequencer 33 triggers said reading. Which chain 15 is then authorized to address the memory PI.

Thus the averaging process of the image being read (instant T) with that stored in PI is carried out according to the normal process described above.

This is interrupted again as soon as a new accumulation cycle is triggered, again thanks to the action of the authorization signal from the counter chain 15. During reading, the CCD sensor provides a similar video signal. to that which it delivers in normal mode, the essential difference being that the duration of its presence is strictly limited to the reading time of the image zone of the CCD sensor. The cyclic sequence of these phases is carried out by the sequencer 33.

Claims

1. charge coupling camera (1) comprising means (5) for eliminating faults altering the quality of the captured image, characterized in that said elimination means are portable, integrated into the housing (4) of the camera and include:

- Electronic filtering means arranged to perform real-time filtering on each pixel of the image captured at the instant Xi, by averaging with the pixel located at the same address of the image captured at the instant Tι-ι .

- And electronic means of comparison between the average image in real time and a reference image taken at a determined time.

2. camera (1) according to claim 1, characterized in that it comprises means for accumulating the video image for a determined sequence time t before restoring the image, said accumulating means being arranged to operate simultaneously with the filtering and comparison means.

3. Camera according to either of Claims 1 and 2, characterized in that the filtering means comprise:

means for acquiring the digital brightness value Xi-_ι of each pixel of the image captured at an instant Xi-_ι,

means for storing these digital values Ii-i at determined addresses corresponding to each pixel,

means for acquiring the digital brightness value Xi of the pixels of the following image located at the same address, at time Xi, means for calculating an average value of the digital values of each of said pixels according to the formula:

with

K: time constant of averaging d: division factor

Mi-i: average value at time Ti-i

means of video display of the value Mi, for each of said pixels,

- And means for storing said value Mi, at the address of each pixel concerned, for constituting the following image.

4. Camera according to claim 3, characterized in that it comprises a control clock with a frequency greater than or equal to 14 MHz.

5. Camera according to any one of the preceding claims, characterized in that the means for acquiring the digital luminosity value are arranged to digitize the information on eight bits.

6. Camera according to any one of the preceding claims, characterized in that the filtering and comparison means comprise two memory planes, namely a first PI memory plan arranged to calculate and store the average image information in real time, said Signal information, for the elimination or reduction of the effect of the noise sources, and a second memory plane P2, of reference, arranged to memorize at a determined instant the image information also averaged, said reference information or White Signal, delivered by the camera during the observation of a homogeneous scene, said filtering and comparison means being arranged to calculate the ratio, called the Signal / White ratio, by comparison between the Signal information extracted from the memory PI and the reference information, extracted from the memory P2.

7. Camera according to any one of the preceding claims, characterized in that the comparison means comprise means for calculating the signal / white ratio arranged so that the value of the signal supplied by the plane PI, for a given address pixel , either routed to the most significant bits of the address bus of an EPROM memory, and the value of the signal supplied by the memory P2, for the pixel located at the same address of the reference plane, or routed to the bits of low weight of the address bus of said EPROM memory.

8. Method for eliminating faults altering the quality of the image captured by a charge-coupled camera, characterized in that each point of the image captured at time Xi is filtered in real time, by averaging with l image captured at the moment Tι-ι. and comparing the real-time averaged image information with reference information to eliminate black spots.

9. Method according to claim 8, characterized in that the video image is also integrated during a time sequence simultaneously with averaging.

10. Method according to any one of claims 8 and 9, characterized in that to filter: - the digital value of brightness Iι-ι of each pixel of the image captured at an instant Ti-i is acquired. each of these digital values Ii-i is stored at determined addresses corresponding to each pixel, - the digital value of brightness i of the pixels of the following image located at the same address is acquired, at time Xi, an average value of the digital values of each of said pixels is calculated according to the formula:

with

K: time constant of averaging d: division factor Mi-i: average value at time Ti-i we display the video value Mi, for each of said pixels, and store said value Mi, at the address of each pixel concerned, for constituting the following image.

11. Method according to any one of claims 8 to 10, characterized in that the digital brightness value is digitized on eight bits.

12. Method according to any one of claims 8 to 11, characterized in that

- the average image is continuously calculated and stored in real time on a first PI memory plane to eliminate or reduce the effect of noise sources,

- the average image delivered by the camera is memorized during the observation of a homogeneous scene, called the White signal scene on a second reference memory plane P2, - and the Signal / White ratio is calculated by comparison between the information d image extracted from memory Pi and the so-called White Signal reference information extracted from memory P2.

13. Method according to claim 12, characterized in that the ratio is calculated Signal / Blank by addressing the most significant bits of the address bus of an EPROM memory by the value of the signal supplied by the PI plane, for a given address pixel, and by addressing the least significant bits of the bus d address of said EPROM memory by the value of the signal supplied by memory P2, for the pixel located at the same address of the reference plane.