WO1999009735A1

WO1999009735A1 - Color channel independent gain for solid state image sensor

Info

Publication number: WO1999009735A1
Application number: PCT/US1997/021226
Authority: WO
Inventors: Warren Severin; Werner Metz
Original assignee: Intel Corporation
Priority date: 1997-08-15
Filing date: 1997-11-17
Publication date: 1999-02-25
Also published as: AU2243299A; TW361044B

Abstract

An improved color image sensor semiconductor integrated circuit (200) where the analog light-generated sensor signals from photosites or pixels of a particular color (111, 112, 121, 122) are amplified independently (120) of signals representing another color, prior to A/D conversion (130). The embodiments aim to improve image quality under low or uneven illumination conditions by reducing quantization noise and thereby improving the signal-to-noise ratio in the color channels having lower illumination. Also, the embodiments may be used to compensate for some manufacturing variation in the sensor IC (410), in the Color Filter Array (CFA), and in the optical interface and component manufacturing processes. The invention may be particularly useful in portable digital image capture systems, such as digital cameras, but may also find use in color scanners and certain color copiers.

Description

COLOR CHANNEL INDEPENDENT GAIN FOR SOLID STATE

IMAGE SENSOR

BACKGROUND INFORMATION

This invention relates to semiconductor integrated circuit techniques for sensing images in color.

Producing high quality images in color has become increasingly desirable for electronic still image capture and video systems. Conventional methods of sensing and producing an image in color include the use of electronic charged coupled device (CCD) or complimentary metal oxide semiconductor (CMOS) sensor arrays exposed to incident light. The sensor array is an array of photosites or pixels, each photosite loosely defined as a region containing photodetecting, processing, and readout circuitry. In the CMOS sensor arrays, the photodetecting circuitry may include photodiodes or photogates. In some cases, a color filter array (CFA) is pasted or otherwise processed over the array of photodetecting circuits, such that each photodetecting portion is covered by a bandpass optical filter that lets light of a particular color pass into and thus be detected by the corresponding circuit. Sensor arrays typically have two or more colors distributed evenly, or according to other schemes, in the array of photosites, where a group of photosites may be assigned a particular color.

The myriad of photosites in a manufactured sensor array may include differences between the photodetecting circuitry and bandpass filters of the CFA that may cause an undesirable imbalance among the sensor signals. Thus, it is desirable to realize a color image capture system that can somewhat compensate for the imbalance caused by manufacturing variations.

In the conventional imaging system, an analog-to-digital (A/D) converter may be coupled to the sensor array to convert the analog, light- generated, sensor or pixel signals into sensor signals having digital format. Such digitized sensor signals, also referred to as color pixel data, can then be processed using many known digital signal processing techniques to yield the desired image data. The A/D converter and advanced digital storage and processing techniques may be readily implemented in modern large scale integrated circuits. Indeed, a completely "digital" or software solution to many electronic systems is often sought by modern designers.

In the imaging arena, software solutions such as image processing algorithms performed by programmed processors have been employed to reduce noise in the color pixel data. But such a software solution may introduce undesirable delays before the final image can be viewed. Moreover, the results of such software corrections are not always predictable or consistent. Therefore, a hardware approach may be desirable to yield somewhat more consistent and predictable reductions in the noise content of color images.

SUMMARY

This invention is directed at a circuit having photosites of different colors that yield light-generated signals, and that includes gain circuitry configured to amplify the light generated signals that correspond to a particular color, independent of amplifying the light-generated signals that correspond to another color.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the invention may be better understood by referring to the Figures, description, and claims below, where:

Figure 1 illustrates an embodiment of the invention as a sensor circuit. Figure 2 shows a sensor circuit configurable with independent color channel gain according to another embodiment of the invention.

Figure 3 is a sensor circuit with independent color channel programmable gain according to yet another embodiment of the invention.

Figures 4 shows the effect on color channel quantization noise of using an embodiment of the invention.

Figure 5 is an imaging system embodiment of the invention.

DETAILED DESCRIPTION

As described in detail below, the embodiments of the invention are image sensor circuits and imaging systems which may be used to support still and video imaging. For purposes of explanation, specific embodiments are set forth below to provide a thorough understanding of the invention. However, as understood by one skilled in art, from reading this disclosure, the invention may be practiced without such details. Furthermore, well-known elements, devices, process steps, and the like, are not set forth in detail in order to avoid obscuring the invention.

Figure 1 illustrates a block diagram of an image sensor circuit 100 according to one embodiment of the invention. The circuit 100 has a sensor array of pixels or photosites 111, 112, ..., arranged in rows and columns for that particular embodiment. A CFA (not shown) may be pasted or otherwise processed over the photodetecting circuitry in the sensor array such that each photosite is in effect configured to detect incident light of a particular color, as indicated by the color label adjacent each photosite.

An exemplary photosite or pixel 111 may have a blue filter, and may include a row select input and an analog output. Not shown in each photosite is conventional read out circuitry which couples a photodetecting element responsive to incident light. The readout circuitry allows the formation and reading of a light-generated analog signal which is provided at the analog output.

In the circuit 100, each photosite is coupled via its analog output to an amplifier such as amplifier 120. The gain of the amplifier 120 may be set using one or more gain control lines 160. Amplifiers that couple photosites of the same color

receive the same gain setting, by being connected to the same color gain control lines. Thus, the gain for each color channel may be set independently of the other color channels.

The embodiment of the invention as circuit 100 also features a column Analog-to-Digital (A/D) converter 130, where the light-generated sensor signals from a given column of photosites are multiplexed and converted by the column A/D converter into digital format, one row at a time. Another scheme may use an A/D converter for each row, so that the digitized sensor signals may be read per column. In either case, the digital sensor signals may then be subsequently stored and further processed as described below.

The sensor array of circuit 100 employs one particular distribution of colors in the CFA. However, the CFA can take on different embodiments, with the colors being distributed across the sensor array according to other patterns. For example, Figure 2 illustrates another embodiment of the invention as circuit 200 wherein the CFA has alternating columns of a particular color. In this case, the gain circuitry includes a single amplifier 220, 222, or 224 for each column. Once again, a row select signal is used to enable a particular photosite output in each column and couple an analog output to the amplifiers 220, 222, and 224. Alternatively, the sensor array may be configured with column select lines, the amplifier 220 being coupled to a row of multiplexed analog outputs. In the latter scenario, digitized light-generated sensor signals may be read per column, and further processed and stored as indicated below.

Although both Figures 1 and 2 show the sensor circuit having three colors red, green and blue, the sensor circuit can also be configured with other combinations of two or more colors in its CFA, there being different colors available other than red, green, and blue, such as cyan, yellow, and magenta.

Figure 3 shows another embodiment of the invention as sensor circuit 300. In this case, the gain circuitry includes the amplifier 320. The analog light-generated sensor signals of the sensor array are multiplexed to the input of the amplifier 320 prior to being converted by A/D converter 330. The multiplexer circuitry in this embodiment includes analog switches 310, 314, 318, and analog multiplexer 324. Multiplexer 324 receives a color or RGB select signal that specifies a particular color channel to be read. The RGB select signal and the gain control signal to the amplifier 320 are keyed so as to properly capture and amplify the analog light-generated sensor signals for the selected channel. The switch signals R_Gn' Gon_' ^anc Bon for each color channel, received by the switches 310, 314, and 318, respectively, are used to enable a path for sensor signals of a particular color channel to reach the multiplexer 324. After turning on the proper switch, row and/or column select signals (as described above for the embodiments in circuits 100 and 200) may then be cycled through in order to read the signals for the corresponding color channel. Other multiplexing schemes may be employed while still maintaining independent gain control for each color channel.

In the embodiments of the invention described above, the gains for the color channels may be related to the color spectrum of the illuminant for each exposure. One technique for obtaining the color spectrum of the illuminant, and thus the relative color channel gain ratios, is briefly described below. First, the gain for a desired color channel amplifier is let to be proportional to the ratio of the measured light energy for a base color to the measured light energy of the desired color during a given exposure time. The sensor array is exposed to an object having uniform reflectivity across all colors (e.g., a 20% gray card) for an initial exposure time which is the same for all photosites. Next, the measured light energy for each color is obtained by, for example, measuring the output of a number of photosites having the same color filter.

One color is then selected as the base color, such as the color with highest intensity. Thus, if the gain for the red channel is set to 1 unit, then the gain G_x for another color X is given by G_x = Gred * Redmeas/Xmeas where Redmeas and Xmeas are proportional to measured light-generated signals for the base color Red and desired color X, respectively. The measured energy levels for each color are obtained using the same exposure time, but because the illuminant may have non-uniform intensity over the different colors, the measured energy levels for different colors may be different. For this example, since the intensity of the illuminant is highest at red, the gain values for blue and green would be larger than for red, in proportion to the ratios described above.

Another technique for obtaining the spectral ratios uses a histogram of digital image data received from the sensor circuit. The needed information about the spectral content of the illuminant may be obtained after exposure of the sensor circuit, uniform in time across the different colors. This should yield for a given detected value (digital) the total number of photosites of the same color which detected that value. The histogram will thus present a distribution of photosites versus color energy, for a given scene and illuminant. By selecting a statistic, such as 95% of the cumulative distribution of pixels, the digital values for the different colors can be used to define a ratio that will be multiplied by a gain value to give the desired gain for a particular color. Other techniques may be used by those skilled in the art to obtain the proper gain values for the different color channels to equalize the analog light- generated signals.

The above embodiments illustrate the invention as a sensor circuit. The invention may also be implemented as an imaging or image capture system, as illustrated in Figure 4. The system includes an optical interface that directs light reflected from an optical subject to a sensor circuit 410. The sensor circuit 410 may include a color sensor array as in any one of the circuits 100, 200, and 300 described earlier. The sensor circuit 410 receives a number of independent gain control signals (one per color) from a control unit 414. The control unit 414 receives data including gain values for each color channel from a color spectrum measurement unit 418. The control unit 414 may be implemented as a hard- ired logic circuit, or as a programmed processor with a suitable I/O peripheral, and may or may not be located in the same IC containing the sensor circuit 410.

The gain values received from unit 418 are used by the control unit 414 to define the gain control signals suitable for the particular amplifiers used in the sensor circuit 410. The gain values, based on the color temperature of the illuminant, may be automatically determined by the color spectrum measurement unit according to any of the techniques described above. Alternatively, the gain values may be set manually by the user. The gain values and control signals for each color may be automatically computed and provided, either by hardwired logic circuits or perhaps from an I O peripheral of a programmed processor, in one or both of the measurement unit 418 and control unit 414.

The embodiment shown in Figure 4 also provides that sensor signals from the sensor circuit 410 be transferred to signal processing unit 412. In one embodiment, the sensor signals may be in analog form to be converted into digital form by an A/D converter in the signal processing unit 412. Alternatively, the A/D conversion units may be part of sensor circuit 410 so that the sensor signals passed to unit 412 are digital. In either case, the A/D conversion units may but need not be located on the same IC as the sensor array.

In addition, the unit 412 may be configured to perform digital image processing such as noise suppression and color space conversion. Such digital processing by unit 412 may be performed by a programmed processor or by dedicated hardwired logic circuits.

After image data has been prepared by unit 412, the data may be stored in a data storage 416 which may be any conceivable type of storage device suitable for storing digital image data. Modern examples include a non- volatile random access memory and a rotating media device such as magnetic and/or optical disk storage. A data link interface 420 permits the image data to be transferred outside the image capture system. For example, the data link interface 420 may include a serial communications interface unit, such as a transceiver, to link a desktop computer.

Some of the advantages of the different embodiments of the invention may be illustrated by Figure 5. Figure 5 illustrates quantization noise problems introduced as a result of the A/D conversion process and aggravated by a non-uniform illuminant, and a solution using the embodiment of the invention for minimizing such noise in the weaker color channels. The quantization noise here is defined as a percentage of the full scale digital output that is available for a given illuminant. The quantization noise is introduced by the A/D conversion process and is approximated by ±1 least significant bit (LSB). The quantization noise for each color channel of an imaging system implemented without and with the invention are compared for exemplary illuminant levels, exemplary colors, and exemplary digitized values. The relevant characteristics of the detected illuminant in analog and digital form are given for an exemplary configuration, where the analog value in millivolts (mV) may appear at the output of a photosite. A typical noise level for such an analog output may be approximately 1% of the maximum available analog output, as illustrated in Table 5-1. Thus, under this non-ideal illuminance condition, red photosites may present an analog output in the range 0 to 100 mV plus or minus 1%, green photosites will provide 0 to 50 mV plus or minus 1% and blue photosites may provide 0 to 20 mV plus or minus 1%. Note that under idealized illumination, such as bright daylight, photosite output across all colors would be up to 100 mV plus or minus 1%.

The analog output of the photosites is then converted into digital format using classic A/D conversion techniques, resulting in the digitized sensor signals of Table 5-2. For example, an input analog level of 0-100 mV may be translated to an 8-bit digital signal having a range 0-263. Under this scenario, the red digital channel may be assigned the range 0-200 and will feature a quantization noise of 1 LSB, equivalent to plus or minus 0.5% of the available full scale value 200. The green channel, however, will provide a digital range of only 0-100 due to the weaker illuminant, with a quantization noise of 1 LSB being equivalent to plus or minus 1% of the available full scale value 100. Finally, the blue channel (the weakest) will provide a digital output of only 0- 40, with a quantization noise of 1 LSB being 2.5% of the available full scale value 40.

The quantization noise introduced by the A/D conversion process for the red channel may be acceptable as one-half of the original noise present in the analog output. However, for the weaker illuminant blue channel, the quantization noise is an undesirable 2.5 times the original analog noise. Such an increase in the noise level may manifest itself as noticeably degraded image quality in the captured digital image. Table 5-3 shows the effect of independent color channel gain using the embodiments of the invention that lessens the quantization noise levels for the weaker color channels. By properly scaling up the signals in the weaker green and blue analog signals prior to A/D conversion, the quantization noise (as 1 LSB) for all three color channels may be limited to 0.5% of the newly available full scale digital value of 200. Thus, if the noise contribution by the independent gain stages is negligible compared to the quantization noise values described above, the overall noise figure of an imaging system according to an embodiment of the invention will remain acceptable for the stronger color channels, and may actually improve for the weaker color channels.

Other advantages of the embodiments of the invention include increased dynamic range for the weaker color channels. By using independent analog gain for each color channel prior to A/D conversion as in the embodiments of the invention described above, the analog signal level for the weaker channel may be increased prior to A/D conversion, thus allowing greater dynamic range in the weaker channel. This effect is illustrated in Tables 5-2 (digitized signals without independent color gain) and 5-3 (digitized signals with independent color gain) as increased full scale digital range for the weaker green and blue channels.

The embodiments of the invention described above may also be used to assist in correcting for manufacturing variations which induce a type of imbalance in the different color channels. For example, the optical filters for a given color channel may have unequal bandpass properties due to differences in the CFA between production batches. Also, the optical components of the imaging system may present non-uniformity across the various colors. Such variations add to the imbalance between the color channels that is created by illuminants having non-uniform intensity. The independent color channel gain control of the various embodiments of the invention helps to equalize the light- generated analog signals between the different color channels, and therefore also help reduce the effects of the manufacturing variations. Another advantageous feature of the circuits 100 and 200 appears when the sensor array is implemented as a single chip. This allows simultaneous rather than sequential duration exposures for all colors, so that the imaging system which incorporates the single chip sensor array may yield higher quality color images of a moving scene.

To summarize, the embodiments of the invention described above present the design of an improved and novel color image sensor circuit that features independent gain control for each color channel. Of course, the embodiments of the invention described above are subject to other variations in structure and implementation. For example, the different sensor circuits of the invention described above may be configured with particular types of photodetecting circuits and readout circuitry such that the amplifiers 120, 220, and 320 may receive as inputs either a voltage, charge, or current. The independent color channel amplifiers of the different embodiments may be configured as transconductance (voltage to current) or transimpedance (current to voltage) amplifiers, depending on the particular system requirements. Also, the amplifiers may be configured to provide discretely or continuously variable gain. For example, one or more of the amplifiers 120, 220, and 320 may be programmable gain amplifiers (PGA) that receive a digital gain control signal and allow the gain to be dynamically changeable by one of several discrete factors .

Also, semiconductor IC fabrication techniques other than standard CMOS may be used to implement the different embodiments of the invention. Thus, the details above should be interpreted as illustrative and not in a limiting sense.

Claims

CLAIMSWhat is claimed is:

1. A circuit comprising: a first plurality of photosites having a first plurality of output lines for providing a first plurality of light-generated signals, each of said first plurality of photosites having an optical filter of a first color; a second plurality of photosites having a second plurality of output lines for providing a second plurality of light-generated signals, each of said second pluraUty of photosites having an optical filter of a second color; and, gain circuitry coupled to the first and second plurality of output lines and configured to amplify said first plurality of light-generated signals independently of said second plurality of light-generated signals.

2. A circuit as in claim 1 wherein the gain circuitry is further configured to amplify the first plurality of signals by one of a plurality of programmable factors.

3. A circuit as in claim 1 wherein the first and second plurality of photosites are arranged to define first and second columns, respectively.

4. A circuit as in claim 1 further comprising: a third plurality of photosites having a third plurality of output lines for providing a third plurality of light-generated signals, each of said third plurality of photosites having an optical filter of a third color, wherein said gain circuitry is further coupled to said third plurality of output lines and further configured to amplify said third plurality of Ught- generated signals independendy of said first and second plurality of light- generated signals.

5. A circuit as in claim 4 wherein the first color is red, the second color is green, and the third color is blue.

6. A circuit as in claim 1 wherein the gain circuitry comprises first and second amplifiers each coupled to one of said first and second plurality of output lines, respectively, to amplify one of said first and second plurality of light-generated signals, respectively.

7. A circuit as in claim 1 wherein the gain circuitry comprises analog multiplexer circuitry for receiving said first and second plurality of light- generated signals and providing a selected one of said light-generated signals in response to receiving selection signals; an amplifier coupled to the analog multiplexer circuitry for amplifying the selected one of said light-generated signals.

8. A circuit as in claim 7 wherein the analog multiplexer circuitry includes a three-to-one analog multiplexer.

9. A circuit as in claim 7 wherein the amplifier is further configured to amplify the selected one of said plurality of Ught-generated signals in accordance with one of a plurality of programmable factors.

10. An imaging system comprising: optical interface; sensor circuit having a plurality of sets of gain control Unes for receiving a pluraUty of gain control signals, each set of gain control lines for a different color channel; signal processing unit configured to receive sensor signals from the sensor circuit and in response provide digital image data; data storage means for storing the digital image data; and, control unit for generating the plurality of gain control signals in response to receiving gain values.

11. An imaging system as in claim 10 further comprising means for determining a gain value in response to a color spectrum of an illuminant.

12. An imaging system as in claim 10 further comprising data link means for transferring the digital image data.

13. An imaging system as in claim 10 wherein the data storage means includes a non-volatile random access memory.

14. An imaging system as in claim 11 wherein the means for determining a gain value includes a programmed processor.

15. An imaging system as in claim 12 wherein the data link means includes a serial communications interface unit.