US20030052987A1 - Image sensor based optical defect correction - Google Patents
Image sensor based optical defect correction Download PDFInfo
- Publication number
- US20030052987A1 US20030052987A1 US10/282,734 US28273402A US2003052987A1 US 20030052987 A1 US20030052987 A1 US 20030052987A1 US 28273402 A US28273402 A US 28273402A US 2003052987 A1 US2003052987 A1 US 2003052987A1
- Authority
- US
- United States
- Prior art keywords
- lens
- correction
- optical defect
- digital
- imaging device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/4143—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a Personal Computer [PC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
Definitions
- the present invention relates to digital imaging devices, and in particular digital cameras for correcting optical defect effects.
- vignetting An example of one such optical defect is vignetting.
- the vignetting effect is when the brightness around the edges of a screen is less than the brightness in the center of the screen, creating a halo effect.
- Professional photographers will sometimes try to intentionally create this effect with an appropriate lens or screen, to produce a softened image at the edges of a picture. However, when this effect appears unintentionally in digital photographs, it is annoying.
- a variety of techniques have been developed to measure and address vignetting effects.
- U.S. Pat. No. 4,884,140 shows an analog circuit for providing vignetting compensation for a video camera using a zoom lens which causes vignetting.
- U.S. Pat. No. 5,434,904 shows measuring the vignetting effect for an x-ray examination apparatus by using a constant brightness image. A correction factor is then stored in the memory for each pixel.
- U.S. Pat. No. 5,576,797 shows the detection of vignetting effects in a camera with a focus detecting device.
- U.S. Pat. No. 5,381,174 shows the correcting of vignetting due to operation of a zoom lens by using a field frequency sawtooth signal.
- U.S. Pat. No. 4,816,663 shows the detection of vignetting between a photo taking lens and focus detecting optical system.
- optical defects include barrel distortion (makes straight lines appear curved), spherical aberration (causes an image that lacks contrast), chromatic aberration and astigmatism.
- the present invention provides a method and apparatus for constructing an inexpensive camera or other imaging device with optical defect correction in the camera or intelligent host electronic hardware, firmware, software or any combination thereof.
- the system is programmable through the use of a memory (e.g., one or more registers) for storing values corresponding to the amount optical defects of a particular lens or lens array used in the camera.
- the register values are provided to the various system correction mechanisms which implement algorithms which correct for the amount of optical defect such as vignetting of pixels.
- the correction mechanisms could be algorithms implemented in camera circuits, integrated circuits, microcontroller or microprocessor firmware or software, intelligent host circuits, integrated circuits, processor firmware or software, or any circuit, integrated circuit, microcontroller, microprocessor, processor, host or client combination thereof.
- the invention thus allows different lens suppliers or lens types to be used in cameras as they are manufactured, by simply programming different computed values into registers.
- the correction is applied on raw sensor data after it is converted into digital form, wherever in the system it is most appropriate for cost and performance optimization (such as but not limited to image quality or frame rate).
- the vignetting or other optical defect correction of the invention maintains the resolution of the image and allows real-time performance. It also makes more accurate statistics collection, thus making the automatic gain control (AGC) in the host, and the automatic white balance (AWB) in the host more accurate.
- AGC automatic gain control
- AVB automatic white balance
- corrections can be made, based on cost and performance. It may be desirable to do the correction in host software if there isn't time or horsepower in the camera, or to preserve a desired frame rate from the camera.
- the product form factor may dictate what corrections are done (such as but not limited to a low profile or wide angle lens, which can create greater distortions that need to be corrected).
- FIG. 1 is a drawing of a pixel array and brightness curve illustrating the vignetting effect.
- FIG. 2 is a block diagram of a camera and computer system with vignetting correction according to one embodiment of the invention.
- FIG. 3 is a block diagram of one embodiment of a hardware vignetting correction circuit according to the invention.
- FIG. 1 illustrates a pixel array 11 and a corresponding brightness curve 13 over the Y-axis.
- a central pixel 14 with value (x 0 , y 0 ) corresponds to the point of maximum brightness of the lens.
- a second example pixel 15 is near the minimum brightness, showing a vignetting effect of having the difference between curve 13 and a maximum brightness value 16 .
- the present invention corrects this vignetting effect in both X and Y directions.
- a threshold indicated by r T is illustrated within which the curve 13 is near the maximum value 16 , and no vignetting correction need be applied. Alternately, the correction can be applied to all of the pixels regardless to avoid any transition effects at the points r T .
- FIG. 2 illustrates a camera and computer system implementing the vignetting correction of the invention. Shown is a video camera 17 and host computer 18 .
- Video camera 17 includes a lens 10 and a photo sensor array, such as a CMOS or CCD array 12 , and is connected to a USB (Universal Serial Bus) 20 .
- USB Universal Serial Bus
- the signals from the CMOS chip are provided to an analog front end 40 which includes a differential amplifier or correlated double sampler (CDS) 42 , which provides the analog signals to an automatic gain control (AGC) circuit 44 .
- ADC automatic gain control
- ADC analog-to-digital converter
- the digitized signals are then provided directly to a digital processing circuit 48 . They are first provided to a video digital signal processor 50 which performs the compression, cropping, scaling and other functions on the data, as well as digital filtering.
- Bus interface 54 which includes a bus controller buffer 56 and a bus controller 58 .
- bus controller buffer 56 is capable of storing two USB frames at the maximum possible rate, or 2 Kbytes. Depending upon the frame rate, compression rate, etc., this may typically vary up to 10 lines, or even to 15-20 lines if decimation is done.
- Bus controller 58 provides a control signal on a line 60 to a timing generator 62 .
- Timing generator 62 provides clocking signals on line 64 to CMOS chip 12 .
- Clocking signals 64 include the horizontal and vertical transfer pulses.
- the analog front end 40 and the timing generator 62 are integrated onto the same semiconductor chip substrate 41 as CMOS array 12 .
- DSP unit 50 includes a vignetting correction circuit 70 according to the present invention.
- This circuit utilizes a constant value in a constant register 72 to perform the vignetting correction.
- its specified vignetting amount can be used to provide constant 72 .
- different lenses could be used in manufacturing a camera by simply programming a different constant value into register 72 . This allows multiple sources to be used for the lenses, and allows compensation for changes in lens quality from lot to lot or manufacturer to manufacturer.
- the vignetting correction is done before a statistics unit 74 collects statistics which are than transmitted to the host processor 18 . In addition, it is prior to any compression performed in bus interface 54 .
- host 18 On the host side of bus 20 , host 18 includes a color processing unit 76 which can perform AGC and AWB (automatic white balance) adjustments based on the statistics provided it. Since the vignetting correction is applied before the statistics are taken, this processing will be more accurate. In addition, the application of the vignetting correction prior to compression and transmission over the bus also improves the accuracy of the signal. After color processing, the signal is provided to a display 78 as shown.
- AGC automatic white balance
- the vignetting defect of a low quality lens may be modeled as pass through in the middle of the lens and a parabolic curve near the edge of the lens.
- the transform function is isotropic. Let p i and p o be the pixel value before and after vignetting correction, and x 0 and y 0 be the coordinate of the center of the lens (should be the center of the image if the lens is mounted properly). Then,
- a is a small positive constant. To be on the safe side, a maximum of 50% vignetting can be corrected, which means:
- FIG. 3 is a block diagram of one embodiment of a hardware system for implementing the equation described above.
- an optional selection circuit 80 is applied to the select input of a multiplexer 82 . Where the pixel position is less than the threshold r 2 , the input pixel pi on line 84 is simply passed through to the output pixel, p 0 , on line 86 . If it is greater than the threshold, a vignetting correction is applied by implementing the formula above with the hardware shown in FIG. 3.
- the current x value of the pixel, x i is applied to arithmetic circuit 88 , where the central pixel, x 0 , is subtracted from it. This value is then squared by applying its input twice, through a multiplexer 90 , to a multiply circuit 92 , where it is accumulated in an add/accumulator 94 .
- the y value y i is provided to a circuit 96 , where the difference from the center y value, y 0 , is determined.
- This is similarly applied as two values through mux 90 to a multiplier 92 , where it is squared and provided to add/accumulator 94 , where it is added to the squared x value.
- the use of the multiplexer simply allows, by time division multiplexing, the same multiplier 92 to be used for both calculations. Obviously, an alternative would be to provide two separate multiply units.
- the output of accumulator 94 is then provided to a second multiplier 96 , where it is multiplied by the constant value from register 72 .
- the value of one is then subtracted in a unit 98 , and this value is multiplied by the pixel value, pi, in a multiplier 100 .
- the output is provided on a line 102 through multiplexer 82 to output line 86 .
- variations of the circuitry could be provided, such as using the same multiply circuit for multipliers 92 , 96 and 100 , with the multiplier being used for different purposes in different time slots.
- Optional circuit 80 provides a comparator 104 which compares the output of add/accumulator 94 (the radius value of the x 2 +y 2 values) to the threshold radius in a register 106 .
- the constant a has the same number of bits as the pixel value, p i .
- p i is 10 bits
- the constant a would be 10 bits.
- a can be scaled by 1 ⁇ 4 if the sensor format is non-scaled QCIF (cropped). Thus, correction is provided where the image is cropped before the vignetting correction.
- correction can be done for the pixel aspect ratio by multiplying the y value by ⁇ fraction (11/12) ⁇ where a TV will be used.
- the correction will take place only outside a predefined diameter. This assumes that the lens performs satisfactorily around the center of the lens, which is often a reasonable assumption. By doing so, a more accurate correction model can be derived. Let r T be the diameter within which the vignetting defect can be ignored. Then,
- the present invention thus allows variation in lenses and also cheaper lenses to be used. It improves the AGC and AWB in the host computer. Additionally, it allows better object motion tracking. Object motion is usually done by assuming the luminance value is the same as the object moves to the edge of a sensor, and thus tracking by looking for the same luminance value. Obviously, vignetting effects can thwart the efforts to locate the object. By applying the vignetting correction close to the sensor, this can be overcome. The invention also provides overall improved video quality.
- a bus such as a universal serial bus (USB)
- USB universal serial bus
- Providing vignetting correction on the sensor side of the bus in the present invention provides the correction prior to compression to transmit over the bus.
- the camera can include an interface to the USB. Alternately, a bus interface to a memory card could be used.
- the lens used could be a multiple element lens, such as a lenticular array, which has a packed array of cylindrical (or spherical or other shape) lenslets.
- the optical defect corrected for could be barrel distortion (makes straight lines appear curved), spherical aberration (causes an image that lacks contrast), chromatic aberration or astigmatism. Accordingly, the foregoing description is intended to be illustrative, but not limiting, of the scope of the invention which is set forth in the following claims.
Abstract
A method and apparatus for constructing an imaging system with optics related image defect correction. These defects include but are not limited to vignetting, distortion, sharpness, etc. The optics related defect correction is implemented in either the optics device hardware, firmware or software, or the host device hardware, firmware or software or any hardware, firmware, software or host, client hardware, firmware, software combination thereof The optics related defect correction is used to optimize either image system cost, performance, form factor or any combination thereof.
Description
- This is a continuation-in-part application of U.S. application Ser. No. 09/345,167, filed Jun. 30, 1999, and entitled “Image Sensor Based Vignetting Correction”, which disclosure is incorporated herein by reference.
- The present invention relates to digital imaging devices, and in particular digital cameras for correcting optical defect effects.
- An example of one such optical defect is vignetting. The vignetting effect is when the brightness around the edges of a screen is less than the brightness in the center of the screen, creating a halo effect. Professional photographers will sometimes try to intentionally create this effect with an appropriate lens or screen, to produce a softened image at the edges of a picture. However, when this effect appears unintentionally in digital photographs, it is annoying. A variety of techniques have been developed to measure and address vignetting effects.
- U.S. Pat. No. 4,884,140 shows an analog circuit for providing vignetting compensation for a video camera using a zoom lens which causes vignetting.
- U.S. Pat. No. 5,434,904 shows measuring the vignetting effect for an x-ray examination apparatus by using a constant brightness image. A correction factor is then stored in the memory for each pixel.
- U.S. Pat. No. 5,576,797 shows the detection of vignetting effects in a camera with a focus detecting device.
- U.S. Pat. No. 5,381,174 shows the correcting of vignetting due to operation of a zoom lens by using a field frequency sawtooth signal.
- U.S. Pat. No. 4,816,663 shows the detection of vignetting between a photo taking lens and focus detecting optical system.
- Other optical defects include barrel distortion (makes straight lines appear curved), spherical aberration (causes an image that lacks contrast), chromatic aberration and astigmatism.
- Digital cameras for use with a personal computer for personal teleconferencing have become cheaper and cheaper. This puts pressure on the camera manufacturers to use cheaper lenses, which have more of a vignetting effect. In addition, the price pressure forces manufacturers to use fewer semiconductor chips, thus making the use of semiconductor memory to store vignetting corrections undesirable. Accordingly, there is a need for a digital camera which is inexpensive to manufacture and also corrects for vignetting of an inexpensive lens.
- The present invention provides a method and apparatus for constructing an inexpensive camera or other imaging device with optical defect correction in the camera or intelligent host electronic hardware, firmware, software or any combination thereof. The system is programmable through the use of a memory (e.g., one or more registers) for storing values corresponding to the amount optical defects of a particular lens or lens array used in the camera. The register values are provided to the various system correction mechanisms which implement algorithms which correct for the amount of optical defect such as vignetting of pixels. The correction mechanisms could be algorithms implemented in camera circuits, integrated circuits, microcontroller or microprocessor firmware or software, intelligent host circuits, integrated circuits, processor firmware or software, or any circuit, integrated circuit, microcontroller, microprocessor, processor, host or client combination thereof.
- The invention thus allows different lens suppliers or lens types to be used in cameras as they are manufactured, by simply programming different computed values into registers. The correction is applied on raw sensor data after it is converted into digital form, wherever in the system it is most appropriate for cost and performance optimization (such as but not limited to image quality or frame rate). Thus, if the data is corrected prior to the optional collection of statistics on the sensor data and the compression of the data for transmission to a host computer over a bus, the vignetting or other optical defect correction of the invention maintains the resolution of the image and allows real-time performance. It also makes more accurate statistics collection, thus making the automatic gain control (AGC) in the host, and the automatic white balance (AWB) in the host more accurate. Alternately, trade-offs can be made in where the correction is done, based on cost and performance. It may be desirable to do the correction in host software if there isn't time or horsepower in the camera, or to preserve a desired frame rate from the camera. The product form factor may dictate what corrections are done (such as but not limited to a low profile or wide angle lens, which can create greater distortions that need to be corrected).
- For a further understanding of the nature and advantages of the invention, reference should be made to the following specific description taken in conjunction with the accompanying drawings. For simplicity, this specific description outlines only vignetting and not other types of optical defect corrections and only a device hardware based implementation approach.
- FIG. 1 is a drawing of a pixel array and brightness curve illustrating the vignetting effect.
- FIG. 2 is a block diagram of a camera and computer system with vignetting correction according to one embodiment of the invention.
- FIG. 3 is a block diagram of one embodiment of a hardware vignetting correction circuit according to the invention.
- FIG. 1 illustrates a pixel array11 and a
corresponding brightness curve 13 over the Y-axis. Acentral pixel 14 with value (x0, y0) corresponds to the point of maximum brightness of the lens. Asecond example pixel 15 is near the minimum brightness, showing a vignetting effect of having the difference betweencurve 13 and amaximum brightness value 16. The present invention corrects this vignetting effect in both X and Y directions. - In one embodiment, a threshold indicated by rT is illustrated within which the
curve 13 is near themaximum value 16, and no vignetting correction need be applied. Alternately, the correction can be applied to all of the pixels regardless to avoid any transition effects at the points rT. - FIG. 2 illustrates a camera and computer system implementing the vignetting correction of the invention. Shown is a
video camera 17 andhost computer 18.Video camera 17 includes alens 10 and a photo sensor array, such as a CMOS orCCD array 12, and is connected to a USB (Universal Serial Bus) 20. - The signals from the CMOS chip are provided to an
analog front end 40 which includes a differential amplifier or correlated double sampler (CDS) 42, which provides the analog signals to an automatic gain control (AGC)circuit 44. The data is then provided to an analog-to-digital converter (ADC) 46. - The digitized signals, rather than being stored in a frame buffer memory as in the prior art, are then provided directly to a
digital processing circuit 48. They are first provided to a videodigital signal processor 50 which performs the compression, cropping, scaling and other functions on the data, as well as digital filtering. - Data from video DSP50 is provided to a
bus interface 54 which includes abus controller buffer 56 and abus controller 58. Preferably,bus controller buffer 56 is capable of storing two USB frames at the maximum possible rate, or 2 Kbytes. Depending upon the frame rate, compression rate, etc., this may typically vary up to 10 lines, or even to 15-20 lines if decimation is done. -
Bus controller 58 provides a control signal on aline 60 to a timing generator 62. Timing generator 62 provides clocking signals on line 64 toCMOS chip 12. Clocking signals 64 include the horizontal and vertical transfer pulses. In one embodiment, theanalog front end 40 and the timing generator 62 are integrated onto the samesemiconductor chip substrate 41 asCMOS array 12. -
DSP unit 50 includes avignetting correction circuit 70 according to the present invention. This circuit utilizes a constant value in aconstant register 72 to perform the vignetting correction. Depending upon theparticular lens 10 which is chosen, its specified vignetting amount can be used to provide constant 72. Thus, different lenses could be used in manufacturing a camera by simply programming a different constant value intoregister 72. This allows multiple sources to be used for the lenses, and allows compensation for changes in lens quality from lot to lot or manufacturer to manufacturer. - As can be seen, the vignetting correction is done before a
statistics unit 74 collects statistics which are than transmitted to thehost processor 18. In addition, it is prior to any compression performed inbus interface 54. - On the host side of
bus 20,host 18 includes a color processing unit 76 which can perform AGC and AWB (automatic white balance) adjustments based on the statistics provided it. Since the vignetting correction is applied before the statistics are taken, this processing will be more accurate. In addition, the application of the vignetting correction prior to compression and transmission over the bus also improves the accuracy of the signal. After color processing, the signal is provided to a display 78 as shown. - An embodiment of an equation for implementation and hardware to perform the vignetting correction is set forth below. The vignetting defect of a low quality lens may be modeled as pass through in the middle of the lens and a parabolic curve near the edge of the lens. The transform function is isotropic. Let pi and po be the pixel value before and after vignetting correction, and x0 and y0 be the coordinate of the center of the lens (should be the center of the image if the lens is mounted properly). Then,
- p o =p i*(a*r 2+1)
- =p i*(a*((x−x0)2+(y−y o)2)+1
- “a” is a small positive constant. To be on the safe side, a maximum of 50% vignetting can be corrected, which means:
- 1.0=0.5*(a*(176*176+144*144)+1)
- a=1.9338*10−5
- Normalized by 226, this will give a=1298. By limiting a to 0 through 1023, and performing the inverse computation, the maximum correctable vignetting is 56%.
- Note that in the formula given above, the pixel aspect ratio is not taken into account. For square pixel, the equation is exact, for 12/11 pixel aspect ratio, the result is slightly off. Also, a will be scaled by ¼ if the sensor format is non-scaled QCIF (subsampled QCIF from CIF by leaving out every other pixels in both horizontal and vertical directions).
- If yi is 10 bits, (x−x0)2 and (y−y0)2 is 16 bits (for CIF size), and y0 is 10 bits, this determines that a needs to be 10 bit (0 through 1023).
- FIG. 3 is a block diagram of one embodiment of a hardware system for implementing the equation described above. In particular, an optional selection circuit80 is applied to the select input of a
multiplexer 82. Where the pixel position is less than the threshold r2, the input pixel pi online 84 is simply passed through to the output pixel, p0, online 86. If it is greater than the threshold, a vignetting correction is applied by implementing the formula above with the hardware shown in FIG. 3. - In particular, the current x value of the pixel, xi, is applied to
arithmetic circuit 88, where the central pixel, x0, is subtracted from it. This value is then squared by applying its input twice, through a multiplexer 90, to a multiply circuit 92, where it is accumulated in an add/accumulator 94. Similarly, the y value yi is provided to acircuit 96, where the difference from the center y value, y0, is determined. This is similarly applied as two values through mux 90 to a multiplier 92, where it is squared and provided to add/accumulator 94, where it is added to the squared x value. The use of the multiplexer simply allows, by time division multiplexing, the same multiplier 92 to be used for both calculations. Obviously, an alternative would be to provide two separate multiply units. - The output of accumulator94 is then provided to a
second multiplier 96, where it is multiplied by the constant value fromregister 72. The value of one is then subtracted in a unit 98, and this value is multiplied by the pixel value, pi, in a multiplier 100. The output is provided on a line 102 throughmultiplexer 82 tooutput line 86. Obviously, variations of the circuitry could be provided, such as using the same multiply circuit formultipliers 92, 96 and 100, with the multiplier being used for different purposes in different time slots. - Optional circuit80 provides a
comparator 104 which compares the output of add/accumulator 94 (the radius value of the x2+y2 values) to the threshold radius in aregister 106. - Preferably, the constant a has the same number of bits as the pixel value, pi. Thus, if pi is 10 bits, while the x and y values are 16 bits, the constant a would be 10 bits. Also, a can be scaled by ¼ if the sensor format is non-scaled QCIF (cropped). Thus, correction is provided where the image is cropped before the vignetting correction.
- In addition, correction can be done for the pixel aspect ratio by multiplying the y value by {fraction (11/12)} where a TV will be used.
- In one embodiment, the correction will take place only outside a predefined diameter. This assumes that the lens performs satisfactorily around the center of the lens, which is often a reasonable assumption. By doing so, a more accurate correction model can be derived. Let rT be the diameter within which the vignetting defect can be ignored. Then,
- If r<=rT,
- po=pi,
- if r>=rT,
- p o =p i*(a*(r 2 −r T 2)+1)
- =p i*(a*(((x−x 0)2+(y−y 0)2)−((x T −x 0)2+(y T −y 0))+1)
- The present invention thus allows variation in lenses and also cheaper lenses to be used. It improves the AGC and AWB in the host computer. Additionally, it allows better object motion tracking. Object motion is usually done by assuming the luminance value is the same as the object moves to the edge of a sensor, and thus tracking by looking for the same luminance value. Obviously, vignetting effects can thwart the efforts to locate the object. By applying the vignetting correction close to the sensor, this can be overcome. The invention also provides overall improved video quality.
- Also, due to the need for image compression for transmitting over a bus, such as a universal serial bus (USB), having the host do vignetting corrections will result in less precision. Providing vignetting correction on the sensor side of the bus in the present invention provides the correction prior to compression to transmit over the bus. The camera can include an interface to the USB. Alternately, a bus interface to a memory card could be used.
- As will be understood by those of skill in the art, the present invention may be embodied in other specific forms without departing from the essential characteristics thereof. For example, the lens used could be a multiple element lens, such as a lenticular array, which has a packed array of cylindrical (or spherical or other shape) lenslets. Instead of vignetting, the optical defect corrected for could be barrel distortion (makes straight lines appear curved), spherical aberration (causes an image that lacks contrast), chromatic aberration or astigmatism. Accordingly, the foregoing description is intended to be illustrative, but not limiting, of the scope of the invention which is set forth in the following claims.
Claims (16)
1. A method for manufacturing a digital imaging device which connects to an intelligent host, comprising:
determining an optical defect effect of a first lens;
installing said first lens in a first camera housing;
determining an optical defect effect of a second lens;
installing said second lens in a second camera housing;
installing in each housing a photosensor array mounted behind said lens;
coupling an analog processing circuit to each said photosensor array;
coupling an analog to digital converter to each said analog processing circuit;
coupling a digital processing circuit to an output of each said analog to digital converter, said digital processing circuit including
a memory for storing a correction value corresponding to said optical defect effect,
a correction circuit, coupled to said memory, for adjusting a value of selected pixels according to a formula using said correction value, to correct for said optical defect effect;
coupling a bus interface circuit to an output of each said digital processing circuit for connecting to a bus for transfer of data to said intelligent host;
storing a first correction value in said memory in said first camera housing, said first correction value corresponding to said optical defect effect of said first lens; and
storing a second correction value in said memory in said second camera housing, said second correction value corresponding to said optical defect effect of said second lens.
2. The method of claim 1 wherein said first lens is a multiple element lens array.
3. The method of claim 1 wherein said imaging device is a camera.
4. The method of claim 1 wherein said optical defect is vignetting.
5. The method of claim 1 wherein said correction circuit does not adjust the value of pixels within a selected distance of a center of said lens.
6. The method of claim 1 wherein said correction circuit also corrects for a pixel aspect ratio.
7. The method of claim 1 wherein said correction value is a constant and has a number of bits equal to a number of bits used to represent a brightness of said selected pixels.
8. The method of claim 1 wherein said correction value is a constant and said correction circuit further comprises a scaling input for scaling said constant.
9. A digital imaging device for connecting to an intelligent host, comprising:
a lens;
a photosensor array mounted behind said lens;
an analog processing circuit coupled to said photosensor array;
an analog to digital converter coupled to said analog processing circuit;
a digital processing circuit coupled to an output of said analog to digital converter, said digital processing circuit including
a register for storing a correction value corresponding to an optical defect of said lens,
a correction circuit, coupled to said register, for adjusting a value of selected pixels according to a formula using said correction value, to correct for said optical defect of said lens; and
a bus interface circuit coupled to an output of said digital processing circuit for connecting to a bus for transfer of data to said intelligent host.
10. The digital imaging device of claim 9 wherein said correction circuit does not adjust the value of pixels within a selected distance of a center of said lens.
11. The digital imaging device of claim 9 wherein said correction circuit also corrects for a pixel aspect ratio.
12. The digital imaging device of claim 9 wherein said correction value has a number of bits equal to a number of bits used to represent a brightness of said selected pixels.
13. The digital imaging device of claim 9 wherein said correction value is a constant, and further comprising a scaling input to said correction circuit, for scaling said constant.
14. The digital imaging device of claim 9 wherein said lens is a multiple element lens.
15. The digital imaging device of claim 9 wherein said device is a camera.
16. The digital imaging device of claim 9 wherein said optical defect is vignetting.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/282,734 US20030052987A1 (en) | 1999-06-30 | 2002-10-28 | Image sensor based optical defect correction |
US10/931,318 US20050030383A1 (en) | 1999-06-30 | 2004-08-31 | Image sensor based vignetting correction |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/345,167 US6833862B1 (en) | 1999-06-30 | 1999-06-30 | Image sensor based vignetting correction |
US10/282,734 US20030052987A1 (en) | 1999-06-30 | 2002-10-28 | Image sensor based optical defect correction |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/345,167 Continuation-In-Part US6833862B1 (en) | 1999-06-30 | 1999-06-30 | Image sensor based vignetting correction |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/931,318 Continuation US20050030383A1 (en) | 1999-06-30 | 2004-08-31 | Image sensor based vignetting correction |
Publications (1)
Publication Number | Publication Date |
---|---|
US20030052987A1 true US20030052987A1 (en) | 2003-03-20 |
Family
ID=23353840
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/345,167 Expired - Lifetime US6833862B1 (en) | 1999-06-30 | 1999-06-30 | Image sensor based vignetting correction |
US10/282,734 Abandoned US20030052987A1 (en) | 1999-06-30 | 2002-10-28 | Image sensor based optical defect correction |
US10/931,318 Abandoned US20050030383A1 (en) | 1999-06-30 | 2004-08-31 | Image sensor based vignetting correction |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/345,167 Expired - Lifetime US6833862B1 (en) | 1999-06-30 | 1999-06-30 | Image sensor based vignetting correction |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/931,318 Abandoned US20050030383A1 (en) | 1999-06-30 | 2004-08-31 | Image sensor based vignetting correction |
Country Status (2)
Country | Link |
---|---|
US (3) | US6833862B1 (en) |
CN (1) | CN1147132C (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030234864A1 (en) * | 2002-06-20 | 2003-12-25 | Matherson Kevin J. | Method and apparatus for producing calibration data for a digital camera |
US6831687B1 (en) * | 1999-07-21 | 2004-12-14 | Nikon Corporation | Digital camera and image signal processing apparatus |
US20080284879A1 (en) * | 2007-05-18 | 2008-11-20 | Micron Technology, Inc. | Methods and apparatuses for vignetting correction in image signals |
US20090033788A1 (en) * | 2007-08-02 | 2009-02-05 | Micron Technology, Inc. | Integrated optical characteristic measurements in a cmos image sensor |
EP2091224A1 (en) * | 2004-06-07 | 2009-08-19 | Nokia Corporation | Method, apparatus, imaging module and program for improving image quality in a digital imaging device |
US20140042310A1 (en) * | 2011-04-25 | 2014-02-13 | Eduard BATKILIN | System and method for correction of vignetting effect in multi-camera flat panel x-ray detectors |
Families Citing this family (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7067473B1 (en) * | 1998-07-14 | 2006-06-27 | Janssen Pharmaceutica N.V. | Neurotrophic growth factor |
US6995794B2 (en) * | 1999-06-30 | 2006-02-07 | Logitech Europe S.A. | Video camera with major functions implemented in host software |
US6833862B1 (en) * | 1999-06-30 | 2004-12-21 | Logitech, Inc. | Image sensor based vignetting correction |
US7143294B1 (en) * | 1999-10-29 | 2006-11-28 | Broadcom Corporation | Apparatus and method for secure field upgradability with unpredictable ciphertext |
US6972790B2 (en) * | 2000-01-21 | 2005-12-06 | Psion Teklogix Systems Inc. | Host interface for imaging arrays |
FI109391B (en) * | 2000-12-08 | 2002-07-15 | Nokia Corp | Method and device for data transmission |
JP2002216136A (en) * | 2001-01-23 | 2002-08-02 | Sony Corp | Distance calculating method and imaging system |
JP2002237998A (en) * | 2001-02-07 | 2002-08-23 | Sony Corp | Image pattern correction method and imaging device |
JP3539394B2 (en) * | 2001-03-26 | 2004-07-07 | ミノルタ株式会社 | Image processing apparatus, program, and recording medium |
JP2003087653A (en) * | 2001-09-06 | 2003-03-20 | Ricoh Co Ltd | Imaging apparatus |
JP4357151B2 (en) * | 2002-03-28 | 2009-11-04 | 富士フイルム株式会社 | Digital camera and image data processing system |
US7502057B2 (en) * | 2002-06-20 | 2009-03-10 | Hewlett-Packard Development Company, L.P. | Method and apparatus for color non-uniformity correction in a digital camera |
US7233352B2 (en) * | 2002-06-20 | 2007-06-19 | Hewlett-Packard Development Company, L.P. | Method and apparatus for color non-uniformity correction in a digital camera |
US7391450B2 (en) | 2002-08-16 | 2008-06-24 | Zoran Corporation | Techniques for modifying image field data |
US7408576B2 (en) * | 2002-08-16 | 2008-08-05 | Zoran Corporation | Techniques for modifying image field data as a function of radius across the image field |
US7388610B2 (en) | 2002-08-16 | 2008-06-17 | Zoran Corporation | Techniques of modifying image field data by extrapolation |
JP3680827B2 (en) * | 2002-08-27 | 2005-08-10 | コニカミノルタフォトイメージング株式会社 | Digital camera |
KR100855957B1 (en) * | 2004-02-09 | 2008-09-02 | 삼성전자주식회사 | Solid state image sensing device compensating brightness of the side display area and driving method thereof |
EP1564681A1 (en) * | 2004-02-13 | 2005-08-17 | Dialog Semiconductor GmbH | Vignetting compensation algorithm |
US7609305B2 (en) * | 2004-04-06 | 2009-10-27 | Samsung Electronics Co., Ltd. | Methods and systems for anti shading correction in image sensors |
FI20045445A0 (en) * | 2004-11-18 | 2004-11-18 | Nokia Corp | A method, hardware, software, and arrangement for editing image data |
US7634152B2 (en) | 2005-03-07 | 2009-12-15 | Hewlett-Packard Development Company, L.P. | System and method for correcting image vignetting |
US7405816B2 (en) * | 2005-10-04 | 2008-07-29 | Nokia Corporation | Scalable test target and method for measurement of camera image quality |
WO2007042853A1 (en) * | 2005-10-13 | 2007-04-19 | Nokia Corporation | Method and system for vignetting elimination in digital image |
JP2009017020A (en) * | 2007-07-02 | 2009-01-22 | Nissan Motor Co Ltd | Image processor and method for generating display image |
US20090278977A1 (en) * | 2008-05-12 | 2009-11-12 | Jin Li | Method and apparatus providing pre-distorted solid state image sensors for lens distortion compensation |
US8384820B1 (en) | 2008-05-30 | 2013-02-26 | Logitech Europe S.A. | Real time video frame masking and processing for auto focus and other quality optimizations |
US8274583B2 (en) * | 2009-06-05 | 2012-09-25 | Apple Inc. | Radially-based chroma noise reduction for cameras |
US8284271B2 (en) * | 2009-06-05 | 2012-10-09 | Apple Inc. | Chroma noise reduction for cameras |
US9398279B2 (en) | 2010-03-29 | 2016-07-19 | Nokia Technologies Oy | Image sensor optimization |
US8571343B2 (en) | 2011-03-01 | 2013-10-29 | Sharp Laboratories Of America, Inc. | Methods and systems for document-image correction |
US8823841B2 (en) | 2012-06-20 | 2014-09-02 | Omnivision Technologies, Inc. | Method and apparatus for correcting for vignetting in an imaging system |
DE102013012810A1 (en) * | 2013-08-01 | 2015-02-05 | Connaught Electronics Ltd. | Method for activating and deactivating an image correction function, camera system and motor vehicle |
CN105843590B (en) * | 2016-04-08 | 2019-01-11 | 深圳航天科技创新研究院 | A kind of parallel instruction set pre-decode method and system running on CUDA platform |
Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4816663A (en) * | 1986-03-31 | 1989-03-28 | Nikon Corporation | Camera system with vignetting calculation and memory means |
US4884140A (en) * | 1987-11-18 | 1989-11-28 | Goldstar Co., Ltd. | Vignetting compensating circuit for a video camera |
US4979042A (en) * | 1989-05-30 | 1990-12-18 | Eastman Kodak Company | Apparatus for correcting shading effects in video images |
US5047861A (en) * | 1990-07-31 | 1991-09-10 | Eastman Kodak Company | Method and apparatus for pixel non-uniformity correction |
US5185883A (en) * | 1990-10-26 | 1993-02-09 | Data Translation, Inc. | System for locating failure signals by comparing input data with stored threshold value and storing failure addresses in alternating buffers |
US5343302A (en) * | 1991-11-15 | 1994-08-30 | Sony Corporation | Video camera with improved shading correction using clipped parabolic wave signal |
US5381174A (en) * | 1991-10-25 | 1995-01-10 | U.S. Philips Corporation | Method of and arrangement for correcting vignetting |
US5434902A (en) * | 1992-03-17 | 1995-07-18 | U.S. Philips Corporation | Imaging system with means for compensating vignetting and x-ray examination apparatus comprising such an imaging system |
US5563405A (en) * | 1995-04-28 | 1996-10-08 | Santa Barbara Research Center | Staring IR-FPA with on-FPA adaptive dynamic range control electronics |
US5576797A (en) * | 1983-10-19 | 1996-11-19 | Nikon Corporation | Camera with focus detecting device for removing vignetting effects |
US5606365A (en) * | 1995-03-28 | 1997-02-25 | Eastman Kodak Company | Interactive camera for network processing of captured images |
US5713053A (en) * | 1995-03-31 | 1998-01-27 | Asahi Kogaku Kogyo Kabushiki Kaisha | TTL exposure control apparatus in an interchangeable lens camera |
US5784100A (en) * | 1994-08-18 | 1998-07-21 | Fuji Photo Film Co. Ltd. | Apparatus and method for performing shading correction by multiplying a differential shading correction factor and input image data |
US5917549A (en) * | 1996-08-07 | 1999-06-29 | Adobe Systems Incorporated | Transforming images with different pixel aspect ratios |
US6055066A (en) * | 1995-02-28 | 2000-04-25 | Pioneer Electronic Corporation | System for correcting image data in an image scanner |
US6104430A (en) * | 1994-09-28 | 2000-08-15 | Ricoh Company, Ltd. | Digital electronic still camera which receives an input/output control program through a detachable communication interface card |
US6195469B1 (en) * | 1997-08-29 | 2001-02-27 | Fuji Photo Film Co., Ltd. | Image processing apparatus for shading correction |
US6538691B1 (en) * | 1999-01-21 | 2003-03-25 | Intel Corporation | Software correction of image distortion in digital cameras |
US6734911B1 (en) * | 1999-09-30 | 2004-05-11 | Koninklijke Philips Electronics N.V. | Tracking camera using a lens that generates both wide-angle and narrow-angle views |
US6833862B1 (en) * | 1999-06-30 | 2004-12-21 | Logitech, Inc. | Image sensor based vignetting correction |
US6853400B1 (en) * | 1998-06-16 | 2005-02-08 | Fuji Photo Film Co., Ltd. | System and method for correcting aberration of lenses through which images are projected |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6278480B1 (en) * | 1997-02-07 | 2001-08-21 | Canon Kabushiki Kaisha | Compound eye camera system |
-
1999
- 1999-06-30 US US09/345,167 patent/US6833862B1/en not_active Expired - Lifetime
-
2000
- 2000-06-30 CN CNB001199846A patent/CN1147132C/en not_active Expired - Fee Related
-
2002
- 2002-10-28 US US10/282,734 patent/US20030052987A1/en not_active Abandoned
-
2004
- 2004-08-31 US US10/931,318 patent/US20050030383A1/en not_active Abandoned
Patent Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5576797A (en) * | 1983-10-19 | 1996-11-19 | Nikon Corporation | Camera with focus detecting device for removing vignetting effects |
US4816663A (en) * | 1986-03-31 | 1989-03-28 | Nikon Corporation | Camera system with vignetting calculation and memory means |
US4884140A (en) * | 1987-11-18 | 1989-11-28 | Goldstar Co., Ltd. | Vignetting compensating circuit for a video camera |
US4979042A (en) * | 1989-05-30 | 1990-12-18 | Eastman Kodak Company | Apparatus for correcting shading effects in video images |
US5047861A (en) * | 1990-07-31 | 1991-09-10 | Eastman Kodak Company | Method and apparatus for pixel non-uniformity correction |
US5185883A (en) * | 1990-10-26 | 1993-02-09 | Data Translation, Inc. | System for locating failure signals by comparing input data with stored threshold value and storing failure addresses in alternating buffers |
US5381174A (en) * | 1991-10-25 | 1995-01-10 | U.S. Philips Corporation | Method of and arrangement for correcting vignetting |
US5343302A (en) * | 1991-11-15 | 1994-08-30 | Sony Corporation | Video camera with improved shading correction using clipped parabolic wave signal |
US5434902A (en) * | 1992-03-17 | 1995-07-18 | U.S. Philips Corporation | Imaging system with means for compensating vignetting and x-ray examination apparatus comprising such an imaging system |
US5784100A (en) * | 1994-08-18 | 1998-07-21 | Fuji Photo Film Co. Ltd. | Apparatus and method for performing shading correction by multiplying a differential shading correction factor and input image data |
US6104430A (en) * | 1994-09-28 | 2000-08-15 | Ricoh Company, Ltd. | Digital electronic still camera which receives an input/output control program through a detachable communication interface card |
US6055066A (en) * | 1995-02-28 | 2000-04-25 | Pioneer Electronic Corporation | System for correcting image data in an image scanner |
US5606365A (en) * | 1995-03-28 | 1997-02-25 | Eastman Kodak Company | Interactive camera for network processing of captured images |
US5713053A (en) * | 1995-03-31 | 1998-01-27 | Asahi Kogaku Kogyo Kabushiki Kaisha | TTL exposure control apparatus in an interchangeable lens camera |
US5563405A (en) * | 1995-04-28 | 1996-10-08 | Santa Barbara Research Center | Staring IR-FPA with on-FPA adaptive dynamic range control electronics |
US5917549A (en) * | 1996-08-07 | 1999-06-29 | Adobe Systems Incorporated | Transforming images with different pixel aspect ratios |
US6195469B1 (en) * | 1997-08-29 | 2001-02-27 | Fuji Photo Film Co., Ltd. | Image processing apparatus for shading correction |
US6853400B1 (en) * | 1998-06-16 | 2005-02-08 | Fuji Photo Film Co., Ltd. | System and method for correcting aberration of lenses through which images are projected |
US6538691B1 (en) * | 1999-01-21 | 2003-03-25 | Intel Corporation | Software correction of image distortion in digital cameras |
US6833862B1 (en) * | 1999-06-30 | 2004-12-21 | Logitech, Inc. | Image sensor based vignetting correction |
US6734911B1 (en) * | 1999-09-30 | 2004-05-11 | Koninklijke Philips Electronics N.V. | Tracking camera using a lens that generates both wide-angle and narrow-angle views |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6831687B1 (en) * | 1999-07-21 | 2004-12-14 | Nikon Corporation | Digital camera and image signal processing apparatus |
US20030234864A1 (en) * | 2002-06-20 | 2003-12-25 | Matherson Kevin J. | Method and apparatus for producing calibration data for a digital camera |
US7151560B2 (en) * | 2002-06-20 | 2006-12-19 | Hewlett-Packard Development Company, L.P. | Method and apparatus for producing calibration data for a digital camera |
EP2091224A1 (en) * | 2004-06-07 | 2009-08-19 | Nokia Corporation | Method, apparatus, imaging module and program for improving image quality in a digital imaging device |
US20080284879A1 (en) * | 2007-05-18 | 2008-11-20 | Micron Technology, Inc. | Methods and apparatuses for vignetting correction in image signals |
US7920171B2 (en) | 2007-05-18 | 2011-04-05 | Aptina Imaging Corporation | Methods and apparatuses for vignetting correction in image signals |
US20090033788A1 (en) * | 2007-08-02 | 2009-02-05 | Micron Technology, Inc. | Integrated optical characteristic measurements in a cmos image sensor |
US8085391B2 (en) | 2007-08-02 | 2011-12-27 | Aptina Imaging Corporation | Integrated optical characteristic measurements in a CMOS image sensor |
US20140042310A1 (en) * | 2011-04-25 | 2014-02-13 | Eduard BATKILIN | System and method for correction of vignetting effect in multi-camera flat panel x-ray detectors |
US9360571B2 (en) * | 2011-04-25 | 2016-06-07 | Generic Imaging Ltd. | System and method for correction of vignetting effect in multi-camera flat panel x-ray detectors |
Also Published As
Publication number | Publication date |
---|---|
CN1147132C (en) | 2004-04-21 |
US6833862B1 (en) | 2004-12-21 |
US20050030383A1 (en) | 2005-02-10 |
CN1292622A (en) | 2001-04-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20030052987A1 (en) | Image sensor based optical defect correction | |
EP3624439B1 (en) | Imaging processing method for camera module in night scene, electronic device and storage medium | |
US6982756B2 (en) | Digital camera, image signal processing method and recording medium for the same | |
US9230306B2 (en) | System for reducing depth of field with digital image processing | |
JP4885179B2 (en) | Image distortion correction method and image processing apparatus employing the correction method | |
US6940550B2 (en) | Digital camera and a method for correcting dust images in the camera | |
US7494293B2 (en) | Digital single-lens reflex camera | |
US8941744B2 (en) | Image sensors for establishing an image sharpness value | |
JP4872797B2 (en) | Imaging apparatus, imaging method, and imaging program | |
US7671892B2 (en) | Image sensing apparatus, and control method, program, and storage medium of image sensing apparatus | |
US9071751B2 (en) | Image processor method and program for correcting distance distortion in panorama images | |
WO2021047345A1 (en) | Image noise reduction method and apparatus, and storage medium and electronic device | |
US9589339B2 (en) | Image processing apparatus and control method therefor | |
JP2018107526A (en) | Image processing device, imaging apparatus, image processing method and computer program | |
CN109697737B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
JP3937924B2 (en) | Imaging apparatus and control method thereof | |
KR20170011555A (en) | Digital photographing apparatus and the method for the same | |
JP2003169255A (en) | Shading correction method | |
US8331722B2 (en) | Methods, apparatuses and systems providing pixel value adjustment for images produced by a camera having multiple optical states | |
JP4427701B2 (en) | Camera device and camera function adjustment method | |
KR101797080B1 (en) | Camera module and method for correcting lens shading thereof | |
US20070147819A1 (en) | System and method for indicating sharpness of images | |
JP5182395B2 (en) | Imaging apparatus, imaging method, and imaging program | |
US20040174456A1 (en) | Image sensing apparatus, autofocus method, program, and storage medium | |
KR20230027576A (en) | Imaging Device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: LOGITECH EUROPE S.A., SWITZERLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LI, WEI;BILLERBECK, BRYED;REEL/FRAME:013454/0632;SIGNING DATES FROM 20020925 TO 20021023 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |