WO2000010116A1

WO2000010116A1 - Method and system for eliminating unwanted shadows on a subject in a facial recognition system

Info

Publication number: WO2000010116A1
Application number: PCT/US1999/018542
Authority: WO
Inventors: Francis J. Cusack, Jr.; Dennis C. Ehn; Jay Bortolussi
Original assignee: Lau Technologies
Priority date: 1998-08-13
Filing date: 1999-08-13
Publication date: 2000-02-24
Also published as: AU5563299A

Abstract

A subject is illuminated by infrared light from an infrared light source. The subject may also be illuminated by visible light from ambient light sources. The infrared light helps to fully illuminate the subject so as to eliminate shadows on the face of the subject. A facial image of the subject is obtained and passed through a filter. The filter removes wavelengths of light in the visible range of the spectrum. The resulting image is digitized and passed to a facial recognition system. The filtered and digitized image is used to attempt to recognize the subject based on upon the facial image.

Description

METHOD AND SYSTEM FOR ELIMINATING UNWANTED SHADOWS ON A SUBJECT IN A FACIAL RECOGNITION SYSTEM

Technical Field

The present invention relates generally to image acquisition systems and more particularly to a method and system for eliminating shadows on a subject in a facial recognition system.

Background of the Invention

Facial recognition systems attempt to identify a party based upon a facial image of the party. The facial image is typically obtained by a video camera that images the party and the facial image obtained by the video camera is compared with a database of facial images to determine if there are any matching facial images in the database. Identity information is stored along with the faces in the database. As such, once a match is found, the stored information may be accessed to retrieve the identity information and determine the identity of the subject.

One difficulty with such conventional facial recognition systems relates to the acquisition of the facial image of the party. Lighting conditions may vary dramatically such that the lighting present when the facial image was obtained differs substantially from the lighting that was present when a corresponding image of the party stored in the database was obtained. One particular area of concern relates to shadows. Often the ambient light is poorly positioned and/or insufficient such that shadows appear on a party's face. These shadows may result in improper matching of the party with facial images in the database. One solution has been to brightly illuminate the party; hence, removing the shadows on the facial image of the party. Such bright illumination by visible light may cause the party to squint and/or result in discomfort to the party. Summary of the Invention

The present invention addresses the limitations of the prior art by eliminating shadows on a face of a subject in a fashion that does not cause the subject to squint and does not result in discomfort to the subject. In one embodiment, the present invention employs one or more near-infrared light sources to illuminate a subject. It is presumed that a subject is illuminated in part by ambient visible light from another light source. A camera, such as a monochrome video camera, is employed to receive an image of the subject. The camera preferably is responsive to both the visible portion of the spectrum and the infrared portion of the spectrum. The filter is positioned to filter the image of the subject prior to receipt by the camera. The filter removes the visible spectral components in the image so that spectral components in the infrared wavelength range are passed to the camera. The camera then digitizes the image and passes the image onto a facial recognition component. The facial recognition component may be realized as a computer system that is programmed to implement a given facial recognition algorithm.

The light source may be realized as an infrared light emitting diode or an infrared emitting lightbulb. Multiple light sources may be utilized to minimize the potential for shadows. Moreover, the light sources may be integrated into the camera, as may the filter.

This embodiment of the present invention eliminates discomfort to the subject and eliminates squinting by the subject by illuminating the subject with light in a portion of the spectrum that is invisible to the subject. However, since the camera is responsive to the light emitted from the source, the camera is able to develop a complete image of the subject with light emitted from the light source. This embodiment has the additional benefits of being implemented both cheaply and efficiently. Brief Description of the Drawing

An illustrative embodiment of the present invention will be described below relative to the following drawings.

FIGURE 1 depicts a system that is suitable for practicing the illustrative embodiment of the present invention in which the light source and filter are separate from the camera.

FIGURE 2 depicts a system that is suitable for practicing the illustrative embodiment of the present invention wherein the light source is integrated into the camera.

FIGURE 3 depicts a system that is suitable for practicing the illustrative embodiment of the present invention wherein the light source and the filter are integrated into the camera.

FIGURE 4 depicts a system that is suitable for practicing the illustrative embodiment of the present invention wherein multiple light sources are utilized.

FIGURE 5 is a flow chart that illustrates the steps that are performed to eliminate shadows in the illustrative embodiment of the present invention.

Description of the Illustrated Embodiment

The illustrative embodiment of the present invention illuminates a subject with one or more light sources that emit light in the infrared (particularly the near-infrared) portion of the radiation spectrum. These light sources help eliminate shadows that might otherwise be visible when only ambient visible light is present. Since the light sources emit light that is invisible to the human eye, the light emitted from the light sources does not cause discomfort and does not result in a subject squinting. The video camera that is used to obtain one or more images of the subject is preferably responsive to both visible light and infrared light. A filter is employed to remove spectral components in the visible range so that an infrared facial image is received by the camera and digitized. The quality of the resulting image is high and readily facilitates use of the acquired image in a facial recognition system. This approach of using the IR responsive camera in conjunction with an IR bandpass filter produces superior quality images relative to those obtained using a conventional camera configuration.

In the illustrative embodiment of the present invention, the filtered and digitized facial image of a subject is passed to a facial recognition component. The facial recognition component may be employed, for example, in a computer system that is programmed to recognize the subject based on the filtered and digitized facial image. A suitable facial recognition system is U.S. Patent Application No. 09/119,485, entitled Real-Time Facial Recognition and Verification System, filed on July 20, 1998. This co- pending application is explicitly incorporated by reference herein. The elimination of the shadows by the light sources helps ensure that shadows do not decrease the efficiency with which the facial recognition system recognizes subjects. The facial recognition system may include a database of facial images. The filtered and digitized facial image can be compared with the database of facial images. The facial images stored in the database can be either complete digital representations of the person or can be templates of people created in a high dimensional image space, such as eigenfaces. Examples of such images and representative facial recognition systems for use with the present invention are described in U.S. Patent No. 5,164,992 of Turk et al, U.S. Patent No. 5,432,864 of Lu et al, and U.S. Patent No. 5,386,103 of DeBan et al, the contents of which are hereby incorporated by reference.

Figure 1 illustrates a system that is suitable for practicing the illustrative embodiment of the present invention. The system obtains a facial image of a subject 10 and attempts to recognize the facial image based upon a database of facial image data. The system may be implemented in a number of different environments. For example, the system may be implemented for automatic teller machines (ATM) to determine whether the subject is who the subject purports to be and is authorized to access the ATM. Likewise, the system may be implemented outside a controlled access area to limit the access area to authorized personnel. Still further, the system can be employed in a card production environment where the person's image is placed on an identification card, such as a drivers license.

In the implementation shown in Figure 1 , a near-infrared light source 14 provides illumination of the subject 10. The near-infrared light source 14 is preferably positioned so as to illuminate fully the face of the subject 10 and to compensate for any shadows that otherwise persist on the face of the subject. It is further presumed that the subject 10 may be illuminated by a visible light source 12. The ambient visible light originates from a combination of sources. For example, the visible light sources might include the sun and light from light bulbs.

Those skilled in the art will appreciate that the near infrared light source 14 can be implemented in a number of different ways. For example, the near-infrared light source 14 may be realized as a light emitting diode. Alternatively, the near-infrared light source 14 may be implemented using a halogen light bulb and a filter that blocks light having wavelengths shorter than 650 nanometers (or alternatively 700 nanometers). Those skilled in the art will appreciate that the near-infrared light source 14 may be positioned at different distances relative to subject 10. Preferably the near-infrared light source 14 is positioned a significant enough distance from the person so as to fully illuminate the face of the subject 10. The near-infrared light source 14, however, is not positioned so far away from the subject 10 so as to only weakly illuminate the subject. The near-infrared light source 14 is also positioned at an angle so that light reflected off the face of the subject 10 is received by the camera 18. Those skilled in the art will appreciate that a number of different angular orientations may be utilized.

The light reflecting off the face of the subject 10 passes through a filter 16. The filter 16 blocks selective spectral components of the reflected radiation from passing to or impinging on the camera 18. In particular, the filter blocks reflected spectral components within the visible range, i.e. wavelengths approximately between 400 to 700 nanometers. Thus light in the infrared range, i.e. between about 0.65 micrometers and about 1.00 micrometers, are permitted to pass to the camera 18. This includes wavelengths in the near-infrared range of between about 0.65 micrometers and about 2.0 micrometers.

The filter 16 can take many forms. For example, the filter can be a physical filter that includes materials for blocking light in the visible portion of the spectrum. Alternatively, the filter can be implemented programmatically to remove the visible components from the image of the subject that is received to the camera 18. Those skilled in the art will appreciate that many different types of filters may be utilized to realize the desired functionality. For example, a black glass filter of Schott RG 9 glass, catalog number 03 FCG 515 from Melles Griot may be used.

The filter 16 may be positioned directly in front of the lens of the camera 18 to ensure that the image received by the camera 18 is properly filtered. Alternatively, the filter 16 may be positioned at other angles relative to the camera based upon the positioning of the light sources 12 and 14 and the subject 10. In Figure 1, the filter 16 is shown as being separate from the camera 18.

Those of ordinary skill will readily recognize that the camera 18 can take many forms. Preferably, the camera 18 is a video camera that records at a video frame rate (i.e. 30 frames per second). In the illustrative embodiment of the present invention it is presumed that the camera 18 is a monochrome camera that is infrared capable (i.e. it has an extended infrared (IR) response). An example of such a camera is the Cohu 4810 with IR response. Those skilled in the art will appreciate that the present invention may also be practiced with color cameras and with cameras that do not have an extensive IR capability. One of the benefits of using the IR capable cameras is that they operate more effectively at lower levels of light, and particularly outdoors due to the sharply falling spectrum content of sky light. The fall off at longer wavelengths implies that the illumination by the near-infrared light source 14 may be effective in the shade without having to resort to extremely bright light sources.

With further reference to Figure 1 , the illustrated camera 18 can also be connected to a computer system 20. The computer system 20 can be a workstation, personal computer, mainframe computer, mini-computer or other computing system. The computer system 20 includes a facial recognition facility 22. The facial recognition facility 22 may be implemented in software to perform a given facial recognition algorithm, as discussed above. The computer system 20 receives a facial image in digital form from the camera 18.

A number of different configurations are available for practicing the present invention. As shown in Figure 2, a light source 24 may be integrated into the camera 18. Moreover, the light source 24 need not emit light in the near-infrared range but may, instead, emit light in other portions of the infrared range. As shown in Figure 3, the filter 16 may also be directly integrated into the camera. Still further, as shown in Figure 4, multiple light sources 24A and 24B may be utilized. Preferably, the light sources 24A and 24B are positioned so as to fully illuminate the face of the subject 10. Those skilled in the art will appreciate that more than two light sources may be utilized.

Figure 5 provides a flow chart that summarizes operation of the system for practicing the illustrative embodiment of the present invention. Initially, the subject 10 is illuminated with visible light and infrared light (step 30 in Figure 5). Those skilled in the art will appreciate that it is not necessary for the subject to be illuminated with the visible light to practice the present invention. The facial image of the subject produced by the light that reflects off the face of the subject from the infrared light source and the visible light passes through the filter 16 (step 32 in Figure 5). The filter removes light within the visible range of the spectrum. Thus, the light that passes through the filter 16 onto the camera 18 is in the infrared portion of the spectrum. The resulting filtered facial image of the subject is received at the camera 18 (step 34 in Figure 5). As has been discussed above, the camera 18 digitizes the facial image and the digitized filtered facial image is passed on to the computer system 20 where the facial recognition system 22 performs facial recognition to attempt to identify the subject (step 36 in Figure 5).

While the present invention has been described with reference to an illustrative embodiment thereof, those skilled in the art will appreciate that various changes as in form and detail may be made without departing from the intended scope of the present invention as defined by the appended claims. For example, three or more infrared light sources may be utilized and an array of infrared light emitting diodes may be used. Moreover, multiple visible light sources may be present to provide ambient light for illuminating the subject.

Claims

1. A system for obtaining and filtering an image of a subject exposed to variable lighting conditions, comprising: an infrared light source for illuminating a subject that is at least partially illuminated by visible light from another source with light in the infrared light wavelength range; a camera for receiving an image of the subject; and a filter for filtering the image of the subject having multiple spectral components before the image is received by the camera to remove spectral components in the image that lie substantially outside of the infrared light wavelength range.

2. The system of claim 1 wherein the infrared light source is positioned on or about the camera.

3. The system of claim 1 further comprising an additional infrared light source for further illuminating the subject.

4. The system of claim 1 wherein a plurality of infrared light sources are positioned on the camera.

5. The system of claim 1 wherein the camera comprises a monochrome camera with an extended infrared response.

6. The system of claim 1 wherein the filter passes spectral components in the infrared light wavelength range.

7. The system of claim 1 wherein the camera comprises a video camera.

8. The system of claim 1 wherein the filter is integrated into the camera.

9. The system of claim 1 further comprising a facial recognition component for recognizing the subject based on the image of the subject received by the camera.

10. The system of claim 1 wherein the infrared light source comprises a light emitting diode.

11. The system of claim 1 wherein the infrared light source comprises a halogen light bulb and a filter that blocks spectral components that are not infrared.

12. A facial recognition system, comprising: at least one light source for emitting infrared light to illuminate a face of a subject; a filter for permitting non-visible spectral components in an image of the face of the subject to pass; a video camera that is responsive to visible light and to infrared light for receiving the spectral components in the image of the face of the subject that pass through the filter to produce a facial image of the subject; and a facial recognition component for receiving from the video camera and processing the facial image of the subject to recognize the subject.

13. The facial recognition system of claim 12 wherein multiple light sources are used.

14. The facial recognition system of claim 12 wherein the lights source is positioned on the video camera.

15. The facial recognition system of claim 12 wherein the light sources emits infrared light.

16. The facial recognition system of claim 15 wherein the light source emits near- infrared light.

17. The facial recognition system wherein the filter is integrated into the camera.

18. The facial recognition system of claim 12 wherein the facial recognition component comprises a computer system programmed to perform facial recognition.

19. The facial recognition system of claim 12 wherein the video camera is a monochrome video camera with an infrared response.

20. In a facial recognition system having a video camera response to visible light and infrared light a method of illuminating a subject to eliminate shadowing; comprising the steps of: providing a near-infrared light source; illuminating the subject with near-infrared light from the near-infrared light source, wherein said subject is already partially illuminated by visible light; filtering a facial image of the subject having multiple spectral components, produced by illumination of the subject by both the visible light and the near-infrared light source, to remove visible spectral components and produce a filtered facial image; and receiving the filtered facial image at the video camera.

21. The method of claim 1 wherein the method further comprises the step of applying a facial recognition technique to the filtered facial image received at the video camera to recognize the subject.