WO2010097211A1

WO2010097211A1 - Method and device for monitoring the state of a traffic light

Info

Publication number: WO2010097211A1
Application number: PCT/EP2010/001143
Authority: WO
Inventors: Clint A. Davis
Original assignee: Gatsometer B.V.
Priority date: 2009-02-24
Filing date: 2010-02-24
Publication date: 2010-09-02

Abstract

The invention relates to a method of monitoring the state of a signal, in particular a traffic light, comprising the steps of capturing an image of the signal, determining a color of the signal in the captured image, and generating an output signal in response to the determined color. The method further includes capturing the image such that it includes a plurality of colors, determining an intensity of at least some of the colors in the captured image, monitoring variations in the determined color intensities, and generating the output signal in response to the variations in the color intensities. The invention also relates to a device for carrying out this method, comprising image capturing means, color determining means connected to the image capturing means, and output signal generating means connected to the color determining means. The image capturing means may include a series of lenses and optical filters, and the color determining means may comprise a plurality of photo detectors. The output signal generating means may comprise a microcomputer.

Description

METHOD AND DEVICE FOR MONITORING THE STATE OF A TRAFFIC LIGHT

The present invention relates to a method of monitoring the state of a signal, in particular a traffic light, comprising the steps of capturing an image of the signal, determining a color of the signal in the captured image, and generating an output signal in response to the determined color. Such a method is known. When detecting traffic violations at locations where traffic is controlled by signals such as traffic lights, it is important to have accurate information about the state of the signal when a vehicle passes. On the basis of this information a distinction can be made between a legitimate pass when the traffic light is green or yellow, and a traffic violation when a vehicle passes a red light.

Conventionally, the state or phase of a traffic light is monitored by means of a wired connection to the traffic light itself. This wired connection directly transmits the current or voltage applied to one of the lamps of the traffic light to a red light camera system. In this way the red light camera system can be triggered whenever a vehicle passes while the wired connection indicates that the red lamp is energized. This conventional approach has the drawback that it requires a buried or overhead cable to be installed. Installing such a cable leads to temporary disruption of the traffic flow at the location of the traffic light - which will normally be a relatively busy road section - and to relatively high installation costs. Moreover, by making an electrical connection to the actual traffic signal there is a risk that a malfunction within the red light camera system or its cabling could affect the reliability, safety or timing of the traffic signal .

A more recent approach is to monitor the state or phase of a traffic light using a camera. In that case a video camera is focussed on the traffic light and captures an image that includes the red, yellow and green lights. The camera is connected to processing means, such as a PC with a frame grabber module and custom software. The software examines a region of each captured image containing the traffic signal in order to ascertain the phase of the light.

This approach has the drawback that the speed of operation is low. This is due to the camera frame rate, the speed of data transfer from the camera to the PC, and software processing latencies within the software. This low speed of operation means that it is not always possible to make an accurate determination of the exact moment of switching the traffic light from yellow to red, which is important for providing irrefutable proof of a red light violation. Moreover, video based monitoring of traffic lights requires a substantial investment in cameras and processing equipment. And finally, it is often impossible to correctly determine the light phase in cases where the traffic light moves or swings in the wind.

The invention has for its object to provide a method of the type described above in which the above-mentioned drawbacks are at least partially obviated. In accordance with the invention, this is accomplished in such a method in that the image is captured such that it includes a plurality of colors, an intensity of at least some of the colors in the captured image is determined, variations in the determined color intensities are monitored and the output signal is generated in response to the variations in the color intensities. By imaging several colors at once and measuring their intensities a swift and reliable determination of the state or phase of a traffic light can be made using relatively low-cost equipment. As the output signal depends on variations in color intensity, rather than on absolute values of the color intensities, the exact moment of switching of the traffic light from green to yellow and from yellow to red may be accurately determined. Moreover, interference from high intensity light sources having the relevant colors, like e.g. neon signs, roadside illumination or a setting sun, is prevented in this way.

The image is preferably captured out of focus so as to include a mix of all colors of the signal . In this way a blurred image is provided, which is highly insensitive to movement of the traffic light due to wind or other effects. In a preferred variant of the method relevant colors of the signal are isolated from the mix before determining the intensity thereof. This facilitates determination of the intensity of specific colors using relatively simple means. In order to provide substantially continuous monitoring, and to provide irrefutable proof of violations, the output signal is preferably updated relatively frequently, in the order of at least several hundreds of times a second, and preferably several thousands of times a second. In this way an accurate determination can be made of the exact moment of switching the traffic light from yellow to red.

These features are especially important when the output signal is combined with a signal generated in response to the presence of a vehicle in order to detect a traffic violation, in particular running a red light. The output signal can thus be used to control a red light camera system. The invention also relates to a device for monitoring the state of a signal, in particular a traffic light, with which the above method can be carried out . A prior art video based traffic light monitoring device of the type described above comprises means for capturing an image of the signal, means connected to the image capturing means for determining a color of the signal in the captured image, and means connected to the color determining means for generating an output signal in response to the determined color. The invention has for its object to provide an improved monitoring device. According to the invention, this is accomplished in that the image capturing means are arranged for capturing an image including a plurality of colors, the color determining means are arranged for determining an intensity of at least some of the colors in the captured image, and the output signal generating means are arranged for monitoring variations in the determined color intensities and for generating the output signal in response to the variations in the color intensities. This device allows the phase of a traffic light to be accurately determined using relatively simple and low-cost components. For instance, the output signal generating means may detect a decrease in yellow intensity followed by an increase in red intensity, and thus deduce that the signal phase has switched to red.

In order to form a blurred or out -of -focus image as discussed above, the image capturing means of the device preferably include at least one lens having a front focal distance that is different from the distance between the device and the signal being monitored. Alternatively or additionally, a blurred or out-of-focus image may be formed when the image capturing means include at least one lens having a back focal distance that is different from the distance between the at least one lens and the color determining means .

In a preferred embodiment of the monitoring device the image capturing means may be arranged for isolating relevant colors of the signal from the mix and for transmitting the isolated colors to the color determining means. To this end the image capturing means may preferably comprise a plurality of lenses, including two convex lenses arranged on opposite sides of a concave lens. In a further preferred embodiment the image capturing means may include a plurality of optical filters, each said filter being transmissive substantially only for one of the relevant colors of the signal . Such lenses and/or optical filters provide an efficient solution to isolate the relevant colors of the signal from the mix and transmit these to the color determining means .

The color determining means of the monitoring device preferably include a number of photo detectors corresponding to the number of relevant colors of the signal . These photo detectors determine the light energy passing through the optical filters, which is a measure of the intensity of the relevant colors.

A simple and low-cost device is further realized when the output signal generating means include a microprocessor that is connected to the photo detectors.

Further preferred embodiments of the monitoring device of the invention are defined in dependent claims 15 and 16.

The invention is now illustrated by way of an exemplary embodiment, with reference being made to the drawings, in which:

Fig. 1 is a schematic perspective view of an intersection where traffic is controlled by traffic lights, and including a red light camera system in combination with a traffic light monitoring device in accordance with the invention, and

Fig. 2 is a schematic perspective view of the main components of the monitoring device of fig. 1. At an intersection 1 where two roads 2, 3 meet the flow of traffic is controlled by a traffic controller 11 which controls four traffic lights 4. Each traffic light 4 is suspended above one of the roads 2, 3 and has a red light 5, a yellow light 6 and a green light 7. These lights can be conventional incandescent bulbs, LEDs or any other suitable light source.

A pair of red light camera systems 8 serves to detect and record traffic violations, in particular vehicles running a red light. Each red light camera system 8 includes means 9 for detecting the passage of a vehicle and a camera 10. The passage detection means 9 could include induction loops arranged in the surface of the roads 2, 3, but preferably include a non- intrusive detection system like radar or laser. Such non- intrusive detection systems are described in EP 1 662 272 and US 7,405,676, respectively, each in the name of the present applicant. The disclosure of these earlier patents is incorporated herein by reference.

Each red light camera system 8 is triggered whenever a vehicle passes a stop sign 12 during a red light phase of the relevant traffic light 4. In order to ascertain the state of the traffic light 4 a monitoring device 20 is used. This monitoring device 20 is also non- intrusive, i.e. it does not require any connection to the traffic light 4 being monitored or the traffic controller 11. Instead it captures an image of the traffic light 4 and determines whether the signal is in the red, yellow or green phase on the basis of this captured image . The monitoring device 20 includes means 21 for capturing an image of the traffic light 4, means 22 for determining the color of the traffic light 4 in the captured image, and means 23 for generating an output signal OS in response to the determined color.

In the illustrated embodiment the image capturing means 21 include a series of lenses, including two convex lenses 24, 26 and a concave lens 25 arranged therebetween. This lens system 24, 25, 26 receives a beam 27 of light from the traffic light 4, which is made up of light rays from the red, yellow and green lights 5, 6, 7. These light rays are comingled and very nearly parallel . The outside lens or entrance pupil 24 may have a focal length which is different from the distance D between the traffic light 4 and the monitoring device 20 - D is approximately 20 to 50 metres (60 to 150 feet) , depending on the situation - so that the image of the traffic light 4 is out of focus or blurred. Additionally or alternatively, a blurred image may also be formed by providing an inside lens or exit pupil 26 having a back focal length which is different from the distance d between the lens and an image plane 38. In this embodiment the rear focal point F lies behind the image plane 38, resulting in a shorter device than if the image plane 38 were behind the focal point F. The series of lenses 24, 25, 26 effectively constitute a telescope having a relatively narrow field of view of e.g. 3 degrees.

Since the basis of the determination of the state of the traffic light 4 is a blurred image, it is highly insensitive to variations due to movement of the traffic light 4, e.g. caused by wind or other effects. It is also insensitive to the exact orientation of the traffic light 4, as long as the signal remains in the optical field of view. In the illustrated embodiment the field of view is twice the size of the traffic light 4, which allows considerable movement of the traffic light 4 without affecting monitoring of its state.

Because the image captured by the image capturing means 21 is out of focus, it includes a mix of all colors of the traffic light 4. The image capturing means 21 isolate the three relevant colors, red, yellow and green from the mix and transmit only these three isolated colors to the color determining means 22. The system of lenses 24, 25, 26 projects light rays originating from the various parts of the traffic light 4 onto different areas in the image plane 38. Although the light rays are very nearly parallel, there are minute differences in their exact directions of travel, depending on the actual light - red, yellow or green - from which they originate. These differences cause light rays originating from one of the three lights 5, 6 or 7 to be guided onto a specific area in the image plane 38, regardless of the point where they enter the entrance pupil 24. The image capturing means 21 further include three narrowband optical filters 28, 29, 30. These filters 28, 29, 30 are located in the three areas of the image plane 38 which correspond with the three colors - red, yellow and green - of the traffic light 4. Each of these filters 28, 29, 30 is transmissive only for the color that is expected to be predominantly present in the respective area, filter 28 for red, filter 29 for yellow and filter 30 for green.

The color determining means 22 determine the intensity of each of the three colors red, yellow and green in the captured image of the traffic light 4 as transmitted by the filters 28, 29, 30. To this end the color determining means 22 include three photo detectors, a photo detector 31 for the color red, a photo detector 32 for the color yellow and a photo detector 33 for the color green. Each of these photo detectors 31, 32, 33 measures the light energy passing through the respective narrowband optical filter 28, 29, 30.

The output signal generating means 23 include a microprocessor 37 that is connected to the photo detectors

31, 32, 33 by respective A/D convertors 34, 35, 36. The microprocessor 37 is programmed to continuously monitor the digitized output signals from the three photo detectors 31,

32, 33 to detect variations in the intensity of the three colors red, yellow or green. These variations indicate which of the three signal phases is active and which are inactive. When for instance the microprocessor 37 detects a decrease in yellow intensity followed by an increase in red intensity, it will deduce that the signal phase has switched from yellow to red and will modify the output signal OS accordingly. This output signal OS is then sent to the red light camera system 8, which is then armed to be triggered whenever a vehicle passes the stop line 12. In order to be recognized as a valid signal phase, the color intensity must exceed a predetermined threshold level that is programmed into the microprocessor 37. If all three color intensities are found to lie below this threshold level, the microprocessor 37 deduces that the traffic light is not operative, and a corresponding output signal OS is generated.

The microprocessor 37 can operate at a sampling rate of several thousand Hertz, thus providing many thousands of intensity measurements every second. This offers the benefit of a very low detection latency; detection is instantaneous. This high sampling rate also affords the possibility of synchronizing the red light camera system 8 with the AC mains phase operating the traffic signal. This is significant in situations where the traffic signal light amplitude varies in time in synchronization with the AC mains instantaneous phase voltage. In this way a synchronized red light camera system 8 may capture an image of a traffic violation at a moment when the traffic light signal is at its maximum brightness. Such synchronization is described in more detail in the present applicant's earlier patent applications US 2005/206728 and EP 1 497 807, the disclosures of which are incorporated herein by reference. In this way the method and device of the present invention allow an accurate determination to be made of the state of a traffic light, without requiring any connection with the traffic light or its controller. Moreover, the monitoring device of the invention is relatively compact and may be made at relatively low cost. It is presently assumed that the device 20 will fit in a tubular housing with a diameter of approximately 8 cm (3 inch) and a length of approximately 25 cm (10 inch) . Moreover, it is envisaged that the device 20 will include no more than 10 active semiconductor devices and 15 passive devices, all combined on a 40 mm (1.5 inch) diameter circuit board. The device 20 may include consumer grade lenses and an outer housing made of a suitable synthetic material like e.g. PVC, which will result in relatively low manufacturing cost.

Although the invention has been described on the basis of an exemplary embodiment, it will be clear that it is not limited to this embodiment and that various modifications can be made to both the method and the device . The scope of the invention is defined solely by the following claims.

Claims

1. A method of monitoring the state of a signal, in particular a traffic light, comprising the steps of: - capturing an image of the signal,

- determining a color of the signal in the captured image , and

- generating an output signal in response to the determined color, characterized in that

- the image is captured such that it includes a plurality of colors,

- an intensity of at least some of the colors in the captured image is determined, - variations in the determined color intensities are monitored, and

- the output signal is generated in response to the variations in the color intensities.

2. The method of claim 1, characterized in that the image is captured out of focus so as to include a mix of all colors of the signal .

3. The method of claim 2 , characterized in that relevant colors of the signal are isolated from the mix before determining the intensity thereof.

4. The method of any of the preceding claims, characterized in that the output signals are updated relatively frequently, in the order of at least several hundreds of times a second, and preferably several thousands of times a second.

5. The method of any of the preceding claims, characterized in that the output signal is combined with a signal generated in response to the presence of a vehicle in order to detect a traffic violation, in particular running a red light.

6. A device for monitoring the state of a signal, in particular a traffic light, comprising: - means for capturing an image of the signal,

- means connected to the image capturing means for determining a color of the signal in the captured image, and

- means connected to the color determining means for generating an output signal in response to the determined color, characterized in that

- the image capturing means are arranged for capturing an image including a plurality of colors,

- the color determining means are arranged for determining an intensity of at least some of the colors in the captured image, and

- the output signal generating means are arranged for monitoring variations in the determined color intensities and for generating the output signal in response to the variations in the color intensities.

7. The device of claim 6, characterized in that the image capturing means are arranged for capturing the image out of focus so as to include a mix of all colors of the signal .

8. The device of claim 7, characterized in that the image capturing means include at least one lens having a front focal distance that is different from the distance between the device and the signal being monitored.

9. The device of claim 7, characterized in that the image capturing means include at least one lens having a back focal distance that is different from the distance between the at least one lens and the color determining means .

10. The device of any of claims 7 to 9, characterized in that the image capturing means are arranged for isolating relevant colors of the signal from the mix and for transmitting the isolated colors to the color determining means .

11. The device of claim 10, characterized in that the image capturing means comprise a plurality of lenses, including two convex lenses arranged on opposite sides of a concave lens .

12. The device of claim 10 or 11, characterized in that the image capturing means include a plurality of optical filters, each said filter being transmissive substantially only for one of the relevant colors of the signal .

13. The device of any of claims 10 to 12, characterized in that the color determining means include a number of photo detectors corresponding to the number of relevant colors of the signal .

14. The device of claim 13, characterized in that the output signal generating means include a microprocessor that is connected to the photo detectors.

15. The device of any of claims 6 to 14, characterized in that the output signal generating means are arranged to operate at a relatively high frequency, in the order of at least several hundred Hertz, and preferably several thousand Hertz.

16. The device of any of claims 6 to 15, characterized in that the output signal generating means are connected to a traffic law enforcement system, in particular a red light camera system.