EP1170715A2 - Method for surface surveillance - Google Patents

Abstract

One or more video sensors monitor one or more traffic locations. A memory storage unit captures and reads in image data from one or more snapshots from the video sensors. The results from the image data are identified. State vectors of the results are calculated from the image data. These vectors are then allocated to an object and object data is produced.

Description

The present invention relates to a method for space monitoring, in particular of the floor space at airports. Air and ground surveillance is part of air traffic control carried out and serves to make air traffic smooth, fast and safe.

In addition to the fast and smooth handling of air traffic, the tasks of air traffic control include the avoidance of collisions between vehicles in the air and on the runways, the tarmac and the parking areas of the controlled airports.

Air traffic capacity is essentially limited by two bottlenecks. So is for due to restricted airways and corridors, the usable airspace is severely restricted. On the other hand, the traffic throughput on the ground is often severely limited because it is inexpensive and intelligent floor space monitoring systems are missing, and there is one besides aircraft Variety of service vehicles takes part in ground traffic.

The known floor space monitoring systems are very expensive because they have a large number of sensors, such as. Airport radar, secondary radar and GPS are required.

As a result, only a small proportion of the national and international airports have a floor space monitoring system equipped. But even if a known floor space monitoring system monitoring is often incomplete, since in general not all areas of the airfield can be observed.

The object of the present invention is therefore to provide a method, an inexpensive and effective room surveillance, especially of the air and ground space from airports, allowed.

• Überwachung mindestens eines Verkehrsortes mit mindestens einem Videosensor,
• Einlesen der Bilddaten mindestens einer Momentaufnahme des zumindest einen Videosensors in eine Speichereinheit,
• Erkennen von Ereignissen,
• Berechnung von Zustandsvektoren der Ereignisse aus den Bilddaten und
• Zuordnen der Zustandsvektoren jeweils zu einem Objekt.

According to the invention, this object is achieved by a method comprising the steps:

• Monitoring of at least one traffic location with at least one video sensor,
• Reading in the image data of at least one snapshot of the at least one video sensor into a storage unit,
• Detection of events,
• Calculation of state vectors of the events from the image data and
• Assign the state vectors to an object.

Under a spatial area or traffic location you can e.g. any place on the airfield understand. This can be the runway or landing road or any access road be on the apron or a gate exit. Even if the method according to the invention can already be used with advantage if only one traffic location with at least is monitored by a video sensor, a whole series of traffic locations are preferred or traffic hubs are each monitored with at least one video sensor.

In principle, the video sensor can be any sensor that reacts to light. Particularly preferred However, digital video cameras are used because they deliver image data that is special can be easily processed. This image data can be in bitmap format, for example with each pixel individually entered in the file. But others can also benefit less memory-intensive data formats are used. The video sensor provides a snapshot his field of view in the form of an image data file, which is suitably in a Storage unit is read. This image data is evaluated for one or more events recognizable, if there are any. Under an event are all irregularities understood in the field of view of the video sensor, in particular movements of objects in the field of view the camera or a foreground object in front of a specific and saved one solid surface. Is e.g. on the tarmac an airplane that is just over, for example rolls a so-called stopbar, so this is an event. For example, if image data is two A group compares time-spaced snapshots of the same traffic location of pixels whose color is in high contrast to the surrounding pixel groups and which takes a different position on the two snapshots, represents an event. In the next Step, a state vector is calculated for each known event from the image data. This In principle, the state vector can be defined arbitrarily, e.g. as the difference between two vectors Sets of image pixels or data. One possibility is that he takes the position of the observed event.

In this respect, recognizing an event also presupposes the definition of what an event is, the above explanations providing an example of such a definition.

Finally, an object is assigned to each state vector. Everyone is under an object understood real object that is located or moves on the airport field.

Of course, it is possible to limit the detection of events to events that have a minimum size. This can prevent, for example, accidentally an object is assigned through the field of view of the bird flying bird video sensor.

The method described enables one or more traffic locations to be monitored efficiently become. The object data can either be sent directly to the air traffic control personnel or to a suitable processing facility can be passed on. The air traffic control personnel is thereby informed at all times whether an aircraft is at one of the observed traffic locations or where the planes are located.

A method is particularly preferred in which image data of at least two temporally spaced Snapshots of at least one video sensor can be read. In that two temporally Staggered snapshots of the same video sensor and thus the same traffic location , it is possible due to the change in the position of the pixel group that the event represents to calculate a speed component of the object. With a suitable installation of the video sensor and the corresponding route of the aircraft corresponds to the speed component the absolute speed. If the route is not straight it can be advantageous if at least one traffic location with several, preferably at least three, video sensors is monitored. In this case, the relative speed component can be for each video sensor of the object can be calculated separately. Knowing the locations of the Video sensors can determine the absolute speed and the direction of movement of the object be determined. A particularly preferred embodiment of the present method therefore provides that the state vector also the speed and / or the direction of movement represents the object.

This makes it possible, for example, to calculate the movement or the position of the object in advance. Provided that the speed of the object does not change significantly, the position of the object can be calculated for the future without further image data from Video sensors are necessary. This means that the entire runway area is not monitored must become. Rather, it is possible to move between two monitored traffic locations from the State vectors to calculate the expected position of the object. This means that Air traffic control personnel or the processing unit at all times the calculated current positions of the aircraft on the ground of the airfield are available without all Areas of the airfield are monitored by video sensors.

Another particularly preferred variant of the method according to the invention provides that the State vector represents the acceleration of an object. To calculate the acceleration at least three snapshots of a video sensor are required. From these three snapshots the acceleration of the object can be calculated. The accuracy of the position prediction the process is significantly improved. This is particularly true in areas in which the speed of the aircraft changes significantly; e.g. due to a braking process, of great advantage. Can namely the acceleration of the state vector Object can be removed, so this can be extrapolated over time the position of the object be taken into account.

For some applications it can be advantageous to assign values to the state vector that are a measure of the statistical probabilities of the object parameters. This Values can also be integrated into the state vector. This measure can reduce the error rate be significantly minimized in event detection. For example, it is possible that the image data of the video sensor due to external influences, e.g. heavy rainfall or Bird flight, do not show the object to be observed. Then it can happen that either a Object is recognized, which is actually not at the traffic location or that is at the traffic location aircraft is not recognized. Therefore, a state vector of an object, that on many consecutive snapshots and from different video sensors a high statistical probability was assigned during a state vector of an object that was only recognized in a snapshot, a very low statistical probability is assigned. The statistical probability is therefore a measure of the quality or quality (i.e. the accuracy and reliability) of the detected Event. It is then possible, for example, to define a limit value and all state vectors, the value of the statistical probability does not fall below this limit to consider.

A particularly preferred embodiment of the method according to the invention provides that the Assignment of the state vector to an object with access to traffic situation data takes place were not won by the video sensors. This traffic situation data can be varied Way can be obtained. For example, it is possible that the air traffic control personnel assigns the individual status vectors to the arriving aircraft at a suitable terminal. In this case, the state vector also contains an object identification number or one Object name. This makes it possible to use the method not only to monitor where a Plane, but even monitor where each plane is located.

Another possibility provides that the traffic situation data from the current flight plan and / or can be taken from the occupancy plan of the parking positions. For example, the existing one Air traffic control system with a system which implements the method according to the invention, be coupled so that the floor space monitoring system whenever an aircraft is on the Runway lands, accesses the flight plan data of the existing air traffic control system and that Airplane identified. This identification is then used during further floor space monitoring maintained. An object that moves away from a parking position can be used in the same way Using the occupancy plan of the parking positions can be identified.

Instead of or in combination with the flight plan data, radar data can also be used. Alternatively, of course, all other position sensors can also be used provide additional traffic situation data. This is where Mode-S radar sensors, Near come for example Range radar networks or GPS receivers and servers in question.

A particularly expedient embodiment provides that the image data are filtered digitally, preferably using discrete Kalman filtering. Alternatively or in combination, at least some of the traffic situation data can also be digitally filtered, with discrete Kalman filtering preferably also being used here. Kalman filtering enables real-time estimation and smoothing of the measured values. Alternatively, other suitable estimation techniques, for example, the MLS-filtering are (m inimum l east s quare) was used. The quality of the method according to the invention can thereby be significantly increased. As already mentioned, it is particularly preferred if the state vectors are calculated further as a function of time. With this method, the position of the individual objects can be predicted with high probability even in the event of a short-term failure of the video sensors.

The state vectors of the same object are preferably those of different ones Sensors were determined, summarized. The same applies to those received from the sensors State vectors and the state vectors from the other traffic situation data. This will make the Floor space monitoring is clearer, since fewer condition vectors have to be followed.

It is not always possible to summarize the state vectors. Especially if Different systems for recording traffic situation data can be used, a summary become difficult or even impossible. In this case it is advantageous if the State vectors of the same object are correlated with one another. Delivers the radar system, for example with large time intervals reliable position data of a certain object, so the state vector of this object can be correlated with the radar data. In other words Whenever the reliable radar data is available, it is determined from the video sensors and, if necessary, time-extrapolated state vector of this object is adapted. This correlation allows the state vectors to be corrected even if the object is not on one traffic location monitored by video sensors.

A method is particularly preferred in which the assignment of the state vectors takes place Objects the step connects:

Generation of a traffic situation display.

This traffic situation can be displayed in any coordinate system. The state vectors of the recognized objects are converted into positions in the traffic situation display and entered into the traffic situation display at these positions. It goes without saying it is possible to visualize the traffic situation so that the air traffic control personnel on can see at a glance where the individual aircraft are and whether they are on somewhere a collision is imminent.

However, such a visual representation is not absolutely necessary. So it is special in one preferred variant of the method provided that the state vectors with regard to possible Collisions are evaluated. Here the state vectors are not only used for generation a traffic situation representation in real time, but there is an extrapolation of the State vectors in the future to be able to predict possible collisions. Particularly preferred The method therefore provides that an alarm signal depending on the collision evaluation is activated. This alarm signal can be, for example, an optical or acoustic signal for air traffic control personnel.

Figur 1

einen schematischen Aufbau eines Bodenraumüberwachungssystems.

Further advantages, features and possible applications of the present method become clear from the following description of a preferred method and the associated figure. It shows:

Figure 1

a schematic structure of a floor space monitoring system.

The only figure shows the basic structure of a floor space monitoring system. The single ones Components of the system are interconnected via a communication network 9 (e.g. TCP / IP-LAN network). The individual ones are above the line 9 in FIG Clients 10, 11 and 12 are shown, while below the line 9 the three servers 2, 3 and 5 are shown are. Server 2 is responsible for merging the types of video sensors. The individual video sensors 7, which recognize the objects 8, are activated via the control and read-out unit 1 connected to the server 2. The server 2 collects the sensor data and filters them with more discrete time variants Kalman filtering and forms the state vectors of objects 8. The state vectors are passed on to the radar server 3 for further processing. The radar server 3 filters one or several airspace surveillance radars with a time-invariant discrete Kalman filter to Generate flight routes. This server carries out a correlation between the video-based trajectories and the radar trajectories together with a correlation with the flight plan data 5. The correlated Information is provided to clients 10, 11 and 12. The flight plan server 5 delivers the flight plan data to the radar server 3 for the correlation of the radar with the flight plan and for the time-based Identification of the aircraft taking off. The flight plan server 5 receives flight plan changes from its clients 6.

The method according to the invention converts video sensor data with other traffic situation data combined to produce a traffic situation representation that provides safe and reliable monitoring of the air and ground space guaranteed. A system that the inventive method realized consists of sensor hardware 1, 7, 8, computer hardware 2, 3, 5, network infrastructure 9 as well as sensor software, communication software, processing software and situation display software.

The video sensor data are recorded with the help of the video sensors 7. First, use With the help of digital cameras 7 pixel data recorded. These include so-called image frames one or more high frequency video sensors (typically greater than 27 Hz). there Image frames are at least two-dimensional vectors of pixels (gray or color values), which reflect the recording of the digital camera with a non-infinite resolution. From these Pixel data, the state data or state vectors are calculated. These can be the following Information includes: object identification number, object classification, position, speed, Direction of movement, acceleration and statistical probability / quality (accuracy, Reliability).

The other traffic situation data can be, for example, radar data that the radar server 3 provides Provides. This is digitized data from, for example, long-range radars and airport radars. These radars have been symbolized by the box with the reference number 4. The flight plan data provided by the flight plan data server 5 can for example, from existing flight plan systems that are connected to a system that realized the method according to the invention, can be connected.

The video sensor data is combined with the other traffic situation data by image data processing, digital filtering, statistical interference, sensor data fusion, track correlation and Time frame correlation. The image data processing includes the transmission of the pixel data selected digital camera to the dialing position display system as well as the recognition and Classification of objects in the pixel data. The digital filtering of the measured values is carried out using the discrete Kalman filtering. With their help, all extracted state vectors from event detection of the video sensor data filtered. The radar data can be transmitted in the same way or the data from other sensors can be filtered. Statistical inference means that in the calculation of future state vectors or the extrapolation of state vectors is taken into account that the output data with a corresponding inaccuracy and unreliability, so that the statistical probability of the new State vector from the propagation of the inaccuracy and unreliability of the original State vectors calculated.

Sensor data fusion is the correct merging of object information for objects, that were detected by several sensors. For example, two different Video sensors complementary motion information of one and the same object received, they are entered in a common state vector. With sensor data fusion However, care must be taken to ensure that there are different objects that may differ come very close, can be correctly distinguished at any time.

The track correlation is understood to mean that those calculated on the basis of different sensors Flight or movement routes for the same objects are checked. In other words the flight routes determined from the video sensors are compared with the flight routes derived from the radar data were determined, correlated, if available.

The time window correlation is understood to mean that the routes based on the sensor data were determined with the movement schedule of the objects in accordance with the available flight plan data can be adjusted within a certain time tolerance window.

The state vectors determined in this way are entered in a positional representation. This is done in sequential cycles. The individual position data are sent from the server 2 to the radar server 3 passed. The servers 2 and / or 3 also take over the conflict determination. That means for Two status vectors are checked to determine whether there is a risk of collision. Furthermore, it is checked whether blocked traffic protection areas are violated by other intruding objects.

If there is a risk of collision or injury, a corresponding alarm is triggered.

With the help of the present method it is possible to inexpensively and extremely effectively and in particular to carry out floor space monitoring. Through this procedure it is possible to effectively extend monitoring into areas that are not detected by any sensor become. The procedure can be carried out very easily with existing air and ground space monitoring systems be combined. Both processes benefit from each other.

Claims (22)

A method of room surveillance comprising the steps: Monitoring of at least one traffic location with at least one video sensor, Reading in the image data of at least one snapshot of the at least one video sensor into a storage unit, Detection of events, Calculating state vectors of the events from the image data, Assigning the state vectors to an object and Output of object data. Method according to Claim 1, characterized in that the monitored spatial area forms at least part of an airfield or airport site. Method according to claim 1 or 2, characterized in that the state vector represents the position of an object. Method according to one of claims 1 to 3, characterized in that image data of at least two temporally spaced snapshots of the at least one video sensor are read. Method according to one of claims 1 to 4, characterized in that the at least one traffic location is monitored with several, preferably with at least three, video sensors. Method according to one of claims 1 to 5, characterized in that the state vector represents the speed and / or the direction of movement of an object. Method according to one of claims 1 to 6, characterized in that the state vector represents the acceleration of an object. Method according to one of claims 1 to 7, characterized in that the state vector is assigned a value which is a measure of the statistical probability of the object or the accuracy and reliability of the object parameters. Method according to one of claims 1 to 8, characterized in that the assignment of the state vectors to an object takes place with access to traffic situation data which were not obtained by the video sensors. Method according to Claim 9, characterized in that the assignment of the state vectors to an object takes place with access to radar data. Method according to Claim 9 or 10, characterized in that the assignment of the state vectors to an object takes place with access to flight plan data. Method according to one of claims 9 to 11, characterized in that the assignment of the state vectors to an object takes place with access to position sensors. Method according to one of Claims 1 to 12, characterized in that the image data are filtered digitally, the discrete Kalman filtering or MLS filtering preferably being used. Method according to one of claims 9 to 13, characterized in that at least some of the traffic situation data is digitally filtered, preferably using discrete Kalman filtering or MLS filtering. Method according to one of Claims 1 to 14, characterized in that the state vectors are further calculated in real time in a time-dependent manner. Method according to one of claims 1 to 15, characterized in that the state vectors of the same object, which were determined by different sensors or traffic situation data, are combined. Method according to one of claims 1 to 16, characterized in that the state vectors of the same object, which were determined from different traffic situation data, are correlated. Method according to one of claims 1 to 17, characterized in that the assignment of the state vectors to objects is followed by the step:
Generation of a traffic situation display. Method according to Claim 18, characterized in that the state vectors are converted into positions in the traffic situation representation and are entered in the traffic situation representation at these positions. Method according to claim 18 or 19, characterized in that the state vectors are evaluated with regard to possible collisions. A method according to claim 20, characterized in that an alarm signal is activated depending on the result of the collision evaluation. Method according to one of claims 18 to 21, characterized in that the traffic situation is updated at preferably fixed, equidistant time intervals.