WO2011036661A1

WO2011036661A1 - System and method for long-range surveillance of a scene and alerting of predetermined unusual activity

Info

Publication number: WO2011036661A1
Application number: PCT/IL2010/000779
Authority: WO
Inventors: Lazar Liveanu; Meir Hahami; Eyal Rosenthal
Original assignee: Elbit Systems Ltd.; Elbit Security Systems Ltd.
Priority date: 2009-09-24
Filing date: 2010-09-21
Publication date: 2011-03-31
Also published as: IL201200A0

Abstract

A method for surveillance of a scene and alerting of activities of objects in the scene, including the procedures of acquiring a preliminary image of the scene, and marking and classifying at least one stationary object in the preliminary image, according to a predetermined classification, each of the classification types of the stationary objects having at least one visual behavioral characteristic. The method also includes acquiring a 2D image of the scene, and identifying objects in the 2D image with marked objects of the preliminary image. The method further includes determining if movement was detected in the image, and acquiring another image, if no movement detected. The method also includes determining if detected movement conforms to characteristics of classified stationary object associated with the movement, and acquiring another image, if the movement conforms to the characteristics of the stationary object. The method further includes alerting when the movement does not conform to the characteristics.

Description

SYSTEM AND METHOD FOR LONG-RANGE SURVEILLANCE OF A SCENE AND ALERTING OF PREDETERMINED UNUSUAL ACTIVITY

FIELD OF THE DISCLOSED TECHNIQUE

The disclosed technique relates to long-range surveillance systems, in general, and to methods and systems for alerting of predetermined unusual or exceptional activities in a scene under surveillance, with reduced false alerts probability, in particular.

BACKGROUND OF THE DISCLOSED TECHNIQUE

Automatic visual surveillance of a scene of interest is a security task approached in various manners. Some video surveillance systems produce an alarm for each detected movement of an object in the scene. Such systems usually employ a background model, wherein a deviation from the background model produces an alarm. Furthermore, such systems usually attempt to "teach" the computer to identify objects in the scene, according to typical object shape, appearance and movement direction. A main issue with visual surveillance systems is undesirable false alarms, (i.e., determining that a detected movement does not represent an alarming event, but rather an event that may be overlooked). Usually trying to reduce false alarms by "brut force" approaches increases the rate of undetected true alarms. Surveillance systems are known in the art.

US Patent Application Publication No. 2008/0002856 A1 to Ma et al., and entitled "Tracking System with Fused Motion and Object Detection" is directed to a system for fusing motion detection and object detection for tracking targets. The system includes a camera, a motion detection module, an object detection module, a motion likelihood image module, a model likelihood image module and a fusion module. The motion likelihood image module is coupled with the motion detection module. The model likelihood image module is coupled with the object detection module. The fusion module is connected to the motion likelihood image module and the model likelihood image module.

The camera acquires an image sequence of the area of surveillance. The motion detection module detects motion of a blob in the images, by background subtraction. The system compiles a model, having features characteristic for a category of an object in the area of surveillance. The object detection module detects an object from the images according to similarity of features of the object with the features of the model. Then, the fusion module fuses the motion of the blob and the object detection to form a hypothesis of a target. After validation of the target hypothesis, a tracking module tracks the target in the sequence of images.

An article entitled "3D Scene Modeling Using Sensor Fusion with Laser Range Finder and Image Sensor", by Ma et al. (Applied Imagery and Pattern Recognition Workshop, 2005. Proceedings, 34^th Volume, 19-21 Oct. 2005 Page(s):6) is directed to a method for generating a three dimensional (3D) model of a scene under surveillance. The 3D model is generated by fusion of laser range sensor and a single camera. An a priori model of the scene can be used by the surveillance module. The surveillance module includes video motion detection, video motion tracking, object classification and video analytic manager. Motion detection is achieved by separating background and foreground regions. Motion tracking is performed by frame-to-frame tracking using a set of heuristic data association rules, an occlusion handling method and a simplified particle filter. Object classification automatically classifies the object as "human", "vehicle" or "other" using statistical decision classifier. The classifier determines the object type based on a set of shape-, boundary-, and histogram-features, and their temporal consistency. After motion detection, motion tracking, and object classification, information is extracted of moving objects in the scene. The appearance information for detected objects is recorded, including instantaneous information of the spatial features of the object, and also temporal information on changes in the object size, direction of movement and speed.

An article entitled "Putting Objects in Perspective", by Hoiem et al. (IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2006) is directed to a method for placing local object detection in the context of the overall 3D scene by modeling the interdependence of objects, surface orientations, and camera viewpoint. The method considers the likely places in an image where an object (e.g., pedestrian) could be found. An estimation of the rough surface geometry in the scene is used to adjust probability of finding a pedestrian at a given image location. Also, an estimate of the camera viewpoint provides a scale of an object in the image. Then, objects are identified in the image, considering the probable locations for such objects in the scene, according to the surface geometry and camera viewpoint.

US Patent No. 7127083, issued to Han et al. and entitled "Video Surveillance System with Object Detection and Probability Scoring Based on Object Class", is directed to a system for identifying and tracking people in a scene and alerting of abnormal behavior of the people. The system computes a first set of trajectories of a first set of objects of a particular object class (i.e., people), hypothesized to have been moving through an area under surveillance at a previous point in time. The class of objects is distinguishable from other objects based on the objects physical appearance. Objects of that class hypothesized to be in the area under surveillance at a current point in time are identified, wherein some of those objects are identified independent of the physical appearance of other objects hypothesized to have been in the area at a previous time. The identification of the objects includes analyzing individual portions of a video image of the area under surveillance to determine if they have features that are characteristic of objects in the particular class using a neural network trained to recognize objects of that class. Top hypothesis is applied to produce alert. There is a software that analyzes top hypothesis with a view toward automatically identifying certain abnormal behaviors, or "alert conditions," that the system is responsible to identify based on predefined rules. An alert reasoning module generates an alert code if any of the predefined alert conditions appear to have occurred. In particular, based on the information in the top hypothesis, the alert reasoning module is able to analyze the behaviors of the objects, which are characterized by object counts, interactions, motion and timing. Thereby the module detects abnormal behaviors, particularly at sensitive zones, such as near the door zone or near the card reader.

The system can also include an unattended object module, which can determine from top hypothesis whether a non-human object appeared within the area under surveillance and was left there. This is detected by observing a change in the background information. Such an event may also be recorded in an activity recorder as following the alert rules and occurring with high likelihood, but as not being a violation to be recorded at a violation recorder in the database. A user such as a review specialist may query the database and access recorded events through a user interface for viewing at a monitor.

SUMMARY OF THE DISCLOSED TECHNIQUE

It is an object of the disclosed technique to provide a novel method and system for alerting of unusual or exceptional activity in a scene under surveillance, with reduced false alerts and increased probability of detection. In accordance with the disclosed technique, there is thus provided a method for surveillance of a scene and alerting of activities of objects in the scene, the method including the procedures of acquiring a preliminary image of the scene, marking and classifying at least one stationary object in the preliminary image, according to a predetermined classification. Each of the types of the stationary objects in the classification having at least one visual behavioral characteristic. The method further includes the procedure of acquiring at least one 2D image of the scene, and automatically identifying objects in the 2D image with classified and marked objects of the preliminary image. The method also includes the procedure of determining if movement was detected in the acquired 2D image, and returning to the procedure of acquiring at least one 2D image, if it is determined that no movement is detected. The method further includes the procedure of determining if the detected movement conforms to the visual behavioral characteristics of a classified stationary object associated with the movement, and returning to the procedure of acquiring at least one 2D image, if it is determined that the detected movement conforms to the visual behavioral characteristics of the classified stationary object. The method also includes the procedure of producing an alert when the detected movement does not conform to the visual behavioral characteristics of the classified object.

In accordance with the disclosed technique, there is thus provided a visual behavioral object including spatial characteristics, associated with the spatial configuration of a stationary object, and allowed visual behavioral characteristics, associated with behavioral actions which are allowed with relation to the spatial characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed technique will be understood and appreciated more fully from the following detailed description taken in conjunction with the drawings in which:

Figure 1 is a schematic illustration of a surveillance system, constructed and operative in accordance with an embodiment of the disclosed technique;

Figure 2A is a schematic illustration of a portion of a scene image, in accordance with another embodiment of the disclosed technique;

Figure 2B is a detailed schematic illustration of a portion of the image of Figure 2A;

Figure 2C is a detailed schematic illustration of another portion of the image of Figure 2A; and

Figure 3 is a schematic illustration of a method for alerting of unusual or exceptional activity in a scene under surveillance, in accordance with a further embodiment of the disclosed technique.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The disclosed technique overcomes the disadvantages of the prior art by providing a method and system for alerting of unusual or exceptional activity in a scene under surveillance, with reduced false alerts and increased probability of detection. A user manually marks and classifies various stationary objects appearing in a pre-acquired three- dimensional (3D) model or two-dimensional (2D) image of the scene, according to a predetermined classification. The classification includes different types of stationary objects. The user also forms a database of classifications of typical behaviors of moveable objects, which are objects likely to move during surveillance of the scene (e.g., various animals). When movement is detected, an internal alarm is triggered (not yet disclosed to the operator), initially considered to be a presumed (or suspected) alarm. Stationary objects are objects which pertain to the background of the scene and may be divided into stationary non-moveable objects, (i.e., objects which are not likely to move during surveillance, e.g. a house), and stationary moveable objects (i.e. objects which are stationary but are likely to move to a certain degree during surveillance, e.g. a tree).

Each type of stationary objects in the classification has a set of visual behavioral characteristics, representing possible visual behavior of that object type. Visual behavioral characteristics may be, for example, spatial behavioral characteristics, relating to movement associated with a stationary object (i.e., movement of, within or around an object). Alternatively, visual behavioral characteristics may be, for example, spectral behavioral characteristics, relating to changes in the wavelength (or set of wavelengths) emitted from an object (i.e., change in one color or more). Specifically, spectral changes in the appearance of an object may occur in wavelengths outside of the visible spectrum.

During surveillance of the scene, an image sensor acquires subsequent two-dimensional images of the scene. Since the location of the image detector is known, features of the two-dimensional image may be identified with features of the 3D model of the scene. Thus, objects which appear in the acquired two-dimensional images are identified according to their corresponding marked and classified objects of the 3D model. In case the pre-acquired image is a 2D image, the 2D image may be acquired by the same image detector of the system, thereby allowing attributing of features in the pre-acquired image and the real-time images.

The system detects movement in the acquired images. The system determines if the movement is associated with a stationary object. The movement may be associated with a marked and classified object, or independent of any marked object. When the movement is independent

(i.e., unassociated) of any classified object, the system compares the movement to the classification of unmarked moveable objects (i.e., in the database). If the movement conforms to a visual behavior from the moveable classification database, then the system does not produce an alert to the user. However, if the movement does not conform to a behavior of the moveable classification database, the system produces an alert to the user, directing her to the location of the detected movement.

When movement is detected, which is associated with a stationary object, the system determines whether this movement conforms to the visual behavioral characteristics of the object. If the movement does not conform to the visual behavioral characteristics, an alert is produced to notify the user of the unusual activity associated with that object.

When movement is detected and conforms to the visual behavioral characteristic, the system does not produce an alert, but rather tags the detection as a potential threat (i.e., "suspicious"), and acquires at least another subsequent two-dimensional image of the scene. The system then detects the movement in the subsequent acquired images, in case the movement exceeds the visual behavioral characteristics. Only if the movement exceeds the visual behavioral characteristics, will the system produce the alert. In this manner, the system does not dismiss the initial detection of the movement as a false alert, but keeps track (i.e., by tagging the detection as suspicious) in case the movement becomes unusual (i.e., genuine alert), which requires an alert. Thus, the system reduces the false/true alert ratio, by reducing false alerts, without reducing simultaneously the true alerts.

Reference is now made to Figure 1 , which is a schematic illustration of a surveillance system, generally referenced 80, constructed and operative in accordance with an embodiment of the disclosed technique. System 80 includes an image sensor 82, a user interface 84, an object behavior analyzer 86, a classification database 88, a visual behavior database 90 and a memory 92. Object behavior analyzer 86 is coupled with user interface 84, image sensor 82, memory 92, classification database 88 and with visual behavior database 90. Image sensor 82 is located in front of a scene 94, at a line distance d from scene 94. Image sensor 82 has a field of view 85, which covers a part of the span of scene 94, defined as sensor footprint 96. Image sensor 82 scans scene 94, such that the location of sensor footprint 96 within scene 94 changes during scan. Image sensor 82 may thus have a dynamic Line Of Sight (LOS), allowing reference to objects in scene 94, located in variable distances from image sensor 82. Classification database 88 is further coupled with visual behavior database 90.

Classification database 88 includes a list of stationary object types (not shown) and moveable object behaviors, which are likely to be found in scene 94. The list of stationary object types of classification database 88 may include, for example, the following object types: house, tree, building, lake, bench, window, door, chimney, electricity pole, light post, water tank, bridge, bus station, road, path, railway, bush and the like. Moveable objects behaviors may be, for example, behaviors of a person, an animal, a vehicle and the like.

Each stationary object type has at least one corresponding visual behavior characteristic (not shown), which is stored in visual behavior database 90. The visual behavior characteristic represents a possible allowed behavior associated with the corresponding object type (e.g., movement of-, within- or around the object, change in color, change in size). The distance d between image sensor 82 and scene 94 may range from a few meters (e.g., 50-100m) to a few kilometers (e.g., 5km-30km), depending on the one hand on the requirement of system 80 and the capabilities of image sensor 82, and on the other hand on the particular location of sensor footprint 96 within the scene 94. System 80 is employed for alerting on unusual or exceptional activity in a scene under surveillance. Prior to the operation of system 80, a high resolution 3D model (not shown) of scene 94 is acquired, for example, by an airborne image sensor (e.g., mounted on an airplane, helicopter or satellite). Alternatively, a pre-acquired 2D image may be acquired. The pre-acquired 3D model or 2D image of the scene includes representations of objects present in scene 94. A user (not shown) views the pre-acquired 3D model or 2D image, and employs a user interface (not shown) to mark and classify the stationary objects appearing in the 3D model or 2D image, according to the object types of classification database 88 (Figure 1 ). The user also employs the user interface to form the database of typical behavior classifications of moveable objects, which are likely to move during surveillance of the scene (e.g., various animals).

Reference is further made to Figures 2A, 2B and 2C. Figure 2A is a schematic illustration of a portion of a scene image, generally referenced 100, in accordance with another embodiment of the disclosed technique. Figure 2B is a detailed schematic illustration of a portion of the image of Figure 2A. Figure 2C is a detailed schematic illustration of another portion of the image of Figure 2A. During operation of system 80, image sensor 82 (Figure 1 ) acquires two-dimensional images, such as image 100, of scene 94 (Figure 1 ). Since the location of image sensor 82 is static and accurately known (e.g., via GPS), and since the 3D model of scene 94 is a high resolution model, features of two-dimensional image 100 may be identified and accurately superimposed (i.e., attributed) to the corresponding features of the 3D model of scene 94 such that it is possible to identify their common boundaries. Thus, objects which appear in the acquired two-dimensional images, such as image 100 (elaborated below), are identified according to their corresponding objects of the 3D model, previously manually marked and classified (corresponding to procedures 150 and 152 in Figure 3, herein below).

With reference to Figure 2A, image 100 includes a plurality of stationary objects, which are identified with marked objects of the pre-acquired 3D model or 2D image. Such objects are, for example, a plurality of houses 102_!, 102₂, a plurality of buildings 108 and 112, a plurality of trees 104^ 104₂, 104₃, 104₄ and 104₅, a park bench 106, a lake 110 and a road 120. Image 100 is a two-dimensional (2D) image, about the X-Y plane of a 3D coordinate system 126.

With further reference to Figure 2B, a portion 122 of image 100 is shown in greater detail. The user has manually marked the detailed features of certain objects in the 3D model or the 2D image, in a preliminary stage (see regarding Figure 3 herein below), features which are now identified with the same features in image 100. For example, in house 102₂₎ windows 130^ 130₂, 130₃, 130₄, 130₅, door 132 and chimney 133. With further reference to Figure 2C, a portion 124 of image 100 is shown in greater detail. The features of trees 104₄ and 104₅ are identified with the marked features of the 3D model or the 2D image, for example, tree trunks 134₄ and 134₅, and upper portions (i.e., branches and foliage) 136₄ and 136₅, respectively. Image 100, with the identified markings of the various objects and features therein, is stored in memory 92 of object behavior analyzer 86 (Figure 1 ).

During surveillance of scene 94, image sensor 82 acquires at least another image (not shown) of scene 94, subsequent to image 100. The subsequent image may be acquired after a relatively long period of time (e.g., a few hours or days), or acquired relatively soon after the previous image (e.g., within a few milliseconds or seconds). Object behavior analyzer 86 detects movement associated with an object in the other image. Object behavior analyzer 86 determines the type of object with which the detected movement is associated. In case of a stationary object, object behavior analyzer 86 determines if the detected movement does not conform to any of the behavioral characteristics of the associated object.

When the movement is independent (i.e., unassociated) of any classified object, object behavior analyzer 86 compares the movement to the classification of unmarked moveable objects (i.e., in the database). If the movement conforms to a behavior of the moveable behavior classification database, then the system does not produce an alert to the user, but rather tags the detection as a potential threat ("suspicious"), and directs image sensor 82 to acquire another image of the scene. For example, a bird flying over a fair background (e.g., sky) would be a behavior classification for a bird (i.e., allowed behavior characteristics). However, if the movement does not conform to a behavior of the moveable classification database, the system produces an alert to the user, directing her to the location of the detected movement.

For example, an allowed visual behavioral characteristic for a window, such as window 130^ may be a spatial characteristic, such that movement occurs only within the spatial limits of the window frame (e.g., when a person passes by the window inside the house). Thus, when system 80 detects movement of an object, or other changes, like an additional object left within the limits of the window frame of window 130_! (e.g., in a spatial location 140 of Figure 2B), this movement is considered to conform to the visual behavioral characteristics of window 130_!. However, when system 80 detects movement on an object in the vicinity of window 130_! outside of the window frame (e.g., in a spatial location 142, when a person steps out of the window or places an object outside of the window), then this movement is considered not to conform to the visual behavioral characteristics thereof. In such a case, object behavior analyzer 86 alerts the user of system 80 that a movement associated with window 130_! does not conform to the visual behavioral characteristics of window 130_!. The user can then observe the detected movement outside of the window, in further subsequent images (e.g., video stream) to visually track the movement. The object behavior analyzer may also activate an automatic tracking module, to track the detected movement in subsequent images of the scene.

It is noted, that when movement is detected in spatial location 140, which is within the limits of the window frame of window 130^ object behavior analyzer 86 does not produce an alert for the user but rather tags the detection as a potential threat (i.e., "suspicious" activity), and directs image sensor 82 to acquire another image of the scene. Object behavior analyzer 86 then detects the movement within window 130_! in the further acquired images, in case it exceeds the window frame of window 130_!. Only after the movement exceeds the window frame (i.e., the movement no longer conforms to a visual behavioral characteristic), will object behavior analyzer 86 produce the alert. In this manner, object behavior analyzer 86 does not dismiss the initial identification of the movement within window 130i as a false alert, nor does object behavior analyzer 86 declare it a real target, but keeps track of the suspicious movement in case the movement becomes unusual (i.e., genuine alert), which requires an alert. In this way false alarms are avoided and real alerts are not missed.

For further example, a visual behavioral characteristic for a tree, such as tree 104₅ (Figure 2C), may be that only upper portion 136₅ spatially moves within a predetermined frame 149 (Figure 2C), (e.g., under windy conditions), while tree trunk 134₅ stays in its fixed spatial location 138. Thus, when the tip 144 of tree 104₅ is detected in a spatial location 145, which is within frame 149, i.e., within the reasonable range for movement of upper portion 136₅, and tree trunk 134₅ is still located in position 138, this movement is considered to conform to the visual behavioral characteristics of tree 104₅. In this case, object behavior analyzer 86 does not produce an alert for the user but rather tags the detection as "suspicious", and directs image sensor 82 to acquire another image of the scene (e.g., in order to make sure a person is not hiding behind the tree). Object behavior analyzer 86 then detects the movement within frame 149 in the further acquired images, in case the movement exceeds frame 149 (e.g., if a person who was hiding behind the tree moves out).

As depicted in dashed lines in Figure 2C, the tip of tree 104₅ is detected to move to a spatial location 148, which is outside frame 149, i.e., outside of the reasonable range for movement of upper portion 136₅, and tree trunk 134₅ has moved to a position 146 (i.e., spatially different than position 138). In such a case, this movement of tree 104₅ is considered as non conforming to the visual behavioral characteristics of tree 104₅, indicating movement in an exceptional manner. Then, object behavior analyzer 86 alerts the user of system 80 that a movement associated with tree 104₅ does not conform to the spatial behavioral characteristics thereof. The user can then observe the area of tree 104₅, which was detected as a moving object, in further subsequent images to visually track the movement thereof, and visually investigate the area.

Each stationary object has spatial characteristics (i.e., its spatial representation and form), which is associated with the spatial configuration of a stationary object. The stationary object has allowed visual behavioral characteristics, associated with behavioral actions which are allowed with relation to the spatial characteristics of that stationary object. According to another embodiment of the disclosed technique, the spatial characteristics and the allowed visual behavioral characteristics together define a visual behavioral object. With reference to the example of Figure 2A and 2B, a visual behavioral object would include spatial characteristics, which are the frame of window 130^ and the allowed visual behavioral characteristics thereof, which include movement within the frame of the window. With reference to the example of Figure 2A and 2C, a visual behavioral object would include the spatial characteristics, which are the form of tree 104₅, and the allowed visual behavioral characteristics thereof, which include movement within frame 149. For further example, a visual behavioral object may include a spatial characteristics, which are the form of a railway (not shown), and the allowed visual behavioral characteristics thereof, which would include movement of a train on and along the railway.

In a preferred embodiment of the disclosed technique, the image sensor is mounted on a pan and tilt unit, located in a fixed location (e.g., on a mast). Thus, the image sensor scans the scene, observes the moveable footprint within the same scene and acquires subsequent images thereof. Typically, such an image sensor may be positioned in the same location for a relatively long time (e.g., a few years to tens of years). Therefore, the user may review an updated image of the scene after every predetermined time period (e.g., a few months, a year, a few years), in order to update the marking and classification of new objects in the scene (e.g., new buildings, trees, roads, and the like).

Reference is now made to Figure 3, which is a schematic illustration of a method for alerting of unusual or exceptional activity in a scene under surveillance, in accordance with a further embodiment of the disclosed technique. The method includes a "preliminary stage" which relates to the procedures taken prior to surveillance of the scene, and an "operational stage", which relates to the procedures taken during surveillance of the scene.

In procedure 150, a 3D model or a preliminary 2D image of a scene is acquired. With reference to Figures 1 and 2A, prior to the operation of system 80, a 3D model or a preliminary 2D image (not shown) of scene 94 is acquired, for example, by an airborne image sensor (e.g., mounted on an airplane, balloon or satellite). The 3D model of the scene includes representations of objects present in scene 94. In procedure 152, at least one stationary object is manually marked and classified in the 3D model or the preliminary 2D image, according to a predetermined classification. Each of the types of the stationary objects in the classification has at least one visual behavioral characteristic. With reference to Figures 1 and 2A, a user (not shown) views the 3D model or the preliminary 2D image, and employs a user interface (not shown) to mark and classify the objects appearing in the 3D model or the preliminary 2D image, according to the object types of classification database 88 (Figure 1 ). The user also employs the user interface to form the database of classifications of typical behaviors of moveable objects, which are likely to move during surveillance of the scene (e.g., various animals). Each stationary object type has at least one corresponding visual behavior characteristic, which is stored in visual behavior database 90. The visual behavior characteristic represents a possible behavior associated with the corresponding object type (e.g., movement of-, within- or around the object, change in color, change in size). Procedures 150 and 152 are considered as the "preliminary stage" of the method depicted in Figure 3. The preliminary stage is conducted prior to actual surveillance of the scene. The preliminary stage may be repeated (i.e., refreshed) when significant stationary changes in the scene occurred (i.e., updated when necessary, normally very seldom).

In procedure 154, at least one 2D image of the scene is acquired, and objects in the 2D image are identified and associated, when possible, with classified and marked objects of the 3D model or the preliminary 2D image. Other objects in the images remain unassociated objects. With reference to Figures 1 and 2A, during operation of system 80, image sensor 82 (Figure 1) acquires two-dimensional images, such as image 100, of scene 94 (Figure 1). Since the location of image sensor 82 is known (e.g., via GPS), features of two-dimensional image 100 may be identified with features of the 3D model of scene 94. Thus, objects which appear in the acquired two-dimensional images, such as in image 100 (elaborated herein above), are identified according to their corresponding marked and classified objects of the 3D model, and are associated with the 3D model. During surveillance of scene 94, image sensor 82 acquires at least another image (not shown) of scene 94, subsequent to image 100. The subsequent images may be acquired after a relatively long period of time (e.g., a few hours or days), or acquired relatively soon after the previous image (e.g., within a few milliseconds or seconds).

In procedure 156, it is determined whether movement is detected in the acquired two-dimensional images. With reference to Figures 1 and 2A, object behavior analyzer 86 detects movement associated with a stationary object in the 2D image. If it is determined that movement is not detected, then the method depicted in Figure 3 returns to procedure 154 for acquiring another image of the scene.

If it is determined that movement is detected in the acquired 2D image, then the method depicted in Figure 3 proceeds to procedure 166. In procedure 166, the type of object is determined, with which the movement is associated. With reference to Figures 1 and 2A, object behavior analyzer 86 determines the type of object with which the detected movement is associated.

If the detected movement is associated with an unassociated object, the method of Figure 3 proceeds to procedure 164. In procedure 164, it is determined if the detected movement conforms to a visual behavioral characteristic of a moveable object With reference to Figures 1 and 2A, in case of a moveable object, object behavior analyzer 86 compares the movement to the classification of unmarked moveable objects (i.e., in the database) to determine if the movement conforms to a visual behavior from the moveable classification database. If it is determined that the detected movement conforms to a visual behavior of a moveable object, then the method depicted in Figure 3 returns to procedure 154 for acquiring another image of the scene. With reference to Figures 1 , If the movement conforms to a behavior of the moveable classification library, then the system does not produce an alert to the user, but rather tags the detection as a potential threat ("suspicious"), and directs image sensor 82 to acquire another image of the scene. For example, when a bird is flying over a fair background (e.g., sky). Object behavior analyzer 86 then detects the movement within further acquired images, in case the movement exceeds the behavior of the moveable object.

If the detected movement is associated with an associated stationary object, the method of Figure 3 proceeds to procedure 158. In procedure 158, it is determined if the detected movement conforms to the visual behavioral characteristics of the associated stationary object. With reference to Figures 1 , 2A and 2B, object behavior analyzer 86 determines if the detected movement does not conform to any of the behavioral characteristics of the associated stationary object.

If it is determined that the detected movement conforms to the visual behavioral characteristics, then the method depicted in Figure 3 returns to procedure 154 for acquiring another image of the scene. With reference to Figures 1 , 2A and 2B, when movement is detected in spatial location 140, which is within the limits of the window frame of window 130^ object behavior analyzer 86 does not produce an alert for the user but rather tags the detection as "suspicious", and directs image sensor 82 to acquire another image of the scene. Object behavior analyzer 86 then detects the movement within window 1 in the further acquired images, in case it exceeds the window frame of window 130_!.

If it is determined that the detected movement does not conform to the visual behavioral characteristics of the stationary object or the unassociated moveable object, then the method depicted in Figure 3 proceeds to procedure 160. In procedure 160, an alert is produced when the detected movement does not conform to the visual behavioral characteristics of the associated stationary object or the unassociated moveable object. With reference to Figures 1 , 2A and 2B, when system 80 detects movement on an object in the vicinity of window 130i outside of the window frame (e.g., in a spatial location 142, when a person steps out of the window or places an object outside of the window), then this movement is considered not to conform to the visual behavioral characteristics thereof. In such a case, object behavior analyzer 86 alerts the user of system 80 that a movement associated with window 130i does not conform to the visual behavioral characteristics of window 130^

In procedure 168, the detected movement is analyzed. For example, a user utilizing the system of Figure 1 receives an alert regarding a detected movement, and analyzes the movement in the acquired images. Subsequently, the method of Figure 3 returns to procedure 154 for acquiring another image of the scene. The operational stage of the method is iteratively repeated to provide constant surveillance of the scene.

Procedures 154, 156, 166, 158, 160, 164 and 168 are considered as the "operational stage" of the method depicted in Figure 3. The operational stage is iteratively conducted during the actual surveillance of the scene.

According to a further embodiment of the disclosed technique, the user feeds the system with input images or video streams, showing cases where the system has made wrong decisions regarding identified movements in the acquired images. The user then alerts the system that it has made a wrong decision, and defines these cases as "wrong" to the system. The user may also include instructions for a correct response to the case. When a similar case would reoccur, the system would follow the correct response scheme. Furthermore, during analysis of detected movement in the acquired images, the user may find that one or more behavioral characteristics were inaccurately defined (e.g., a window frame defined too large, and the like). In such a case, the user may return to the classification database and the visual behavior database and modify the relevant definition, so that it be more accurately defined, thus improving the accuracy of identifications, and reducing false alerts or missed alerts. Thereby, the system may be considered to be in a "continuously learning mode", adjusting its response to various cases. It is noted that the user may employ the user interface in order to modify the definitions of the relevant behavioral characteristics, without causing any physical change or modification to any of the components of the system (i.e., while retaining the same system configuration).

According to an alternative embodiment of the disclosed technique, a system for alerting of unusual or exceptional activity in a scene under surveillance (such as system 80 of Figure 1) may be mounted on a moving platform. Such a moving platform may be a cart moving on a fixed rail or a freely moving vehicle. The position of the platform, on which the system is mounted, is known at each moment in time (e.g., from a GPS system or an INS system or both). Thus, it is possible to calculate the position of the image sensor of the system relative to the scene under surveillance, provided that the whole scene observable by the movable system is contained within the previously acquired 3D model or 2D image.

Prior to the surveillance, when the image sensor is fixed in a known position in front of the scene, the user manually marks, defines and classifies the objects in the scene, as described herein above. During surveillance, when the image sensor is moving, it acquires images of the scene from various angles, depicting different views of the scene. The current position of the image sensor relative to the original fixed position is known. Since the first image of the scene is a 3D image, the analyzer can generate a current marked image of the scene, based on the markings on the first image. Thus, the analyzer compares a currently acquired image of the scene (taken from a different position than the first image), with a generated image view with marking of objects therein, corresponding to the current location of the image sensor. Then the analyzer may detect movement and determine if the movement does or does not conform with a spatial behavioral characteristic of the associated object, as described herein above with reference to Figures 1 , 2A, 2B and 2C.

It will be appreciated by persons skilled in the art that the disclosed technique is not limited to what has been particularly shown and described hereinabove. Rather the scope of the disclosed technique is defined only by the claims, which follow.

Claims

1. A method for surveillance of a scene and alerting of activities of objects in the scene, the method comprising the procedures of:

acquiring a preliminary image of said scene;

marking and classifying at least one stationary object in the preliminary image, according to a predetermined classification, each of the types of said stationary objects in the classification having at least one visual behavioral characteristic;

acquiring at least one 2D image of the scene, and automatically identifying objects in said 2D image with classified and marked objects of the preliminary image;

determining if movement was detected in said acquired 2D image, and returning to said procedure of acquiring at least one 2D image, if it is determined that no movement is detected;

determining if said detected movement conforms to said visual behavioral characteristics of a classified stationary object associated with said movement, and returning to said procedure of acquiring at least one 2D image, if it is determined that said detected movement conforms to said visual behavioral characteristics of said classified stationary object; and

producing an alert when said detected movement does not conform to said visual behavioral characteristics of said classified object.

2. The method of claim 1 , further comprising the procedure of analyzing said detected movement in the acquired images and returning to said procedure of acquiring at least one 2D image of the scene.

3. The method of claim 1 , wherein said procedure of marking and classifying is performed manually.

4. The method of claim 1 , wherein said procedure of marking and classifying further includes classifying typical behaviors of moveable objects,

said method further comprising the procedure of determining if said detected movement conforms to a visual behavior classification of a moveable object, and returning to said procedure of acquiring at least one 2D image, if it is determined that said detected movement conforms to a visual behavior classification of said moveable object,

said procedure of producing an alert further includes producing an alert when said detected movement does not conform to said visual behavior classification of a moveable object.

5. The method of claim 1 , further comprising the procedure of tagging said detection of movement as a potential threat, and returning to said procedure of acquiring at least one 2D image for further monitoring of said movement.

6. The method of claim 1 , wherein said preliminary image is a 2D image of the scene.

7. The method of claim 1 , wherein said preliminary image is a 3D model of the scene.

8. The method of claim 2, further comprising the procedures of:

identifying inaccurately defined behavioral characteristics of a respective stationary object, according to said analysis of movement in said acquired 2D images; and

modifying said inaccurately defined behavioral characteristics, according to said analysis and said identification.

9. A visual behavioral object comprising:

spatial characteristics, associated with the spatial configuration of a stationary object; and

allowed visual behavioral characteristics, associated with behavioral actions which are allowed with relation to said spatial characteristics.