WO2008105841A2

WO2008105841A2 - Robotic sentry with low dispersion acoustic projector

Info

Publication number: WO2008105841A2
Application number: PCT/US2007/021241
Authority: WO
Inventors: Curtis E. Graber
Original assignee: Graber Curtis E
Priority date: 2006-10-05
Filing date: 2007-10-03
Publication date: 2008-09-04
Also published as: US8248473B2; WO2008105841A3; US20080084787A1

Abstract

A system of remotely locatable sentries or watchtowers (1 OA, 10B, 10C) support monitoring cameras (32, 34) which operate over a network (120) to allow a relatively small number of operators to monitor a wide geographic area (55). The sentries support non-lethal, sound-based instrumentalities (11) for discouraging entry into and passage through a secured zone (53) by an identified object (58).

Description

ROBOTIC SENTRY WITH LOW DISPERSION ACOUSTIC PROJECTOR

Technical Field

[001] The invention relates to wide area surveillance and is specifically directed to a robust, outdoor automated sentry equipped both for monitoring an area and for challenging unauthorized entry to the area under surveillance.

Background Art

[002] Unmanned, wide area video monitoring for perimeter or boundary control has become steadily more common in recent years as cameras and monitor components have declined in price. Data processing and network communication links are capable of handling large quantities of digitized, raw data. Video monitoring can work under both daylight conditions, using the usual visual bandwidth, and under low light conditions using the near infrared spectrum. Techniques for automated identification of object recognition have advanced greatly in recent years. The various techniques of object recognition require associating groups of pixels together as the "object". An object can be identified by shape (including a dynamically changing shape as might be produced by a flock of birds), form, and location. An identified object may have trajectory and velocity information associated therewith as "pixel metadata. See United States Patent No. 6,947,590.

[003] While monitoring of an area is well developed, most security systems have relied on directing a human to the locale of a detected intrusion. The lack of a non-lethal or non-crippling methods of repelling intruders has also hampered deployment of automatic or remotely controlled instrumentalities for discouraging encroachment.

Disclosure of the Invention

[004] The invention utilizes remotely locatable sentries, or watchtowers and masts, to support facilities for monitoring an area. The monitoring facilities typically operate as part of a network to secure a border or perimeter over a wide area. Monitoring facilities may include video and audio monitoring systems. Deterrent facilities, preferably including non-lethal, sound-based instrumentalities for discouraging entry into and passage through a secured zone, operate from the same support facilities as the monitoring facilities and are controllable over the network. Alternative non-lethal measures may include laser dazzlers or microwave devices.

[005] Sentries may be mobile or airborne. The towers are fixed positionally and serve as redoubts, intended to limit unauthorized tampering or disabling of surveillance and other equipment mounted on the tower. A tower, which is the preferred embodiment of the invention at the time of filing, increases the field of coverage of the sensors (e.g. cameras) by raising them well above the surrounding terrain. Local power may be provided by conventional, buried power lines if available or by solar cells and batteries or a generator. A computer, located either locally or located remotely and accessed over a network data link provides for automatic data capture for archiving and analysis of the images collected by cameras/imagers which cover fields of view. Objects passing into or through the field are located and characterized, preferably using algorithms such as trajectory algorithms which match trajectory behavior with a likely identification on the obejct. It is anticipated that the system will be trained for the detection of human infiltrators, taking into account local terrain, but the system could readily be applied to animal control.

[006] An acoustic projector is installed on an altazimuth mount allowing it to be freely aimed under the control of the computer or an operator. Based on the tentative identification and the location of the object, an automated message may be broadcast over the projector targeted on the object. Under automatic operation or operator control the acoustic projector may be used to hit the object with discomforting sound beams or microwave radiation in order to discourage further penetration of the zone into the secured area.

[007] Communication between the tower and the operator is over a bidirectional communication link between the computer and the operator station, which in some circumstances affords highly adaptive real time responses to incidents. The acoustic projector may include acoustic sensors allowing use of the acoustic projector as a listening device both to aid in automatic identification of a target, to allow automatic or operator communication, and even to allow dialogue with the target. The ability to eavesdrop on conversations allows an operator/agent to obtain information useful for dialogue between the agent and the target individuals.

Brief Description of the Drawings

[008] Fig. 1 is a perspective view of a watchtower for the invention.

[009] Fig. 2 is an elevation view of the watchtower of Fig. 1.

[0010] Fig. 3 is a front elevation of a camera stand and low dispersion, sound collector/projector used to implement monitoring and control features of the invention and located at the top of the watchtower of Fig. 1.

[0011] Fig. 4 is a top plan view of the camera stand of Fig. 3.

[0012] Fig. 5 is a map illustrating fields of view for a series of watchtowers.

[0013] Fig. 6 is a block diagram of the system of the invention.

[0014] Figs. 7A-E are high level flow charts illustrating a method of implementing the invention.

[0015] Fig. 8 is a cross-sectional view of a low dispersion, focusing sound collector and projector used with the invention.

[0016] Fig. 9 is a front elevation of the projector of Fig. 8.

[0017] Fig. 10 is a menu of operator options for manual control of a watchtower.

[0018] Fig. 11 is a perspective view of a microwave radiator dish suitable for installation on a top or on a mobile platform used in carrying monitoring elements of the invention.

[0019] Fig. 12 is a front plan view of a combination microwave radiator and acoustic energy radiator.

[0020] Fig. 13 is a cross sectional view of a combination microwave radiator and acoustic energy radiator of Fig. 12.

Best Mode for Carrying Out the Invention

[0021] Referring to Fig. 1 a watchtower 10 used to implement area monitoring and securing is shown. Watchtower 10 is anticipated to be one of plurality of towers disposed at approximately 1 kilometer intervals along a boundary or perimeter of a region to be secured. Watchtower 10 has three major sections, a reinforced base 12, a retractable mast 23 supported on and extending upwardly from the reinforced base 12, and a top 30 located at an elevated location on the retractable mast 23. Top 30 carries the local monitoring equipment, deterrent devices, wireless communication instrumentalities and data processing elements for a broad area security system.

[0022] Reinforced base 12 has an interior which is accessed through a port 20. Reinforced base 12 may be used to house a power supply such as batteries or a diesel generator. Reinforced base 12 may be further protected by placing bollards 22 around the reinforced base in a protective constellation.

[0023] The present invention provides deterrence features which are capable of automatic or manual operation. The deterrence instrumentalities are one or more energy projectors such as energy projector 11 installed on top 30. While in theory various types of "energy" may be projected, the preferred embodiment of the present invention utilizes acoustic energy, and the projector is an acoustic projector such as described in a copending International Patent Application for an Acoustic Energy Projection System, appl. No. PCT/US2007/014011 , filed 14 June 2007, which is incorporated herein by reference.

[0024] The present system provides for detection of intruders. Positionally fixed cameras 32 and a movable targeting camera 34 are the principal tools used to implement such detection. Energy projector 11 and targeting camera 34 are supported on an altazimuth mount 36 which is in turn supported directly or indirectly from top 30. Directly supported by top 30 are the fixed cameras 32 arranged to provide coverage of an area (the field of view) adjacent or near to watchtower 10. Top 30 may also carry solar panels (not shown) if such are used as part of the power supply system, a spotlight (a detection aid, not shown), which would be added to the altazimuth mount 36, a laser dazzler (a deterrent, not shown), a focused microwave projector of millimeter wavelength (as a substitute or a supplement to the sound projector) and a housing 40 for local data processing and wireless communication equipment. Local data processing and wireless communication equipment may also be located in a mast cap 46 (Shown in Fig. 2A) directly below top 30 or within the energy projector 11.

[0025] Figs. 2A-B illustrate possible locations for equipment used with watchtower 10. A power supply 44, such as batteries or a diesel engine, is located protected in reinforced base 12. Upper and middle mast sections 28, 26 nest in base mast section 24 as indicated by arrow "R". A mast retraction and raising system 42 is located within mast base section 24. Energy projector 11 is preferably an acoustic energy projector base on a cone reflector 14. Figure 38 illustrates the relative size of an average person against the size of the watchtower 10.

[0026] Figs. 3 and 4 provide more detailed views of the top 30, and the arrangement of equipment on the top. A laser dazzler 48 or spotlight can be positioned on an altazimuth mount 36 as shown. A microwave projector could be located centered in an LDAP projector, substituting for the inner cone 14. Fixed cameras 32 are installed on mounts 49 which allow for adjustment on installation to take into account terrain of the area to be covered. Generally four fixed cameras 32 are used to provide a field of view (FOV) with an approximately 180 degree arc around the watchtower. Projector 11 is covered by a screen 50 which obscures the internal arrangement of the projector and thereby renders more difficult any action to render the projector inoperable. The projection axis of projector 11 is parallel to the center of view of targeting camera 34.

[0027] Fig. 5 illustrates border and perimeter control using a group of watchtowers 10A-C. Watchtowers 10A-C have a wireless connection 67 to a monitoring station 68, which may be remotely located and which is preferably manned. Watchtowers 10A-C have fields of view 60, 61 , 62, which are illustrated as overlapping, though it is not required that this be so, to provide surveillance over an area 55 intrusion into which by a person 58 is supposed to be detected. Area 55 is divided into zones 51 , 52, 53, although, depending upon the application, the number of zones could be increased or decreased. Zone 51 may be termed a warning zone. A person 58 moving through this zone along trajectory A might be periodically warned that they were approaching areas to which entry was forbidden them. Typically no direct action would be taken to exclude the person beyond periodic repetitions of the warning so long as the person did not cross into the next area. Warning frequency might be made dependent upon the degree of penetration into zone 51. A flock 57 of birds is shown moving through zone 52 on trajectory B. Data processing equipment could be programmed to recognize flocks as objects. Identification as an object does not necessitate a response. A object identified as a person in the same area would receive heightened warnings in this zone and ordered to retreat. Zone 53 may be termed an exclusion zone. Selected objects moving through this area would be subject to exclusion. A four legged animal 59 is shown in the area. If the system were applied to wildlife control around a major park it could be used to monitor movement of a predator, such as a wolf, leaving the park in the direction of agricultural areas, in which case the system could respond with non-lethal deterrent actions as described below. A contour line 56 is illustrated winding its way in and out of area 55. The system can use trajectory analysis as a way of identifying objects and as a way of characterizing objects as being of interest. The physical terrain covered by a given watchtower can, particularly if rugged, affect the trajectory of objects (e.g., relative bearing, acceleration, velocity) moving through the fixed cameras' field of view. Knowledge of local contour may control the selection of the trajectories of interest for watchtower 1OC but not 1OA, but it is not necessary that the trajectories of interest be determined be preprogrammed. They may be identified by operators based on experience.

[0028] Hardware support for the present invention can be configured in a variety of ways. Fig. 6 is a high level block diagram of an illustrative arrangement of a system 101 providing for such hardware support. A watchtower 10 may be equipped with a local computer 126 which receives power from a local, or secure commercial, power source 128. Local computer 126 generates control signals for altazimuth positioning motors 124 which allow deterrent devices 122, the low dispersion acoustic projector and sound collector (LDAP) 111 and the targeting camera 34 to be aimed. Control outputs to the deterrent devices and loudspeakers used for acoustic energy projection are applied by the local computer to deterrent devices 122 and LDAP 111. Sound collected by LDAP 111 microphones is in turn applied to local computer 126 as is the video signal generated by targeting camera 34. The targeting camera 34 has a telescopic zoom 130 under the automatic control of local computer 126. The remote operator may direct responses of the local computer 126 to the extent permitted by programming. Fixed cameras 32 also provide video signals to local computer 126. The video data may be archived locally or on a database server. The signals are subject to digital signal processing before application to the local computer 126.

[0029] It is anticipated that one human operator will be assigned groups of adjacent watchtowers 10. Such operators can be located remotely at an operator station 68 equipped with an operator station computer 140 and appropriate input and output devices 142, 144 allowing the operator to monitor anything picked up by the cameras and sound collecting system of a watchtower or sentry and to exercise control over the deterrent devices 122 such as the LDAP 111 acoustic energy projector. Inputs 142 will naturally include a pointing device and a keyboard, but will preferably include a microphone for generating audio messages to be transmitted by the LDAP 111 and may also include a joystick for aiming the systems supported on the altazimuth mount and for control of the zoom 130 feature.

[0030] System 101 provides a network to connect the local computers 126 of a group of watchtowers 10 to the operator station 68. Network 120 is illustrated as providing access to database servers 104, which may be located at the operator station 68, or dispersed among various agencies of the institution charged with control of the system. Database servers 104 may include object identification characteristics supporting the automatic determination by particular watchtowers 10 that an object entering the field of view (FOV) of fixed cameras 32 is an object of interest. Database servers 104 may provide facial and voice recognition systems allowing instant identification of known individuals entering an area. Other databases 104 might include one of criminal records, etc., as well as object qualification criteria.

[0031] Referring now to Figs. 7A-E, implementation of the system is exemplified by relation of the functions carried out by system 101 to a flow chart. System 101 , and more particularly a given watchtower 10 within system 101 , normally operates in an automatic mode until detection of an event which triggers an alarm for an operator. Essentially fixed cameras generate of an image for a field of view which is scanned for motion of objects within the FOV (step 202). As long as object motion is not detected (step 204, the NO branch) the routine cycles through the object and motion detection operations without spawning of a sub-process. Once motion of an object is detected (illustrated as the YES branch from step 204), a sub- process is spawned and the object tagged (step 206). The detection of "object motion" may appear to assume that an object is identified before its motion can be detected, however, motion of an object may be inferred from value changes of a group of localized pixels before "object" identification has occurred. Thus step 204 can be taken as simultaneous qualification of changing pixel values used for both object and motion detection.

[0032] It is intended that system 101 provide a tentative identification of objects, first by associating groups of pixels in the image with one another as an object based on pixel data and metadata than characterization of the object by shape, size, form, and trajectory. Steps 208, 210 and 212 refer to characterization and identification of the object. Information accumulated about an object includes its shape, form, size (from range) and its trajectory. All of these object characteristics may be compared to a library of objects previously determined to be of possible interest. Because terrain effects can alter measurable characteristics of images of physically like objects, such as typical trajectories, databases of objects may be limited to a locale, even if stored on a remote server. The characteristics should be defined as ranges to avoid generation of an overly large database.

[0033] Development of the object database for a locale is expected to be heuristic. That is, there will initially be few objects in the database, and objects will be qualified based on operator input. Accordingly, step 214 provides for comparison to the library of an object for which data and metadata has been generated. If no match is generated step 216 is executed to alert an operator and the object is made the current focus of the local watchtower(s) which have it "in sight", that is the targeting camera 34 is used to track the unidentified object. At step 218 the object is identified as being either "of interest" or "not of interest". Next, at step 220 the routine branches between objects of interest and those not of interest. Objects that were identified at step 214 are also handled at step 220. Objects of interest follow A to Fig. 7B. Other objects result in the sub-process following B to Fig. 7C.

[0034] Objects, once identified, can be considered as logically separate from the field of view as indicated at step 222. It is possible that more than one object of interest is moving through the field of view, so a count is kept of the number of objects at step 224. One reason for this is that, as the system is presently configured, only one targeting camera 34 is provided per tower, though it would be possible to provide more than one. At step 226 an operator is notified of the detection of an object of interest, as well as the location. If the object count is greater than one, and only one targeting camera is available it is necessary to set (and repeatedly reset) one object as the current focus which the targeting camera will track. Thus at step 228, if the count is not greater than one, the focus is set to the only current object of interest in the field of view (step 230). If the object of interest count is greater than one a focus object must be selected following the YES branch from step 228. At step 236 a subroutine to select the focus is called, and, after a focus has been returned, or set at step 230, the process moves on to step 234 to determine if a focus object has made an incursion into a surveillance or exclusion area. If No incursion is detected path "C" is followed from step 234. If an incursion has been detected the YES branch is followed from step 234 to step "D".

[0035] Turning to Fig. 7C, processing of a object which is not of interest (step 238) is considered. Such an object is excluded from the field of view at step 240 until the conditions for terminating the subprocess, such as prolonged non-movement, or long repeated cyclic movement (such as a tree blowing in the wind), cause a time out at step 242.

[0036] The process of Fig. 7D represents the process called at step 236 to determine a focus from a plurality of objects of interest. At step 244 the individual objects are scored by selected criteria, such as proximity to the area under control, or, if trajectory considerations are used, estimated time until the object will clear the zone under surveillance in the direction of an area which the object is to excluded from. The focus is set to the highest scoring object, which is returned to the calling process. The process of Fig. 7D continues as a background process. Following. step 246 step 248 is executed to determine if the process has been interrupted by operator action. If yes a delay loop (step 250) is entered along the NO branch from step 248. If not the NO branch is followed to step 252 to execute the object termination conditions. If all objects remain viable the NO branch is followed from step 252 to step 244 and to rescore of the objects of interest. If an object has met the termination conditions, the YES branch is followed to step 254 to terminate the object and decrement the count of objects. At step 256 it is determined if a plurality of objects of interest remain in the field of view, and if so, the process loops back to rescore the surviving objects. If the count is less than two the subprocess is terminated until recalled.

[0037] Fig. 7E reflects the handling of movement of objects of interest into and through the various zones of Fig. 6. These steps follow after detection of an incursion in step 234. At step 258 the remote operator is alerted. The operator may intervene at any time (indeed no alert is required), however, operator assumption of control is reflected by a decision box at step 260. The NO branch from step 260 reflects the operator leaving the system on automatic mode. The process of the Fig. 7E is entered only after an incursion into at least zone 1 (or the least restrictive zone). Therefore it is only necessary to determine if the object has progressed into zones 2 or 3, or retreated. [0038] At step 262 the range to the object is taken. This may be accomplished in a number of ways, for example, if an object is in an area of two intersecting fields to view relative bearings from two towers may be taken to obtain a range by triangulation. More typically a laser range finder, a radar system or an echo location system mounted on the altazimuth mount is employed. Range markers may even be placed in the field of view. Once a range is taken it can be determined if the object of interest is inside the exclusion zone or not. If not, the NO branch is taken and a warning issued at step 280. After a delay another range to the object of interest is taken to determine if the object has retreated outside of the warning zone (step 282). If Yes the process can be exited. If No, the process returns to step 262 to determine if the exclusion zone has been entered. Once the exclusion threshold is crossed and the YES branch followed from step 262 it is determined if the object has crossed into the response zone. If NOT, the NO branch is followed from step 264 to step 268 and the threat level is raised to reflect the increased threat level represented by the object. Automatic responses at this point may include the projection of uncomfortable sound toward the target or use of a laser dazzler. Following use of the deterrent appropriate to the level of incursion, it is determined if the object has retreated at step 270. If YES, and the process returns to step 262 to determine if the object continues to retreat. If the object does not retreat the process loops back to step 264 to determine if penetration of the area progresses. If YES is followed from step 264 to step 266, the object has moved into a response zone and the highest level deterrent is deployed.

[0039] Returning now to the circumstance where an operator assumes control along the YES branch from step 260. Step 272 simply notes that options are displayed to the operator and responses taken as the operator instructs. Step 274 following step 272 reflects return to automatic control, either by the operator forcing the system not to follow a target or back into the automatic response mode.

[0040] Referring to Figs. 8 and 9 an acoustic radiator and collector incorporates a reflector with an inner cone surface and an outer annular reflecting surface suitable for use in the system of the present invention. The particular configuration is by way of example only, and it is not the only such system which could be used. However, the use of an inner cone reflector is favored because, conventionally, sound focusing system are based on a simple concave dish with the transducer element mounted centered in the dish. Such systems are more vulnerable to damage by gunfire than the present system. The cone reflecting system allows more acoustic energy to be input into the system. The inner reflecting surface 82 is provided by the cone reflector 14, which is preserved from the first embodiment of the invention. A second, outer reflecting surface 84 is provided by a forward concave annular ring 16. Outer reflecting surface 84 is preferably parabolic in its sections, but differs from a conventional parabolic dish in that the bases of the parabolic sections to not meet at a single point in the base of the dish, but instead surround an annular gap in which cone reflector 14 may be placed. Outer reflective curve 84 can be fitted to curves other than parabolic curves. A radial speaker/microphone housing 18 is located centered under the inner cone 14. The housing 18 may be moved in and out (double headed arrow F) parallel to the radiating axis of the system to change the range to its focal point forward from the dish. Outer ring 74 is an array of microphones for collecting sound. Where the system is used for controlling the movement of people across a border it is believed that it could be used to eavesdrop on people at great distances, and an operator could then listen to any conversation, possibly to the extent of learning the identity of the people, or at least the names they are using among themselves. The operator could then use the system literally to talk to the individuals on a first name basis. If only an inner cone is used, microphones may be interspersed with the transducers. Housing 78 tucked under the cone 14 is used to house system electronics.

[0041] A radiator/collector used in the present system may take many forms, but the form illustrated has several advantages. Where more than one reflecting surface is used, the radiant axes of the surfaces are coincident.

[0042] Fig. 10 reflects a menu of options which may selected from by an operator. As mentioned already the system can allow the operator to eavesdrop (listen) or speak. He/she can select identity checks, including criminal records if available. The deterrent systems available can be displayed. Any other options available may be shown. For example, a system including GPS on response vehicles may be able to display response times. Search lights may be made available, etc. As will now be apparent to those skilled in the art, video cameras may be provided in infrared as well as visible light. The system may be automatically programmed to direct a variety of irritating sounds or microwave radiation toward an intruder. [0043] Microwave projectors have been proposed which operate in a frequency range where the microwave radiation causes a burning sensation to an animal exposed thereto without actual tissue damage. A microwave projector 300 may be built based on similar collecting principals proposed for the acoustic projectors which the preferred energy projection device used here. An example of such a system in shown in Fig. 11 , where a microwave projector 300 is constructed based on a ring 305 of microwave generators 304, oriented inwardly to direct energy against a central spike 302 for common reflection forward from the spike along a projection axis.

[0044] Referring to Figs. 12 and 13, a microwave projector 300 may be combined with an acoustic projector as a mixed energy type projector 340. The ring of microwave generators 304 are located in an inwardly oriented ring backed up against a plurality of outwardly oriented acoustic projectors 312 which direct sound energy into a conic section ring 310 for projection on the same axis as the microwave energy. Base units 316 are placed under the rings of microwave generators 304 and acoustic projectors 312.

Claims

Claims:

1. Boundary control system with a sentry station having video imaging facilities, a operator station including image displays and controls for directing or focusing the video imaging facilities on the sentry station, and a communication link between the operator station and the sentry station, characterized in that field of view cameras (32) define a covered area (60, 61 , 62) and a targeting camera (34) is supported on a steerable mount (49) allowing isolation on and tracking of an object moving through the covered area (60, 61 , 62), the targeting camera (34) and a directed energy projector (11) are supported on the steerable mount (49) and the directed energy projector (11) is oriented to have a projection axis aligned with the targeting camera (34).

2. Boundary control system according to claim 1 , characterized in that data processing systems and libraries (104, 126) provide for automated identification of objects (57, 58, 59) within the covered areas (60, 61 , 62).

3. Boundary control system according to claim 2, characterized in that an operator station computer (140) provides for alerting an operator upon identification of a focus object (58).

4. Boundary control system according to claims 1 or 3, characterized in that the directed energy projector 11 is a sound projector.

5. Boundary control system according to claim 4, characterized in that the directed energy projector (11) operates as a sound pick up device and loudspeaker, and the communication system (120) provides two way communication between the operator station (68) and the sentry station (10A, 10B, 10C) to allow an operator to eavesdrop on focus objects (58) and potentially to direct audible communication with the focus objects (58).

6. Boundary control system according to claim 4, characterized in that the directed energy projector (11) may be used to as a loudspeaker.

7. Boundary control system according to claims 5 or 6, characterized in that the sentry stations (1OA, 1OB, 10C) include masts 10 with supporting redoubts 12 and tops (30) supporting the directed energy projector (11), field of view cameras (32) and targeting camera (34).

8. Boundary control system according to claim 1 , characterized in that the directed energy projector (11) may include a laser dazzler (48) or a microwave source.