WO2002071685A1

WO2002071685A1 - Digital watermarking and maps

Info

Publication number: WO2002071685A1
Application number: PCT/US2002/006858
Authority: WO
Inventors: Geoffrey B. Rhoads; Trent J. Brundage; Neil E. Lofgren; Philip R. Patterson; Lorie R. Clements
Original assignee: Digimarc Corporation
Priority date: 2001-03-05
Filing date: 2002-03-05
Publication date: 2002-09-12
Also published as: US7506169B2; US20080123154A1; US8127139B2; US20100016016A1; US7249257B2; US7992004B2; US20080089550A1; US20020147910A1

Abstract

Digital watermarking technology herein, is described in a four step process, of (figure 4, S1-S4), and is used in conjunction with map dat, such as is acquired by satellite and other sensors and may be generated from image and ground truth databases (extract watermark location information) (figure 4, element S1). The second step, S2, determines a physical location (e.g., GPS). The third step of (figure 4, element S3), compares the location information with the physical location. The fourth step of (figure 4, S4), provides feedback of the comparison.

Description

DIGITAL WATERMARKING AND MAPS

FIELD OF THE INVENTION The present invention relates to digital watermarking, and more particularly relates to use of watermarking in applications relating to the compilation, synthesis, indexing and use of map data.

BACKGROUND AND SUMMARY OF THE INVENTION Acquisition of aerial imagery traces its history back to the Wright brothers, and is now commonly performed from satellite and space shuttle platforms in addition to aircraft. While the earliest aerial imagery relied on conventional film technology, a variety of electronic sensors are now more commonly used. Some collect image data corresponding to specific visible, UV or

IR frequency spectra (e.g., the MultiSpectral Scanner and Thematic Mapper used by the Landsat satellites). Others use wide band sensors. Still others use radar or laser systems (sometimes stereo) to sense topological features in 3 dimensions. The quality of the imagery has also constantly improved. Some satellite systems are now capable of acquiring image and topological data having a resolution of less than a meter. Aircraft imagery, collected from lower altitudes, provides still greater resolution.

For expository convenience, some embodiments of the present invention are particularly illustrated in the context of a Digital Elevation Model (DEM). A DEM, essentially, is an "elevation map" of the earth (or part thereof). One popular DEM is maintained by the U.S. Geological Survey and details terrain elevations at regularly spaced intervals over most of the U.S. More sophisticated DEM databases are maintained for more demanding applications, and can consider details such as the earth's pseudo pear shape, in addition to more localized features. Resolution of sophisticated DEMs can get well below one meter cross-wise, and down to centimeters or less in actual elevation. DEMs - with their elevation data - are sometimes supplemented by albedo maps (sometimes termed texture maps, or reflectance maps) that detail, e.g., a grey scale value for each pixel in the image, conveying a photographic-like representation of an area.

There is a large body of patent literature that illustrates DEM systems and technology. For example: U.S. Patent 5,608,405 details a method of generating a Digital Elevation Model from the interference pattern resulting from two co-registered synthetic aperture radar images.

U.S. Patent 5,926,581 discloses a technique for generating a Digital Elevation Model from two images of ground terrain, by reference to common features in the two images, and registration mapping functions that relate the images to a ground plane reference system. U.S. Patents 5,974,423, 6,023,278 and 6,177,943 disclose techniques by which a Digital

Elevation Model can be transformed into polygonal models, thereby reducing storage requirements, and facilitating display in certain graphics display systems.

U.S. Patents 5,995,681 and 5,550,937 detail methods for real-time updating of a Digital

Elevation Model (or a reference image based thereon), and are particularly suited for applications in which the terrain being mapped is not static but is subject, e.g., to movement or destruction of mapped features. The disclosed arrangement iteratively cross-correlates new image data with the reference image, automatically adjusting the geometry model associated with the image sensor, thereby accurately co-registering the new image relative to the reference image. Areas of discrepancy can be quickly identified, and the DEM/reference image can be updated accordingly.

U.S. Patent 6,150,972 details how interferometric synthetic aperture radar data can be used to generate a Digital Elevation Model.

From systems such as the foregoing, and others, a huge quantity of aerial imagery is constantly being collected. Management and coordination of the resulting large data sets is a growing problem. In accordance with some embodiments of the present invention, digital watermarking technology is employed to help track such imagery, and can also provide audit trail, serialization, anti- copying, and other benefits.

Digital watermarking, a form of steganography, is the science of encoding physical and electronic objects with plural-bit digital data, in such a manner that the data is essentially hidden from human perception, yet can be recovered by computer analysis. In physical objects, the data may be encoded in the form of surface texturing, or printing. Such marking can be detected from optical scan data, e.g., from a scanner, optical reader, input device, digital camera, or web cam. In electronic objects (e.g., digital audio or imagery — including video), the data may be encoded as slight variations in sample values. Or, if the object is represented in a so-called orthogonal domain (also termed "non-perceptual," e.g., MPEG, DCT, wavelet, etc.), the data may be encoded as slight variations in quantization values or levels. The assignee's U.S. Patent No. 6,122,403 and U.S. Application No. 09/503,881 (published as WO0152181) are illustrative of certain watermarking technologies.

Digital watermarking systems typically have two primary components: an encoder that embeds the watermark in a host media signal, and a decoder that detects and reads the embedded watermark from a signal suspected of containing a watermark (e.g., a suspect signal). The encoder embeds a watermark by altering the host media signal. The decoder component analyzes a suspect signal to detect whether a watermark is present. In applications where the watermark encodes information, the decoder extracts this information from the detected watermark. U.S. Application No. 09/503,881 (WO0152181), discloses various encoding and decoding techniques. United States Patent No. 5,862,260 discloses still others. The analysis of the detected data can be accomplished in various known ways. Presently, most steganographic decoding relies on general purpose microprocessors that are programmed by suitable software instructions to perform the necessary analysis. Other arrangements, such as using dedicated hardware, reprogrammable gate arrays, or other techniques, can of course be used.

In accordance with some embodiments of the present invention, watermarking is performed in stages, at different times. For example, a unique identifier can be watermarked into an image relatively early in the process, and other information (such as finely geo-referenced latitude/longitude) can be watermarked later. A single watermark can be used, with different payload bits written at different times. (In watermark systems employing pseudo-random data or noise (PN), e.g., to randomize some aspect of the payload's encoding, the same PN data can be used at both times, with different payload bits encoded at the different times.).

In other embodiments of the invention, different watermarks can be applied to convey different data. The watermarks can be of the same general type (e.g., PN based, but using different PN data). Or different forms of watermark can be used (e.g., one that encodes by adding an overlay signal to a representation of the image in the pixel domain, another that encodes by slightly altering DCT coefficients corresponding to the image in a spatial frequency domain, and another that encodes by slightly altering wavelet coefficients corresponding to the image. Of course, other watermarking techniques may be used as suitable replacements for those discussed above.). In some multiple- watermarking embodiments of the invention, a first watermark is applied before a satellite image is segmented into patches. A later watermark can be applied after segmentation. (The former watermark is typically designed so as to be detectable from even small excerpts of the original image.)

A watermark is applied by an imaging instrument in some embodiments of the invention. For example, the image may be acquired through an LCD optical shutter, or other programmable optical device, that imparts an inconspicuous patterning to the image as it is captured. (One particular optical technique for watermark encoding is detailed in U.S. Patent No. 5,930,369.). Or the watermarking can be effected by systems in a satellite (or other aerial platform) that process the acquired data prior to transmission to a ground station. In some systems, the image data is compressed for transmission — discarding information that is not important. The compression algorithm can discard information in a manner calculated so that the remaining data is thereby encoded with a watermark.

A ground station receiving the satellite transmission can likewise apply a watermark to the image data. So can each subsequent system through which the data passes, if desired.

Preferably, such watermarking processes are secure and cannot be replicated by unauthorized individuals.

In accordance with still other embodiments of the present invention, digital watermarking techniques are used to ease navigation and map orientation. In some embodiments, digital watermarking techniques are combined with GPS systems. Other embodiments serve fields such as government work and field reconnaissance, commercial or recreational boating, hiking, mountaineering, travel, orienteering, geography, education, exploration, entertainment, sight seeing, etc.

In accordance with still other embodiments of the present invention, incoming imagery is automatically geo-referenced and combined with previously-collected data sets so as to facilitate generation of up-to-date DEMs and maps.

In accordance with yet other embodiments of the present invention, a digital watermark is employed as an enabler to access a related family of images, linked in a database (or other data structure) via digital watermark identifiers. Watermark identifiers can also be used to identify the source of an image, track images and documents, document a distribution chain, and identify unlabeled hard copy images. According to another aspect, digital watermarks help to provide security, monitoring and gatekeeper-like functions. In accordance with yet other embodiments of the invention, auxiliary data is steganographically embedded within an image to permit locations of points within the image to be determined.

In accordance with still other embodiments of the invention, data imagery, including images having unique features, is pieced together using embedded data or data indexed via embedded data. In accordance with still another aspect of the present invention, a so-called "geovector" is carried by or indexed with a digital watermark.

The foregoing and additional features and advantages of the present invention will be more readily apparent from the following detailed description, which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 illustrates a map, which is divided into blocks. Fig. 2 illustrates various components of a watermark reading device. Fig. 3 illustrates the device of Fig. 2 in relation to the map of Fig. 1. Fig. 4 is a flow diagram illustrating a method according to one embodiment of the present invention.

Fig. 5 is a functional block diagram illustrating a digital watermarking process. Fig. 6 illustrates components of an image management system.

Fig. 7 illustrates associating related images and information with a digital watermark identifier. Fig. 8 is a functional block diagram illustrating gatekeepers in a network.

Figs. 9 and 10 are flow diagrams illustrating gate-keeping methods and processes according to the Fig. 8 embodiment.

Fig. 11 shows a (blank) image on which different grids and indicia relating to one embodiment of the invention are shown. Fig. 12 illustrates imagery, which is segmented into image patches.

Figs. 13a and 13b illustrate a correlation of image patches.

Fig. 14 is a flow diagram illustrating an image management method according to one aspect of the present invention.

Fig. 15 is a flow diagram illustrating a method of embedding a geovector in image data. Fig. 16 is a flow diagram illustrating a method of decoding an embedded watermark to access a database.

DETAILED DESCRIPTION First Set of Embodiments (For expository convenience, the following specification focuses on satellite and aerial

"imagery" to illustrate the principles of the invention. The principles of the invention, however, are equally applicable to other forms of aerial surveillance data and other topographic/mapping information. Accordingly, the term "image" should be used to encompass all such other data sets, and the term "pixel" should be construed to encompass component data from such other data sets.) When new aerial imagery is received, it is generally necessary to identify the precise piece of earth to which it corresponds. This operation, termed "georeferencing" or "geocoding," can be a convoluted art and science.

In many systems, the georeferencing begins with a master reference system (e.g., latitude and longitude) that takes into account the earth's known deformities from a sphere. Onto this reference system the position of the depicted region is inferred, e.g., by consideration of the satellite's position and orientation (ephemeris data), optical attributes (e.g., resolution, magnification, etc.) of the satellite's imaging system, and models of the dispersion/refraction introduced by the earth's atmosphere.

In applications where precise accuracy is required, the foregoing, "ephemeris," position determination is refined by comparing features in the image with the placement of known features on the earth's surface (e.g., buildings and other man-placed objects, geological features, etc.) and compensating the georeference determination accordingly. Thus, for example, if the actual latitude and longitude of a building is known (e.g., by measurement from a ground survey - "ground truth"), and the corresponding latitude and longitude of that building as indicated in the georeferenced satellite imagery is different, the reference system applied to the satellite data can be altered to achieve a match. (Commonly, three or more such ground truth points are used so as to assure accurate correction.)

Ground-truthing is a tedious undertaking. While computer methods can be used to facilitate the process, the best ground truth correction of imagery generally requires some human involvement. This is impractical for many applications. Let us consider the basic principle of cost/meter as a useful metric, and imagine that various applications for exploiting satellite data are willing to pay different amounts in order to achieve given levels of geocoding accuracy. The following disclosure hypothesizes that there are ways (possibly novel, alluding to the idea that the author lacks detailed knowledge of the state of the art, and presumes no novelty nor lack thereof) to utilize all collected satellite data, properly identified and stored as a huge intercorrelated reference system - itself anchored by ground truth data - as a means to automatically geocode incoming raw pixels to the massive overall data set. The accuracy of this automated geocoding would hopefully be higher than that obtainable from ephemeris-type systems alone, but would probably be less accurate than "manually instigated" precision geocoding based directly on ground truth. The hope and goal would be that a lower core cost/meter geocoding accuracy could be achieved.

Such a system may involve the following elemental components:

1) An ideal sphere with an arbitrary time origin (as the starting point for the DEM model)

2) A time-evolving DEM 3) A time-evolving master-correlate albedo texture map

3A) A finite layered index map, organizing current raw data contributors to map

4) Ground Truth Data

5) Nominal ephemeris data per contiguous datastream The ongoing automation process includes:

1) Creating initial sphere, DEM, and texture map using existing ground truth

2) Creating a layered index map 3) Each newly acquired datastream is cloud-masked, DEM-projection-and refraction-corrected

4) The masked-corrected data - using nominal ephemeris data as a starting point - is correlated to a master DEM/albedo map, itself projected along nominal ephemeris

5) The quality of the new data is evaluated, and incrementally added to the master albedo map and index map if it is deemed acceptable 5 A) a pseudo infinite impulse response (based on time and quality of data) in coming up with current albedo map pixel value (omnidirectional pixel value)

At the core of building the albedo-map (and also the DEM) is the need to always track its inputs for several reasons: • redundant checking for accuracy and veracity of inputs;

• indexing of what data is contributing to the master albedo map;

• coordination of data from similar or even vastly different sources, all contributing to either the master maps or to existing relational databases.

As detailed below, watermarking can play an important role in the achieving these objects. The foregoing will be clearer from the following.

Consider an illustrative DEM system with a 10 meter horizontal resolution, and featuring continual refresh and georeferencing. At two bytes per pixel, and a model size of 4M by 2M pixels, the model comprises 16 Terabytes of data. The albedo map is on the same order of resolution, with the same data storage requirements. The database storing this information desirably is arranged to easily graph necessary correlation scenes.

Presume an existing master DEM and albedo map. These may have been formed by a dozen or more redundant component data sets (e.g., aerial images, ground surveys), acquired over the previous days, months or years, that have been composited together to yield the final DEM/map ("model").

As shown in Fig. 12, aerial imagery can be segmented into area sets (e.g., image "patches"). These patches can be pieced together (or "composited") in a quilt-like manner to form a master map. (A "master" map is used generally herein to represent a map or other area representation, typically which will include a plurality of image patches. Image patches are defined broadly and may include image segments, photographs, separate images, etc.). An image patch may include imagery representing an area, such as a 1 x 1 meter area, a 1 x 1 kilometer area, etc. Often, an image patch is combined with adjacent patches, which were gathered on different dates. For example, an image taken last week (e.g., Patch C in Fig. 1) may be quilted together with image patches taken today (e.g., Patch B), or a year ago (e.g., Patch A), to form a larger area map. Also, patches may be replaced over time to reflect new area developments or movements. (Of course, a master map need not be physically pieced together, but may instead be electronically maintained by a computer database, which correlates the patches or stores information, e.g., coordinates, patch locations, etc.).

Similarly, image patches can be pieced together with other images taken from different aerial platforms (e.g., satellites, airplanes, unmanned aircraft, etc.) or taken with different imagery characteristics. (Imagery characteristics may include resolution, angle, scale, rotation, skew, time, azimuth, device characteristics, altitude, attitude, physical conditions such as cloud cover and magnification, etc.)

Images typically undergo auto-correlation processes to reconcile differences between adjacent patches, prior to being composited (or arranged) with other patches. A variety of known mathematical techniques can be utilized in this operation, including dot product computation, transforming to spatial frequency domain, convolution, etc. In a lay sense, the correlation can be imagined as sliding one map over the other or matching pieces in a puzzle-like fashion until the best registration between the two image patches is obtained.

Now imagine a new satellite image is acquired corresponding to part of the region represented by the master model. The particular terrain depicted by the satellite image can be inferred from ephemeris and other factors, as noted above. By such techniques, the location of the depicted image on the earth's surface (e.g., the latitude and longitude of a point at the center of the image) may be determined within an error of, say 5 - 500 meters. This is a gross geo-referencing operation.

Next a fine geo-referencing operation is automatically performed, as follows. An excerpt of the master model is retrieved from the database - large enough to encompass the new image and its possible placement error (e.g., an area centered on the same latitude/longitude, but extending 250 meters further at each edge). A projective image is formed from this master DEM/map excerpt, considering, e.g., the satellite's position and atmospheric effects, thereby simulating how the master model would look to the satellite, taking into account — where possible - the particular data represented by the satellite image, e.g., the frequency bands imaged, etc. (The albedo map may be back-projected on the 3D DEM data in some arrangements to augment the realism of the projective image.)

The projective image formed from the master DEM/map excerpt differs somewhat from the image actually acquired by the satellite. This difference is due, in part, to the error in the gross georeferencing. (Other differences may arise, e.g., by physical changes in the region depicted since the master DEM/map was compiled.) The projective image is next automatically correlated with the satellite image. A variety of known mathematical techniques can be utilized in this operation, including dot product computation, transforming to spatial frequency domain, convolution, etc. In a lay sense, the correlation can be imagined as sliding one map over the other until the best registration between the two images is obtained. From the correlation operation, the center-to-center offset between the excerpt of the master DEM/map, and the satellite image, is determined. The satellite image can thereby be accurately placed in the context of the master model. Depending on system parameters, a fine placement accuracy of, e.g., between 5 cm and 5 meters (i.e., sub-pixel accuracy) may be achieved.

(In some embodiments, affine transformations can be applied to the satellite data to further enhance the correlation. E.g., particular geological or other features in the two data sets can be identified, and the satellite data (e.g., map or image) can then be affine-transformed so that these features correctly register.)

With the satellite image thus finely geo-referenced to the master DEM/map, it can be transformed (e.g., resampled) as necessary to correspond to the (typically rectilinear) reference system used in the master model, and then used to refine the data represented in the model. Buildings or other features newly depicted in the satellite image, for example, can be newly represented in the master model. The master model can be similarly updated to account for erosion and other topological changes revealed by the new satellite image.

Part of the finely geo-referenced satellite data may be discarded and not added to the master model, e.g., due to cloud cover or other obscuring phenomena. The remaining data is assessed for its relative quality, and this assessment is used in determining the relative weight that will be given the new satellite data in updating the master model.

In one embodiment, the finely geo-referenced satellite data is segmented into regions or area sets, e.g., rectangular patches corresponding to terrain 1000 meters on a side, and each patch is given its own weighting factor, etc. In a system with 10 meter resolution (i.e., a pixel size of 10m², the patch thus comprises an array of 100 x 100 pixels. (In some embodiments, the fine geo-referencing is done following the segmentation of the image, with each patch separately correlated with a corresponding area in the master model.) Each patch may take the form of a separate data file.

When the new satellite data is added to update the master model, old data may be discarded so that it no longer influences the model. Consider an area that is imaged monthly by a satellite. Several months' worth of image data may be composited to yield the master model (e.g., so cloud cover that obscured a region in the latest fly-over does not leave part of the model undefined). As each component image data gets older, it may be given less and less weight, until it no longer forms any part of the master model. (Other component data, in contrast, may be retained for much longer periods of time. Map information collected by ground surveys or other forms of "ground truth" information may fall into this category.)

The master model may be physically maintained in different ways. In one exemplary arrangement, a database stores the ten sets of data (e.g., acquired from different sources, or at different times) for each 1000 x 1000 meter patch. When interrogated to produce a map or other data, the database recalls the 10 data sets for each patch, and combines them on the fly according to associated weighting factors and other criteria (e.g., viewing angle) to yield a net representation for that patch. This composite patch is then combined (e.g., graphically stitched) with other adjoining, similarly-formed composite patches, to yield a data set representing the desired area.

In another embodiment, the component sets of image data are not separately maintained. Rather, each new set of image data is used to update a stored model. If the new image data is of high quality (e.g., good atmospheric seeing conditions, and acquired with a high resolution imaging device), then the new data maybe combined with the existing model with a 20/80 weighting (i.e., the existing model is given a weight four-times that of the new data). If the new image data is of low quality, it may be combined with the existing model with a 5/95 weighting. The revised model is then stored, and the new data needn't thereafter be tracked.

(The foregoing examples are gross simplifications, but serve to illustrate a range of approaches.)

The former arrangement - with the component data stored - is preferred for many applications, since the database can be queried to yield different information. For example, the database can be queried to generate a synthesized image of terrain as it would look at a particular time of day, imaged in a specified 1R frequency band, from a specified vantage point.

It will be recognized that a key requirement - especially of the former arrangement - is a sophisticated data management system. For each data set representing a component 1000 x 1000 meter patch stored in the database, a large quantity of ancillary data (meta data) must be tracked. Among this meta data may be a weighting factor (e.g., based on seeing conditions and sensor attributes), an acquisition date and time (from which an age-based weighting factor may be determined), the ID of the sensor/satellite that acquired that data, ephemeris data from the time of acquisition, the frequency band imaged, the geo-referenced position of the patch (e.g., latitude/longitude), etc., etc. (Much of this data may be common to all patches from a single image.)

Classically, each component source of data to the system (here referred to as an "image" for expository convenience) is associated with a unique identifier. Tapes and data files, for example, may have headers in which this identifier is stored. The header may also include all of the meta data that is to be associated with that file. Or the identifier can identify a particular database record at which the corresponding meta data is stored. Or hybrid approaches can be used (e.g., the header can include a file identifier that identifies a data base record, but also includes data specifying the date/time of data acquisition).

In the final analysis, any form of very reliable image identification may suffice for use in such a system. The header approach just-discussed is straightforward. Preferable, however, is to embed one or more identifiers directly into the image data itself (i.e., "in band" steganographic encoding using digital watermarking). A well-designed watermarking name-space can in fact become a supra-structure over several essentially independent serial numbering systems already in use across a range of satellite sources. Moreover, rudimentary georeferencing information can actually be embedded within the atermark name-space. For example, on initial acquisition, an initial watermark can be applied to satellite imagery detailing the ephemeris based gross georeferencing. Once the image has been finely georeferenced, the existing watermark can either be overlaid or overwritten with a new watermark containing the georeferencing information (e.g., "center lat: N34.4324352, long: W87.2883134; rot from N/S: 3.232; x2.343, y2.340, dxθ.123, dy493, etc."). These numbers essentially encode georeferencing info including projective and atmospheric distortions, such that when this image is DEM-projection corrected, high accuracy should be achieved.

Another way to explain the need for watermarking might be the following: Pity the first grade teacher who has a class of young upstarts who demand a lengthy dissertation on why they should simply

Corrupted TIFF IMAGE: no OCR available

As indicated, the watermark(s) can identify the imaging system, the date/time of data acquisition, satellite ephemeris data, the identity of intervening systems through which the data passed, etc. One or more watermarks can stamp the image with unique identifiers used in subsequent management of the image data, or in management of meta data associated with the image. A watermark can also serve a function akin to a hyperlink, e.g., as detailed in application

09/571,422 (published as WO00/70585). For example, a user terminal can permit an operator to right- click on a region of interest in a displayed image. In response, the system can respond with a menu of options - one of which is Link Through Watermark(s). If the user selects this option, a watermark detection function is invoked that decodes a watermark payload from the displayed image (or from a portion of the image in which the operator clicked). Using data from the decoded watermark payload, the terminal interrogates a database for a corresponding record. That record can return to the terminal certain stored information relating to the displayed image. For example, the database can present on the terminal screen a listing of hyperlinks leading to other images depicting the same area. By clicking on such a link, the corresponding image is displayed. Or the database can present, on the user terminal screen, the meta-data associated with the image.

In some embodiments, watermarks in component images may carry-through into the master DEM/map representation. If an excerpt of the master DEM/map is displayed, the user may invoke the Link Through Watermark(s) function. Corresponding options may be presented. For example, the user may be given the option of viewing each of the component images/data sets that contributed to the portion of the master model being viewed.

(It will be recognized that a variety of user interface techniques other than right-clicking, and selecting from a menu of options thereby displayed, can be employed. That interface is illustrative only.) In some embodiments, a watermark can be applied to each DEM/map from the master database as it is retrieved and output to the user. The watermark can indicate (i.e., by direct encoding, or by pointing to a database record) certain data related to the compiled data set, such as the date/time of creation, the ID of the person who queried the database, the component datasets used in preparing the output data, the database used in compiling the output data, etc. Thereafter, if this output data is printed, or stored for later use, the watermark persists, permitting this information to be later ascertained.

Watermarks can be applied to any data set (e.g., a satellite image, or a map generated from the master database) for forensic tracking purposes. This is particularly useful where several copies of the same data set are distributed through different channels (e.g., provided to different users). Each can be "serialized" with a different identifier, and a record can be kept of which numbered data set was provided to which distribution channel. Thereafter, if one of the data sets appears in an unexpected context, it can be tracked back to the distribution channel from which it originated. Some watermarks used in the foregoing embodiments can be "fragile." That is, they can be designed to be lost, or to degrade predictably, when the data set into which it is embedded is processed in some manner. Thus, for example, a fragile watermark may be designed so that if an image is JPEG compressed and then decompressed, the watermark is lost. Or if the image is printed, and subsequently scanned back into digital form, the watermark is corrupted in a foreseeable way. (Fragile watermark technology is disclosed, e.g., in applications 09/234,780 (W09936876), 09/433,104 ( 00133495),

09/498,223, 60/198,138 (WO0180169), 09/562,516, 09/567,405 (WO0186579), 09/625,577

(WO0209019), 09/645,779, and 60/232,163.) By such arrangements it is possible to infer how a data set has been processed by the attributes of a fragile watermark embedded in the original data set. Assuming that early testing proves out that the addition of "watermarking energy" into the normal workflow of satellite imaging systems does not materially disturb the function of most of the output of that system, nevertheless certain "watermark removal" tools can be built to alleviate any problems in cases where unacceptable impact is identified. This can either be a generic tool or one highly specialized to the particular application at hand (perhaps employing secret data associated with that application). In a second generation system (without too much fanfare) a fairly simple "remove watermark before analyzing this scene" function could be automatically included within analysis software such that 99% of image analysts wouldn't know or care about the watermarking on/off/on/off functionality as a function of use/transport.

Second Set of Embodiments

With reference to Fig. 13a and 13b, watermarks can assist in correction or correlating imagery characteristics (e.g., such as scale, rotation, resolution, skew, time-matching, etc.). For example, an embedded watermark payload may indicate the angle of the imaging device (e.g., optical camera, imaging sensor, etc.), the height to the imaging device, the relative position (e.g., skew, rotation, etc.) of the device with respect to a target area, and the resolution of the device and image. (Such measurements can be provided from sensing and positioning equipment on board or in communication with the aerial platform. Such characteristics may be alternatively determined by the georeferencing techniques discussed above. Of course, other imagery characteristic determining techniques may be suitably interchanged with the present invention.). Returning to Figs 13a and 13b, imagery characteristics provide information to help manipulate patches A and B (Fig. 13a) into a standardized or compatible format (Fig. 13b). Information pertaining to the imaging characteristics can be used to improve and expedite the auto-correlation processes discussed above. In addition, once the imaging characteristics are known, straightforward processing can manipulate an image patch to conform to adjacent patches (or to the map itself). For example, some or all of the patches in a master map are mathematically manipulated to achieve the same scale, orientation, and/or resolution.

With respect to a watermark payload, the imaging characteristics can be directly encoded as the watermark payload. Alternatively, an index (or identifier) associated with a set of these characteristics may be encoded in the payload. To illustrate, a numerical code or index represents a set possible imagery characteristics (or a subset of such). The imagery characteristics are stored in a data base record. The code or index, once extracted from a watermark, is used to interrogate a database to obtain the corresponding data record. (As an alternative, a payload signifies a predetermined set of values, e.g., a payload of 1237 signifies a predetermined scale, rotation, skew, etc. Or the index relates to a predetermined range of characteristics. For example, the range may specify that the scale is in a particular range, or that the resolution falls within a range, etc.). A watermark payload size and complexity can be reduced with a database/index system.

Embedding imagery characteristic in the form of a digital watermark assists in downstream processing and handling of an image. An automated-quilting process can be employed to match patches according to the georeferencing and/or imagery characteristics provided by a digital watermark. These georeferencing and/or imagery characteristics can also serve to preserve the historic information about the image and depicted area. Individual patches can also be watermarked to include coordinates or master map locators. With reference to Fig. 12, patch E may include coordinates or a plurality of coordinates that identify its master map location, coordinates for corners or edges (e.g., either physical geo-coordinates or coordinates relative to its master map location), or its relationship with adjacent patches. Such a locator can be added once a master map is composited (e.g., by watermarking the master map). Alternatively, such locators can be embedded before quilting, such as when imagery is collected or processed.

A time-tag (or stamp) may also be embedded in imagery. The time-tag can be used to categorize images based on time (e.g., hour, minutes, date, etc.), and to help identify stale or outdated imagery. The time-tag may optionally include a plurality of fields, such as time-taken, time processed, time integrating in a master map, etc. Such time-tagging methods improve management of images. (In one embodiment, an automated process searches a master map database, looking for stale or outdated patches, based on predetermined criteria. Once found the stale image patch is preferably removed and an updated image patch is inserted in its place.).

Fig. 14 illustrates a flow diagram of an inventive method according to one embodiment of the present invention. Image data is received into a system or process. The image data is embedded with image characteristics (step S10). Alternatively, the image is embedded with an identifier (index) for database interrogation. The embedded image data is then correlated or manipulated to conform to adjacent patches or to map requirements (step SI 1). In this regard, the correlation may either render adjacent patches to have approximate (e.g., similar or in a range of) imagery characteristics, or to have nearly identical imagery characteristics. Or the correlation may group neighboring patches into a set. A map is then generated or constructed (SI 2). (A map can be quilted together to include many image patches. The digital watermark identifiers are used to correlate the image.). Digital watermarking is now disclosed as a central element in a digital asset management system, particularly for photograph assets (including "digital images"). Copyright labeling, active copyright communications, marketing links, etc., have been explored in the watermark art. This section discloses how digital watermarking (and related database linking properties) and georeferenced photography inter-relate. In one embodiment, digital watermarking is used as a platform to simplify and transform georeferenced photography.

Within the universe of subject matter for photography is what is broadly referred to as "remote sensing." For this discussion, remote sensing is defined to include all types of photography, which somehow images the Earth's surface or its landscape. Of course, while remote sensing may be facilitated with aerial platforms, such is not required. Add to the remote sensing class, all photography, which somehow has an innate connection to a location on the Earth - referred herein as "georeferenced photography." In the final analysis, virtually all photographs, one way or another, have innate geographic properties. (Even purely synthetic images are created by an author located "somewhere."). Most photographs, including swept-scan satellite imagery and radar, also including vacation snaps at, e.g., Niagara Falls, can be described as having innate, if not explicit, geographic properties. "Time" can also be included as an identifying property. (To simplify the discussion, the terms photograph, image, and photography are used interchangeably hereafter. Such terms may include remote sensing, georeferenced photography, image, imagery, photos, electronic images or imagery and/or aerial photo, etc., etc.). Virtually all images can be referenced by a dimensional location vector (e.g., a "geovector") relative to the Earth's coordinate system. In one embodiment, the geovector is presented as a six (6) element vector, including:

Latitude; Longitude;

Height/ Altitude (e.g., as compared to a mean-sea level sphere with an arbitrary time origin);

Time (including date);

Cardinal Direction; and

Azimuth.

The cardinal direction and azimuth elements can be used to determine a viewpoint depicted in a photograph (e.g., the azimufhal direction of a viewpoint for a given geo-position.). In a modification, cardinal direction and azimuth indicate the vantage point of the imaging sensor. In still another modification, azimuth and cardinal direction are used to represent other directional indicators. Of course, the cardinal direction can be used to orient an image depicted in the photograph. (Although the term

"geovector" is introduced in connection with a six (6) dimensional vector, the present invention is not so limited. Indeed, a geovector is defined broadly herein to include information conveying location and/or directional specifying data, regardless of vector size.).

In a modification to the first embodiment, a geovector includes "6+1" elements. The extra "+1" dimension can be multi-dimensional in nature, generally representing "sensor geometry." Sensor geometry is defined broadly herein to include a coherent set of optical (or electrical) sampling functions, e.g., corresponding to each pixel (or pixel block) and or a microdensity region of a photograph. Of course, there is a variety of other types of sensor geometry, each associated with various rules on how the geometry is defined and how it affects the referencing parameters (e.g., latitude, longitude, height, etc.). A common form of sensor geometry is a rectangular fan or pyramid centered on a camera's aperture, which can be used as a stand-in for many others forms. Of course, there are many other geometry forms known to one of ordinary skill in the art, which are suitably interchangeable with the present invention.

The march of technological progress is transitioning more photography from the "innate" category to the "explicit" category through the use of global positioning system (GPS) technology and or local wireless technologies. GPS can be used to determine a physical location (e.g., including properties of a geovector). As will be appreciated by those skilled in the art, GPS is a satellite-based radio navigation system capable of providing continuous position, velocity, and time information. GPS receiver units receive positioning signals from a constellation of satellites deployed in various orbits about earth (e.g., 12-hour orbits). The satellites continuously emit electronic GPS signals (or telemetry) for reception by ground, airborne, or watercraft receiver units. By receiving GPS signals from a plurality of satellites, a properly configured receiver unit can accurately determine its position in at least three dimensions (e.g., longitude, latitude, and altitude/height). Some GPS systems also provide compass-like functionality, in which cardinal direction and azimuth are determined. (Alternative methods can be used to determine a geovector. For example, many terrestrial-based stations emit navigational beacons that can be used to determine geo-location and relational-direction. Wireless systems may also be used to triangulate signals emitted from a mobile device. Such signals are collected at multiple receiving locations and based on the relative reception time and/or strength a geo-location is determined for the mobile device. Similarly, a mobile device can triangulate its position based on received beacons.). The georeferencing techniques discussed above and in the cited documents can also be used to determine geovector information corresponding to a location depicted in a photograph. (E.g., a GPS or wireless system can provide geovector information. Or geovector information can be obtained from an image capture device, among the other techniques discussed.). In one embodiment, geovector data is obtained via an online (e.g., internet or network) database. A user simply enters in a street address or map-grid location, and the database returns corresponding geovector data (e.g., longitude, latitude, height, etc.). Or the geovector information is obtained from a user or agency based on human or computer analysis of an image. Artisans know other ways to determine and obtain geovector information. Of course, such other known techniques are suitably interchangeable herein.

Beginning with the area of remote sensing, and extending to all photography with an innate geovector, digital watermarking is extended to embrace this fundamental set of information inherent in each and every photograph. Just as a "copyright" is fundamentally a part of every photograph, so too is a "geovector" a fundamental part of every photograph, and digital watermarking can expressly convey this geovector information.

Once obtained, a geovector is either contained in the embedded watermark information itself, or contained in a database to which the watermark represents a pointer, or both (see Fig. 15). Indeed, the geovector can be included in a watermark message or payload. In one embodiment, a watermark embedder performs error correction coding of a watermark's binary message (e.g., representing the geovector), and then combines the binary message with a carrier signal to create a component of a watermark signal. There are several error correction coding schemes that may be employed. Some examples include BCH, convolution, Reed Solomon, and turbo codes. These forms of error correction coding are sometimes used in communication applications where data is encoded in a carrier signal that transfers the encoded data from one place to another. In the digital watermarking application discussed here, raw bit data can be encoded in a fundamental carrier signal. It then combines the watermark signal with a host signal (e.g., a photograph). A watermark embedded within a photograph may serve as (or carry) a database index or pointer. For example, the watermark includes an index, which once decoded, is used to interrogate a database (see Fig. 16). The database preferably contains data records including geovector information. The watermark index is used to identify a corresponding data record for the respective photograph (e.g., the photograph in which the watermark is embedded within). Of course, the database may be local or may be remotely accessed. In one embodiment, the watermark includes data corresponding to a URL or IP address, which is used to access a website. See Assignee's U.S. Patent Application No. 09/571,422 (published as WO00/70585), mentioned above, for a further discussion of watermark-based linking. (The data may directly include the URL or may be used to access the URL.). A database associated with the website may be interrogated to retrieve the corresponding geovector information for a photograph.

(In another embodiment, a watermarking reading device defaults to a URL or to an IP address, or queries a default database, upon detection of a watermark in a photograph.).

In yet another embodiment, geovector information is redundantly provided in header structures and watermark payloads. Standardization efforts are currently underway, which are extending the idea of the geovector well beyond the examples presented above. See, for example, the Open GIS Consortium, an international consortium seeking to foster collaborative development of the OpenGIS Specifications to support full integration of geospatial data and geoprocessing resources into mainstream computing (http://www.opengis.org). (Of course, there are other known groups and companies focusing on geospatial and geographic information and services efforts. The "digital earth" concept is also known.). Such proposed standards have straightforward coordinate systems at their core.

We have determined that the standardization proposals lend themselves to conveying georeferencing in several different formats, including conveying information with digital watermarks, classic header structures, and pointer-to-elements in an associated database. Upon closer examination, however, we believe that digital watermarking techniques provide enhanced benefits when compared to these other techniques.

In today's world, where photography is rapidly becoming digital, a method of securely attaching identifying information (e.g., geovector information) to a corresponding photograph is needed. Digitally watermarking photographs provides a solution for the attachment problem. As discussed, a watermark may provide geovector information (or access to such information). A photograph many even be redundantly embedded with multiple copies of a watermark, further ensuring robust attachment of information to the photograph. Contrast our digital watermarking procedure with a procedure, which appends geovector information via headers. Whereas headers may be able to provide geovector information, they have a higher chance of separation from the underlying data- defeating a secure attachment feature.

Some embodiments of the present invention involve a step of identifying a photograph (e.g., digitally watermarking a photograph with a binary identifier or a geovector) and, if using an index or identifier, storing information related to the index in some database or across a group of distributed databases. Adding a dimension of geovector information to the management of photographs results in a database or set of coordinated databases, which represent a searchable platform suitable for geographically based queries. The implications of such are tremendous. For example, a fisherman may search the database(s) for a photograph of a favorite fishing hole in Wyoming, based on a search criteria for a given time period, a range of time periods or by geo-location. The applications are endless - expanding far beyond dispelling fish stories. Friends of the fisherman may decode a watermark geovector or index embedded within the fisherman's watermarked photographs (e.g., by a compliant watermark reading device) to determine whether an area depicted in a photograph corresponds to a trout farm or to a high mountain lake — allowing "fish stories" to be verified. This information is readily available via a geovector associated with the image. The fisherman can maintain a photo-journal of his fishing trips, knowing that the embedded watermarks provide sufficient information to help retrace his steps and travels. To aid this endeavor, digital cameras are envisioned to be equipped with watermark embedding software and geovector gathering modules such as GPS units. Or the geovector information can be added when images are stored in a database or processed after the fishing excursion.

Digitally watermarking photographs helps to provide a collision- free serial numbering system for i entifying imagery, owners, and attributes.

There are additional benefits in creating a georeferenced system of images using digital watermarks. A classic notion in most standardizations across all industries is a notion of a "stamp" or "seal" or a similar concept of indicating that some object has successfully completed its appointed rounds. Call it branding, call it formality, or call it a soft form of "authenticity;" the historical momentum behind such a branding concept is huge. In one embodiment, to ensure that a given image is properly georeferenced (under a chosen standard), digitally watermarking the given image is a final step representing a formalized "seal of approval." The digital watermark itself becomes the seal. In one embodiment, a watermark identifier is obtained from an online repository, which issues and tracks authentic identifiers. The repository can be queried to determine the date and time of issue. Or the identifier can be linked to a seal or company logo. Software and/or hardware is configured to routinely read embedded digital watermarks and display an appropriate brand logo, seal, or certification. The "seal" itself then becomes a functional element of a standardization process, serving many functions including permanent attachment to standardized and dynamic metadata (e.g., a geovector).

Photographs by their very nature can be inter-processed, merged, split, cut up, etc., and so forth, as described in the prior art. This tendency is especially applicable to various geo-referenced imagery applications where some data sets are merged and viewed as derivative images. Tracking image pieces is a daunting task. Digital watermarks, in many such applications, are a good way of coordinating and keeping track of highly diverse image components. For example, an image is redundantly embedded with multiple copies of a watermark including a geovector for the image. When the image is cut up (or merged, etc.), each image piece preferably retains at least one of the redundantly embedded watermarks. The watermark is then decoded to identify the respective image piece, even when the piece is merged or combined with other image pieces. A geovector may also provide sufficient information for stitching together map quilts, as discussed above, particularly if boundary or corner coordinates are provided. Instead of focusing on imagery characteristics, the map is quilted together based on the embedded geovector information.

This technology includes many applications beyond identifying and associating data with photographs. Consider embedding a digital watermark in a particular region of a map or photograph (e.g., corresponding to a location for a fire hydrant, tree, road, building, lake, stream, forest, manhole, water or gas line, park bench, geographical area, stadium, hospital, school, fence line, boarder, depot, church, store, airport, etc., etc.). These region-specific watermarks preferably include unique watermark payloads. A watermark payload conveys geovector information (or map coordinates) corresponding to its particular region of interest. (E.g., a geovector corresponding to a fire hydrant reveals the hydrant's location in latitude/ longitude, etc. coordinates.). Now consider a modification in which, instead of uniquely watermarking individual map or photograph regions, a digital watermark is redundantly embedded throughout the map or photograph. In this modification, geovector information is conveyed via the redundant watermark payload for all fire hydrants's depicted on the map or photograph. Alternatively, instead of a payload conveying such geovector information, the payload comprises an index, which is used to interrogate a database to retrieve geovector information. (It should be appreciated that a fire hydrant is used for explanatory purposes only. Of course, other regions, structures, roads, land areas, bodies of water, buildings, etc. can be similarly watermarked.).

In another embodiment, a utility company watermarks a map or photograph to include geovector information corresponding to specific depicted objects, such as power stations, transformers and even transmission lines. Such information assists in locating areas for repair or inspection. Additional information can be stored in a database according to its geovector. For example, a power line's capacity, age, maintenance record, or rating can be associated in a database according to the line's geovector. Such principles can be applied here as well. In another embodiment, a city, municipal, state or government agency digitally watermarks geovector location information on its maps and charts, corresponding to streets, country areas, buildings, manholes, airports, ports, water systems, parks, etc.

In another embodiment, school age children carry bracelets, book bags, tags, ID cards, shoelaces, or necklaces, etc., each watermarked with geovector information identifying their home, parents work address or school location. When lost, the preschooler presents her bracelet (or other object) to a police officer, school official, or automated kiosk. The embedded watermark is decoded to reveal the geovector information. The child's home or school, or a map route, can be identified from such.

Tags or collars for domestic animals or livestock can be geo-watermarked to assist in recovery when lost. In still another embodiment, documents are embedded with geovector information. Consider embedding geovector data on a deed or property listing. Additional information regarding the property (e.g., title history, tax information, community information, recording information, photographs, etc.) is obtained via the geovector data link. For example, the additional information can be stored in (or referenced by) a database. The geovector data or other pointer serves as the index for the database. Geovector information can also assist in notarizing (or authenticating) a document. Data is embedded in the document, which may indicate the document time (e.g., date and time) and location of creation (or execution). Upon presentment to a compliant watermark-reading device, the embedded data is extracted and read for verification. In yet another embodiment, geovector information is the common factor, which binds information together. For example, information is stored according to its geovector information (e.g., according to creation geo-location, subject matter geo-location, ancillary relationship to a geo-location, etc.). Database searching for information is carried out via the geovector data. To illustrate, the database is searched for all information pertaining to a specific geo-vector (e.g., the Washington Monument). All data (or a subset of the data) pertaining to the geovector (e.g., the Washington Monument) is returned to the user. The data can include reports, web pages, maps, video and audio clips, pictures, statistical data, tourist information, other data, musings, related sonnets, governments information, just to name a few types of data.

The foregoing are just exemplary implementations of this technology. It will be recognized that there are a great number of variations on these basic themes.

For example, digital watermarks can be applied to any data set (e.g., a satellite image, or a map generated from the master database) for forensic tracking purposes. This is particularly useful where several copies of the same data set are distributed through different channels (e.g., provided to different users). Each can be "serialized" with a different identifier, and a record can be kept of which numbered data set was provided to which distribution channel. Thereafter, if one of the data sets appears in an unexpected context, it can be tracked back to the distribution channel from which it originated.

Some watermarks used in the foregoing embodiments can be "fragile." That is, they can be designed to be lost, or to degrade predictably, when the data set into which it is embedded is processed in some manner. Thus, for example, a fragile watermark may be designed so that if an image is JPEG compressed and then decompressed, the watermark is lost. Or if the image is printed, and subsequently scanned back into digital form, the watermark is corrupted in a foreseeable way. By such arrangements it is possible to infer how a data set has been processed by the attributes of a fragile watermark embedded in the original data set.

Certain "watermark removal" tools can be built to alleviate visibility or processing problems in cases where unacceptable impact of a digital watermark is identified. This can either be a generic tool or one highly specialized to the particular application at hand (perhaps employing secret data associated with that application). In another embodiment, a "remove watermark before analyzing this scene" function is included within analysis software such that 99% of image analysts wouldn't know or care about the watermarking on/off/on/off functionality as a function of use/transport. While a geovector is described above to include, e.g., "6+1" dimensions, the technology is not so limited. Indeed, a geovector can include more or less vector elements, depending on the referencing precision required. (To illustrate, altitude may be immaterial when other geovector coordinates are provided. Or a camera sensor geometry (e.g., "+1") element may not be needed to uniquely identify a location or to account for sensor geometry. Alternatively, a map identifier or locator can be included to achieve similar functionality instead of a geovector. In other cases, where only rough referencing information is needed, providing only longitude and latitude coordinates may be sufficient. Of course, in the event that geospatial or geography information and services standards are formalized and/or updated, the geovector can be formatted to include the reference locators described in that standard. Similarly, instead of a geovector, geo-coordiήates or other location information can be provided via a watermark or watermark index.).

There are many embodiments discussed herein which may benefit from the inclusion of two different watermarks. For example, a first watermark may include information regarding (or pointing to) geovector information, while a second watermark includes a database identifier or location. The second watermark may alternatively include (or point toward) information pertaining to events, people or animals identified in the photograph, occasions, groups, institutions, copyright ownership, etc. Or the embodiment may include both a robust geovector watermark and a copy-tamper fragile watermark.

Third Set of Embodiments A further set of embodiments according to the present invention employs digital watermarking techniques to ease navigation and map orientation.

In some embodiments, digital watermarking techniques are combined with GPS systems. Applications of such embodiments include implementations in fields such as government work and field reconnaissance, commercial or recreational boating, hiking, mountaineering, travel, orienteering, geography, education, exploration, entertainment, sight seeing, etc.

Global positioning systems (GPS) have improved navigation by providing accurate location feedback. As will be appreciated by those skilled in the art, military and civilian water, ground, and airborne vehicles often use GPS systems for navigation. GPS is a satellite-based radio navigation system capable of providing continuous position, velocity, and time information. As earlier noted, GPS receiver units receive positioning signals from a constellation of satellites deployed in various orbits about earth (e.g., 12-hour orbits). The satellites continuously emit electronic GPS signals (or telemetry) for reception by ground, airborne, or watercraft receiver units. By receiving GPS signals from a plurality of satellites, a properly configured receiver unit can accurately determine its position in three dimensions (e.g., longitude, latitude, and altitude). There are many known GPS systems. For example, U.S. Patent No. 5,990,826 discloses an interbuilding and urban canyon extension solution for global positioning systems.

U.S. Patent No. 5,861,841 discloses a compact GPS receiver/processor. The GPS system including an antenna to receive Global Positioning System (GPS) signals from two or more GPS satellites and a credit card size GPS signal processing Smartcard. The Smartcard is attached to the antenna that receives the GPS signals and determines and displays the present position of the antenna.

U.S. Patent No. 5,964,821 discloses a navigation system for offering navigational assistance to a mobile user. The navigation system receives GPS position information signals, which are processed to determine current position latitude and longitude coordinates and direction of travel.

Of course, there are many other GPS systems known to those of ordinary skill in the art. Returning to watermarking, determining orientation of embedded data can be discerned by reference to visual clues. For example, some watermarked objects include subliminal graticule data, or other calibration data, steganographically encoded with the embedded data to aid in determining orientation. Others objects can employ overt markings, either placed for that sole purpose (e.g. reference lines or fiducials), or serving another purpose as well (e.g. lines of text), to discern orientation. Edge- detection algorithms can also be employed to deduce the orientation of the object by reference to its edges.

In one example, subliminal graticule data can be sensed to identify the locations within the image data where the binary data is encoded. The nominal luminance of each patch before encoding (e.g., background shading on a map ) is slightly increased or decreased to encode a binary "1" or "0."

The change is slight enough to be generally imperceptible to human observers, yet statistically detectable from the image data. Preferably, the degree of change is adapted to the character of the underlying image, with relatively greater changes being made in regions where the human eye is less likely to notice them. Each area thus encoded can convey plural bits of data (e.g., 16 - 256 bits). One problem that arises in many watermarking applications is that of object or positioning corruption. If the object is reproduced, skewed, or distorted, in some manner such that the content presented for watermark decoding is not identical to the object as originally watermarked, then the decoding process may be unable to recognize and decode the watermark. To deal with such problems, the watermark can convey a reference signal. The reference signal is of such a character as to permit its detection even in the presence of relatively severe distortion. Once found, the attributes of the distorted reference signal can be used to quantify the content's distortion. Watermark decoding can then proceed — informed by information about the particular distortion present.

The assignee's U.S. Application Nos. 09/503,881 (WO0152181) and 09/452,023 detail certain reference signals, and processing methods, that permit such watermark decoding even in the presence of distortion. In some image watermarking embodiments, the reference signal comprises a constellation of quasi-impulse functions in the Fourier magnitude domain, each with pseudorandom phase. To detect and quantify the distortion, the watermark decoder converts the watermarked image to the Fourier magnitude domain and then performs a log polar resampling of the Fourier magnitude image. A generalized matched filter correlates the known orientation signal with the re-sampled watermarked signal to find the rotation and scale parameters providing the highest correlation. The watermark decoder performs additional correlation operations between the phase information of the known orientation signal and the watermarked signal to determine translation parameters, which identify the origin of the watermark message signal. Having determined the rotation, scale and translation of the watermark signal, the reader then adjusts the image data to compensate for this distortion, and extracts the watermark message signal as described above.

Such watermarking techniques, and many others known to those skilled in the art, may be suitably employed to improve navigation, ease road journeys and enhance education, among other benefits.

In accordance with one such embodiment, a map 10 includes plural-bit data steganographically encoded therein. (The term map is used broadly herein and includes, for example, navigational tools and documents, road maps, atlases, wilderness maps, area maps, city maps, tourist maps, location guides, walk or run routes, path-layouts, 3-dimensional models, vegetation maps, building maps, structure maps, stadium seating and concert hall locations, park maps, amusement or theme park maps, DEM maps, master maps, topographical maps, globes, relief maps, to name just a few. A map may also include a digitized map for display on a monitor, TV, LCD, etc.). Map 10 can be printed or drawn on any suitable surface including paper, fibers, fabric, wood, plastic, metal, metal-alloys, objects, plaster, laminates, etc., etc. A digitized map image may include plural-bit data steganographically encoded therein.

The encoding of the map can encompass artwork or printing on the map, the map's background, lines on the maps, a laminate layer applied to the map, surface texture, etc. If a photograph, line design, or drawing is present, it too can be encoded. A variety of watermark encoding techniques are detailed in the patent documents discussed herein; artisans in the field know many more.

Preferably, map 10 is embedded with a plurality of watermarks. As shown in Fig. 1, map 10 is divided into a plurality of areas A - P (e.g., blocks, patches, or segments, etc.). Of course, the number of blocks or areas can be varied, with enhanced precision resulting from an increased number of blocks. Each area is preferably embedded with at least one watermark. (In one embodiment, each individual block is redundantly embedded with multiple copies of a respective unique watermark - further enhancing the robustness of the map. In another embodiment, some of the blocks are subdivided, with at least one unique watermark embedded in each subdivision.).

A watermark typically includes a payload (e.g., 16-256 bits) that provides area (or location) identifying information. For example, the payload may include the geo-coordinates (e.g., "center lat: N34.4324352, long: W87.2883134; rot from N/S: 3.232; x2.343, y2.340, dxθ.123, dy493, etc.") for the center of the area, the coordinates of each area corner or boundary, the area of the boundary, a range of coordinates for the area, coordinates in relation to the overall area depicted by the map, and/or the coordinates for a dominate (or well-known) structure, road, area, etc., within the area block. (For example, area A is embedded with at least one watermark having coordinates corresponding to area A's center or corners, etc.). The payload may simply be a number that is associated with a block location on the map. For example, if a map comprises 32 by 64 watermarked blocks, each block is encoded with a number between 1 and 2048.

Further, the payload may include additional fields, one conveying a map ID and another conveying the block number. The map ID may be used to identify the map as one of a collection of potentially many different maps. To program a map reader (as discussed below) for a particular map in the collection, the user passes the reader over the map to read the map ID (or simply enters the map ID). The reading device is then connected to a database, via a docking station at a personal computer or through a wireless connection. The reading device queries the database with the map ID and the database returns information associated with items of interest on the particular map (such as restaurants, scenic areas, camp sites, etc.) The user then disconnects the reader from the database and is ready to use the map. In another embodiment, some bits in the watermark payload identify the latitude/longitude of the map origin, while other payload bits identify the coverage extent of the map. Additional payload bits may even identify the offset of a chosen block from the origin. As an alternative, the lat long and extents could be read from an initialization section, e.g., a map legend, or corner area, etc. As an alternative, the location information may include an index or identifier, which is used to interrogate a database to find physical coordinates or location information. Upon extraction by a watermark decoder, the index is provided to a database. The decoder may communicate with a database via a network (e.g., wireless network, LAN, WAN, the internet, intranet, etc.). Alternatively, the database may be maintained locally, or stored on a computer readable medium such as a compact disk (CD), magnetic tape, magnetic storage device (disk drive, removable media, floppy disks, etc.), electronic memory circuits, etc. Related information that is stored in the database is indexed via the watermark index.

A grid (or orientation) signal can also be included in the watermark and/or location information. Preferably, the entire map uses the same grid signal, so that all blocks in a map can be used to determined rotation and scale of the map. Such a grid signal may assist in detecting watermarks.

(Alternatively, such a gri signal can be used to help orient a map. For example, an orientation signal may be used to designate magnetic North, or another map orientation. Feedback can be supplied to a reader (e.g., watermark decoder) to help orient a watermark reader with respect to a map and the physical surrounding area. As discussed below, a watermark reader may be provided with compass-like functionality to assist with such orientation.).

The watermark may be embedded such that it conveys both orientation and message information. For example, the modifications made to encode message symbols may be made in a manner that creates a recognizable pattern, such as a known array of peaks in a particular transform domain, such as the autocorrelation domain, the Fourier domain, or some other transform domain. One approach is to embed the message symbols by modulating a carrier signal with autocorrelation properties that form peaks in the autocorrelation domain. In particular, by performing an autocorrelation of an image captured of the watermarked map, the watermark detector generates peaks that can be compared with an expected pattern of peaks attributable to the carrier signal. The detector then performs pattern matching with the peaks to determine the scale and rotation of the captured image relative to the watermarked map. Another approach is to use peaks in a particular transform domain (such as the Fourier domain) for both orientation and message information. In this approach, the watermark is represented as collection of N possible peaks at particular locations in the Fourier magnitude domain. The detector first identifies some subset of the N peaks to determine orientation, and then determines message information by identifying the presence or absence of peaks at predetermined locations. In one such embodiment, each block on a map has a similar payload structure, e.g., each including the same grid signal and perhaps the coverage area of the map (e.g., latitude/longitude, range of coordinates, etc., of the map.). The remaining payload bits would then be used to identify the individual areas (e.g., blocks A - P) on a map 10.

Phase-correlation can be employed to even further improve the location resolution within a map block. Once the detector has determined the rotation and scale of the captured image relative to the watermarked map, it can realign the captured image using these rotation and scale parameters. The watermarked block locations can then be detected by using the known phase attributes of the watermark signal and correlating these known attributes with the realigned, captured image. These block locations then serve as reference points for decoding watermark message data from a particular block. Also, in one embodiment, resolution down to a pixel within a block is achieved once rotation and scale are resolved.

A watermark reading device can take various forms. Exemplary is a handheld reading device

20. (With reference to Fig. 2, a hand-held device 20 may allow better placement of the device in close proximity to map 10, or sections of map 10 itself, permitting precision navigation. A hand-held reading device 20 is typically portable - an advantageous feature for outdoor and wilderness applications.). Examples of such hand-held device 20 include stand alone hardware implantations, portable computing devices, personal digital assistants (PDAs), portable devices such as those manufactured by Compaq (e.g., the iPac line) and Handspring, cellular and satellite phones, smart pens and watches, etc. A handheld reading device 20 may include a general purpose or dedicated computer, incorporating electronic processing circuitry (e.g., a CPU) 22, memory 24, an interface 26 to an input device 28, an output device 30 (e.g., a display screen, LED indicators, LED arrows, speakers, and/ or audio-synthesis chip, etc.), and optionally a network connection 32. (Of course, interface 26 may be integrated with other device 20 circuitry, particularly if input device 28 is built onboard device 20. Also device 20 preferably includes sufficient bus or other structure to facility electronic signal communication between the various device components, where needed.). The network connection 32 can be used to connect, through a wireless or satellite connection to a network (e.g., intranet, internet, LAN, WAN, etc.). In one embodiment the input device (or reading device 20) is tethered to a desktop or laptop computer. A network connection is achieved via the connected computer. Preferably, a handheld reading device includes (or, alternatively, is in communication with) a global positioning system receiver 34. As will be appreciated by one of ordinary skill in the art, suitable software programming instructions executing via CPU 22 can be used to effect various types of functionality (including watermark detection and reading) as discussed herein.

The illustrated handheld reading device 20 includes (or is in communication with) an input device 28. The input device 28 may include an optical reader, an imaging mouse, a digital camera, a laser or pen scanner, a digital eye module, etc. Digital eye modules (such as those provided by LightSurf, Inc.) typically include features such as a complete camera on a chip, CMOS imaging sensor, miniaturized lens and imaging software. Other imaging devices include a CCD image sensor. Of course, input devices can be packaged in variety of forms to suit particular applications. (In one embodiment, an input device 28 is tethered to a personal computer having watermark decoding software executing therein.).

The handheld reading device 20 may include additional features to improve reading and facilitate accurate placement of the device 20 on map 10. For example, the input device 28 may optionally include a targeting guide, such as a cross-hair-like viewer (or other visual indicator). A targeting guide helps achieve precise placement of the input device 28 on the map 10. The area within the cross-hairs (or targeting guide) preferably corresponds with the area captured by the input device.

(Alternatively, the area within the cross-hairs could also be a known offset from the area captured by the input device. Watermark rotation and scale information could be used to determine actual location.). In another embodiment, a see-through window is provided to locate and target an exact map location. (Of course, the input device 28 may employ a beam-splitter or reflective lens to ensure that the viewed area is the same as that captured by the input device 28.). The viewable window area corresponds with that captured by the input device 28. In still another embodiment, a camera-pen or laser-pointer allows a user to pin-point a map location, which is scanned (e.g., image capture). A typical application of this embodiment of the present invention involves a map user placing a handheld reading device 20 (or an associated input device 28) near or on a map 10 (Fig. 3). (Of course, Fig. 3 is not intended to limit the size relationship between the map 10 and device 20). The input device 28 reads a map area (e.g., area O, or a sub-area within area O), which preferably includes an embedded watermark. The input device provides an output signal (e.g., representing the captured map area) to the handheld reading device 20. Decoding software running on the handheld device 20 identifies a watermark (if present) from the signal and extracts the embedded location information from the watermark. In one embodiment, map location feedback is presented to the user via the reader device's output (e.g., an audio signal, a text display). The user is then able to compare the map coordinates to coordinates taken from a GPS receiver. In such an embodiment, the handheld reading device 20 includes (or is in communication with) a GPS receiver 34. Location information is extracted from a watermark (step SI, Fig. 4). The current, physical location of the handheld reading device 20 is determined via the GPS receiver (Step S2). In step 3, the handheld reading device 20 compares (and/or correlates) its physical location to the map location scanned by the input device 28. (Of course, such correlation may be realized by software computational instructions and/or with database/table look-up.). The map user is presented with appropriate corrective feedback, if needed (Step S4). For example, in one embodiment, the handheld reading device 20 may prompt the user to move the input device 28 (or reading device 20) to a map location, which corresponds with the actual physical location of the handheld reading device 20. (In one embodiment, the prompting is by a visual indicator (e.g., arrows or LEDs) indicating the direction in which the device must be moved. In another embodiment, an audio indication is presented, for example, such as "move six inches to the left and one inch down," or "move to grid number E8." In still another embodiment, a display screen displays a digitized replica of the map, or a portion of the map, and displays both the current physical location and the map location scanned by the input device. In still another embodiment, the output device indicates when the input device is properly located on the map with respect to the user's current physical location.).

In a related embodiment, the watermark's encoded data includes identification of a map's grid system. The reading device 20 correlates (e.g., via formula or table/database look-up) the grid system to the GPS coordinate system and conveys to a user her current map grid location (e.g., tells her that she is currently located in grid F-9). Consider the following examples to even further illustrate the many possible applications of the present invention.

A map user is examining a map of the Western United States of America while personally (e.g., physically) being located in Boise, Idaho. The map user places the input device 28 so as to scan a location near Tualatin, Oregon. The handheld reading device 20 can convey (e.g., via the output device

30) to the user to move the input device 28 East and South, until the input device 28 corresponds with the user's physical location in Boise. (GPS coordinates can help to facilitate such functionality.). In this way, the user can identify her present location with respect to the map. Alternatively, as in another embodiment, the output device 30 can indicate that the user must travel West and North to reach the scanned location near Tualatin. In still a further alternative, particularly if an index or code is embedded in the watermark, additional data can be provided to the user. To illustrate, the index is used to interrogate a networked (or local) database. Information, e.g., directions, preferred routes or roads, mileage between the current physical location and the scanned location, dining or tourist information, etc., is provided to the handheld reading device for presentation to the user. As a further example, a user is going on a trip and obtains watermarked maps (e.g., from a store, service provider, etc.). The maps are indexed by a MAP ID in a database, which can receive current information (e.g., promotions) from hotels, resorts, restaurants etc. The information can be downloaded, streamed, or stored in media (e.g., CD-ROM, electronic or magnetic media, etc.). The database may be maintained by the handheld reading device or may simply be queried via a communication channel. As the user travels, she can place her handheld reading device over various spots on the map to extract corresponding watermarks. The watermarks are then used to index the database to retrieve the downloaded information regarding hotels, resorts, etc. Hence, the watermarked map provides the user with information for places on the map just by passing the handheld reading device 20 over the map. (In contrast, database-linked GPS systems typically require the user to be in a particular location to get such information about that location.). In another example, a user enters (e.g., alphanumerical values via keypad entry or scanning the area on the watermarked map) a map location of a desired destination. The reading device determines the present location (e.g., via a GPS receiver) and provides feedback to the user. In one embodiment, the feedback is a visual indication (e.g., arrows, LEDs, text directions, grid map coordinates, visual display, etc.) of how to move the handheld device to locate the destination point, from the current location. In another embodiment, the feedback is a print out (or display) of travel directions (or map grid numbers) from the current location to the scanned, destination location. In another embodiment, the handheld device includes a known laser pointer and sensing array. Once the destination location is selected, the laser pointer points (e.g., traces) a direction from the current location to the scanned, destination location, based on the positioning of the pointer as determined by the sensing array.

Now consider a potential life-saving example in which a hiker becomes lost in the mountains. The hiker presents a watermarked map to her handheld reading device 20. The associated input device 28 scans a location on the map. The reading device then gives feedback (e.g., audio, visual, text, etc.) to the hiker to reposition the input device 28 (or the reading device 20) on the map to correspond with her present physical location with respect to the mountains. The hiker can then immediately identify where she is with respect to the map. (Of course, the handheld reading device 20 may include compass-like functionality, common in some GPS receivers. Such functionality is helpful to orient a map with respect to physical surroundings, e.g., to align the maps N/S reference orientation with the environmental conditions.).

A watermark may also include (or reference) information regarding the map within which it is embedded. For example, the watermark may include a range (or boundaries) of coordinates, which define the map 10. The handheld reading device 20 may then determine (e.g., calculate based on the

GPS coordinates or via a table look-up) if the user's present, GPS-determined location is viewable on the map 10. The user can be prompted by the device 20 to change maps when her present location is not viewable on the scanned map. In some cases, the handheld reading device may prompt (e.g., via an audible signal, text or graphic, etc.) the user to scan a specific map. This functionality is realized particularly well with the aid of additional information, such as that stored in a database. For example an atlas or an interconnected series of maps can be stored according to their corresponding coordinates (or a range of coordinates). When the watermark index does not match the present physical location (or fails to fall within the boundaries of the map), the user is prompted to select another map. If a particular set of maps is pre-programmed, e.g., preloaded onto the device, stored via CD, or via a specific on-line database, an appropriate map ID or map number (or even map name) corresponding to the scanned coordinates can be presented to the user. Street (or road) signs may also be digitally watermarked as discussed above with respect to objects. Consider the implications of such. An automobile is equipped with an watermark decoder (e.g., an input device such as an optical reader, digital camera, laser reader, etc. and watermark detecting and decoding software). In one embodiment, the input device is configured with magnification enhancements, such as a zoom lens, signal amplifier, etc., to allow capture of road sign images from a far. The captured images (or corresponding signals) are input for analysis by the decoding software. The watermark payload is used to convey related information.

For example, a speed limit sign may include a watermark having a corresponding speed message embedded therein. For example, upon decoding the watermark, the read device 20 signals (or itself enables) an audio message to announce the speed limit. To achieve such functionality, the payload may include an index that is used to retrieve an audio or wave file. The wave file could be played via a media player (and output device, e.g., car stereo system), or passed through a digital-to-analog converter and piped through the car's stereo system. Alternatively, the payload itself may include enough information to be converted into an audio message.

In another example, the watermark includes an index that is used to interrogate a database. The database may be stored locally in the automobile, or may be accessed via a wireless network. A corresponding data record is found and returned. The returned data record may include a wide variety of information and data. To illustrate, the return information may include data about upcoming rest stops, tourist attractions, motels (and vacancy in such), speed limits, restaurants, location data, etc. A mobile traveler, upon receipt of such information, may even make a reservation at a motel via a link provided in the return information.

Consider also watermarking tourist, restaurant, convenience and regulatory signs. These too can be embedded with plural-bit data. A compliant reader extracts embedded information to facilitate the retrieval of additional data regarding the sign. (The term "compliant" in this context implies that the device is able to detect and decode watermarks.). An embedded index or identifier is communicated to a database to index related information (e.g., the database may be maintained in a CD-ROM, electronic or magnetic memory residing locally in the automobile, or may be an on-line database accessed via a wireless connection. In an online database embodiment, a web browser may be installed on a compliant device, e.g., a general purpose computer, to help handle the transfer of information, e.g., HTML code, wave files, data files, etc.). Upon receipt of the related data, a traveler may access related data such as menus and prices of local restaurants, vacancies, gas prices, hours of operation, directions and maps, local attractions, etc., etc. (The functionality and systems for linking an identifier to additional data is even further described in assignee's U.S. Patent Application No. 09/571,422 (published as WO00/70585). Signs at (or maps and photographs of) theme parks, sporting stadiums, concert halls, event centers, convention centers, zoos, office buildings, government buildings, manufacturing plants, universities, shopping malls, parks, schools, museums, etc., etc., may be similarly digitally watermarked. A handheld reading device may be used to coordinate a participant's location with respect to a map and to gather related information as discussed above.

Fourth Set of Embodiments With reference to Fig. 5, an image (or image data) 310 is captured from an aerial platform 311, such as an aircraft, satellite, balloon, unmanned aircraft, etc. The image 310 is communicated to a receiving or ground station 312. (In some instances, the image signal may be relayed through various aerial and/or other ground stations before reaching ground station 312.). Ground station 312 preferably includes a watermark embedder 312a, which embeds a digital watermark with the image 310, to produce a digitally watermarked image 313.

A digital watermark is typically embedded in a digital representation of the image 310. Although not required, the digital watermark preferably survives transformation to various analog representations (e.g., printing) as well. The digital watermark includes a watermark identifier (ID). In the preferred embodiment, each image is digitally watermarked to include a unique watermark ID. The ID typically includes plural-bit data, e.g., in the range of 2-256 bits. In one embodiment, a digital watermark (and identifier) is redundantly embedded within an image to improve robustness. For example, an image is divided into tiles or sections, and each tile or section is embedded with the digital watermark (and ID). Alternatively, a subset of the sections are embedded. Such techniques help to ensure the robustness of a watermark, particularly when an image is to be manipulated (e.g., clipped, cut- and-pasted, resized, rotated, etc.).

Digitally watermarked image 313 is stored in a database 314. (A watermarked image can be directly communicated to database 314, transferred via a storage medium and/or relayed to database 314.). Database 314 preferably manages images and/or related data. Database software, e.g., such as provided by Microsoft, Oracle, Sun Microsystems, etc., can be executed by a computer or server to help maintain database 314. Of course, database 314 can be maintained by a ground station 312 system, or be maintained in a remotely located network. In one embodiment, database 314 communicates with a network, such as a LAN, WAN, dedicated network, private network, etc. In some embodiments, database 314 includes a plurality of databases. In this case, at least one database maintains image data, while at least a second database maintains related information (e.g., metadata, related files, comments, file history, and/or security clearance information, etc.). Here, metadata is broadly defined to include a variety of information such as creation data, geo-location information, ancestry data, security information, access levels, copyright information, security classifications, usage rights, and/or file history, etc.

Image 313 and/or any related information is preferably stored and indexed according to watermark IDs. For example, a watermark ID provides a thread by which images and related information are grouped, stored and/or indexed. (The dashed lines in Figure 5 represent this optional embodiment.).

Optionally, image data is communicated to a second database 315. Database 315 can be used to maintain original image 310 and/or an original watermarked image 313.

As indicated above, a problem faced by image management systems is how to efficiently manage an image's ancestry and related information. Normal image processing (e.g., scaling, cropping, rotating, clipping, resizing, cut-and-pasting image blocks, and/or marking, etc.) of an "original" image results in a "derivative" image. In conventional systems, derivative images frequently retain minimal, if any, related metadata. The metadata, such as that stored in header or footer files, is easily separable from derivative images. Separation results in a significant loss of information, particularly for a derivative image. One conventional solution is to manually record an image identifier as an image moves through an exploitation (or derivative) process. This manual recording process is labor intensive and cumbersome at best.

A better solution, as disclosed in this application, is to place a unique digital watermark ID within an image to enable database linking and indexing. Metadata contained within the database can be then associated with a specific image, or with a family of images, via the unique watermark ID. With reference to Fig. 6, a user terminal 318 retrieves a digitally watermarked image 001 from database 314.

User terminal 318 preferably includes a processor, memory and suitable software instructions to facilitate digital watermark detection and/or embedding. The user terminal 318 will preferably include an operating system, such as Windows, Windows NT, Linux, etc., and image-handling (and editing) software. Suitable image-handling software can be obtained from Microsoft, Adobe, SRI and Erdas, among others. Preferably, both the watermark detecting software and the image-handling software are compatible with various types of image formats, such as bit-maps, JPEG files, TIF files, etc. (However, such compatibility is not required.).

Watermark detection software executing in user terminal 318 analyzes image 001. The watermark detection software can be called by the imaging software, may operate as a plug-in, or may be even integrated with the image-handling software, operating system, or other software module. The watermark detection software extracts the unique watermark identifier (e.g., 1D-1) embedded within image 001. Having obtained the identifier (ID-1), the user terminal 318 can optionally communicate with database 314 to retrieve related information, such as metadata, files, and related images. For example, the watermark 1D-1 is used to interrogate database 314 to retrieve information regarding the geo-coordinates for the image, the time and date taken, analyst comments, and/or analyst information, etc. Preferably, the watermark ID-1 can also be used to index any derivative images, e.g., derivative 001.

(In this case, derivative 001 is an image derived from image 001.).

In en exemplary embodiment, since each image includes a unique identifier, derivative 001 includes a watermark identifier (e.g., ID- 5), which is unique from the corresponding original image 001 (e.g., identifier ID-1). Derivative 001 and image 001 are associated (e.g., linked) together in database 314, via identifier ID-1 (and, optionally, via ID- 5).

In some instances, user terminal 318 will create additional derivatives. Take for instance, an example when user terminal 318 enlarges the derivative 001 image, thus creating a new derivative 001a. This new derivative is preferably uniquely identifiers with a digital watermark. A process of digitally watermarking a derivative typically involves removing the original watermark from the derivative and replacing the watermark with a new unique identifier. (In an alternative embodiment, the original watermark is altered, e.g., by changing one or more message bits, to create the new unique identifier. In another embodiment, a second watermark is added to the derivative image to complement the first (or more) watermark. In this case, the first watermark identifies the original image, and the second watermark identifies the derivative.). Preferably, upon creating derivative 001a, the digital watermarking software removes the derivative 001 watermark (or at least a portion of the watermark, e.g., identifier ID- 5) from the derivative 001 image. Assignees' U.S. application 09/503,881 (WO0152181) discusses some techniques for such. Artisans know others still. Derivative 001a is then embedded with a unique digital watermark identifier (e.g., ID- 10).

The watermark embedding software can determine an appropriate identifier in a number of ways. In one embodiment, the embedding software queries database 314 for an appropriate, or available, identifier. In another case, embedding software (or user terminal 318) is assigned a range of identifiers, and an identifier is chosen from the available range. In still another embodiment, the embedding software randomly or pseudo-randomly selects the identifier, or alters a portion of the original image identifier, e.g., 2-32 bits of the original watermark identifier.

An image and a watermark identifier are combined to produce a digitally watermarked image (or derivative image) preferably in the same format and density as the input image. As an optional feature, software provides an indicator to signal success or failure in the watermarking effort. For example, the software can analyze whether the watermark was embedded, and/or whether the image contains the same format and density as the original input image. Upon a failure, user terminal 318 re- embeds the digital watermark or aborts the process.

Derivative image 001a is stored in database 314. Related information can also be stored in database 314. (As discussed above, database 314 may include a plurality of databases. One such database may manage images, while another database manages related information. Preferably, however, the unique identifiers are used consistently between the plurality of databases to link related images and information.). Database 314 links derivative 001a with image 001, derivative 001 and any related information (e.g., metadata, comments, files, history, security, etc.). Accordingly, image ancestry and any related information is efficiently maintained.

Figure 7 is a diagram further illustrating linking images, derivatives and related information via a unique watermark identifier. An original image 320 is watermarked with a unique identifier 322. A first derivative image 324 (e.g., perhaps an enlarged or cropped image corresponding with area 320a) and a second derivative 326 (e.g., corresponding to area 320b) are created. Each of the first derivative 324 and second derivative 326 are encoded with a unique watermark identifier. The derivative identifiers are associated with the unique identifier 322 in database 314. Such linking effectively groups image families together, permitting a user to gain access to image ancestry. Similarly, related information can be linked via unique identifier 322. Returning to Fig. 7, related information 328 corresponding to first derivative 324 are linked to identifier 322. Files 330 corresponding to the original image 320, as well as files 332 for the second derivative 326, are likewise linked. Accordingly, entire image families (and related information) are efficiently maintained by linking via the unique identifier 322. Of course, files 328 and 330 optionally can be separately, and respectively, linked to derivatives 324 and 326, via the derivatives' unique identifiers (e.g., as shown by the dashed lines in Fig. 7).

In one embodiment, a watermark provides information related to a permission level or a security clearance level. Such information can be reflected in a unique identifier or in a payload message.

Alternatively, the watermark identifier can be used to interrogate a database to retrieve related security level requirements. Such security information can be used to regulate access to images and related information. For example, only users (or user terminals) having a corresponding permission level or security clearance are allowed to access the corresponding image. Suitable software instructions can examine the permission level (or security clearance) to determine whether a user (or terminal) has the necessary clearance.

One aspect of certain embodiments of the present invention is to employ "rewritable" watermarks. A rewritable watermark includes a watermark of which all or a portion of which may be changed. In a preferred embodiment, only a portion (e.g., a portion of the payload) of a watermark is rewritten to update permission levels, reflect derivative work, etc.

There are often situations where it is desirable to carry some form of security access indicator in an image, e.g., via a digital watermark. The security access indicator defines a level of security required to view, edit or comment with respect to an image. Access to the image is then controlled by appropriately enabled software, which extracts the indicator (or receives the indicator from a watermark decoder) and determines usage. In one embodiment, the indicator indicates defines a required level. If a user's security level is equal to or greater (e.g., as determined from a password, user terminal identifier, login, linked security clearance level, etc.) to that carried in a security access indicator, then a user is allowed access to the image or data. In another embodiment, a security code may indicate that a particular user can view the image, but cannot edit or store comments regarding such. Consider the following example. An image "A" is defined to include an "unclassified" security classification. Image A's watermark then includes a unique identifier and additional plural-bits set to a predetermined number, e.g., all set to zero (or to a predetermined number or pattern). These additional plural-bits define the unclassified security classification. An image "B" is a derivative of image A, and has a "secret" security classification encoded in the plural-bits. Before either image A or B is opened (or requested) the security level contained within the watermark is validated against the security level of the individual requesting access, and permission is only granted to those with adequate clearance. In one embodiment, local software (e.g., executing on a user terminal) validates the security access by decoding the watermark, extracting the security bits, and comparing the security bits (or corresponding security level) with the security clearance of a user (or terminal). In another embodiment, software running on a central server monitors and validates security access. Or in another embodiment software associated with the database regulates the security access.

Application interface software, residing on a user terminal, helps to facilitate communication between image-handling software and database 314. The interface can be incorporated in such image- handling software, operate as a plug-in, be integrated with the operating system, or may even be called by certain operations (e.g., data retrieval, editing, saving, etc.). Preferably, the interface generates (or works in connection with) a graphical user interface (GUI) for a user. The GUI helps to facilitate user login, data retrieval, and image creation and saving. Creation is defined broadly to include any alteration to an existing image, or generation of a new image. Initially, a user is requested to enter a password or pass code to interact with database 314. After a successful log in, user access is preferably regulated based on security clearance. In other embodiments, permission levels or payment schedules are used to regulate access. An image, and related metadata, files, etc., should only be accessible when security access is permitted. In one embodiment, an image is selected from a directory, and the selected image is examined for watermarks. A watermark is extracted and security bits are determined. The security bits are validated against a corresponding security access allowed for the logged-on user. A user is permitted to access (e.g., retrieve, open, or edit) the image if she has an appropriate clearance. In an alternative arrangement, a database is queried to determine the security level required for all (or a subset of all) possible images in a directory or list. Only those images corresponding to the user's security clearance (or permission) are presented as options to open for the user. Even the names of the images can be screened from a user if her security clearance is insufficient.

The interface also facilitates communication in a normal image editing and creation processes. Preferably, the interface will be invoked as part of a saving process.

The creation process typically involves determining a new image identifier. As discussed above, there are many ways to determine an image identifier. In one case, the interface queries the database 314 to obtain a new image identifier. The retrieved image identifier is embedded in the newly created image as a portion of a digital watermark. The embedded image is then saved in database 314. Optionally, the database will signal that the save operation has been successfully completed. In the case of derivatives, the database is preferably updated to indicate that the new image identifier is related (or linked) to the identifier of an original image. In one embodiment, the above-mentioned steps (e.g., creating, watermarking, and saving) are considered a transaction, e.g., where all of the steps must be carried out for the transaction to be complete.

Another aspect of certain embodiments of the present invention is a gatekeeper module. With reference to Figure 8, a gatekeeper (or "sentry") 342 resides on network terminals 340 and 344.

Terminals 340 and 344 communicate, e.g., via a network, direct link, e-mail, etc. Sentry 342 monitors the flow of digital watermarked images and related information by extracting digital watermark identifiers or embedded security information from transmitted images. The sentry 342 can compare extracted information against user (or terminal) security clearance information. In one embodiment, sentry is an independent software module, although sentry 342 may be incorporated into other software components (e.g., applications, operating system, etc.) of a network terminal 340 and 344. Sentry 342 monitors and controls the flow of images at various points in a network system. Such activity is logged (e.g., recorded, stored, etc.) in database 346. To monitor an image transmitted from user terminal 340 to user terminal 344, sentry 342a decodes an embedded watermark identifier from an image to be transferred (step S10, Fig. 9). The identifier, destination address, and optionally a date/time stamp are communicated to database 346 (step S12), where such information is recorded as a data record (or file, log, table, database entry, history, etc.) as in step S14. Preferably such transmission activity is associated with the unique identifier of the transferred image.

Sentry 342 is also gatekeeper in that it analyzes whether a user's security or permission level is sufficient to receive a watermarked image into or from a workstation (e.g., whether terminal 344 can receive the image transmitted from terminal 340). Sentry 342b preferably includes watermark-decoding software, which extracts unique identifiers (and any security bits) from watermark images (step S20, Fig. 10). If the security level of an image is stored in a database, sentry 342b queries the database with the identifier to determine the required access level. Or if the watermark includes security bits, then sentry 342b determines an access level directly from extracted security bits. Sentry 342b determines whether the user's security clearance sufficiently corresponds with the received image's clearance requirements (step S22). If so, sentry 342b allows terminal 344 to receive and open the subject image (step S24). If not, sentry 342b denies terminal 344 access to the image (step S26). In either case, sentry 342b preferably communicates such information to database 346 (step S28). For example, sentry 342b records whether the image is passed to terminal 344, or whether the image is denied. (As an optional function, sentry 342b notifies terminal 342 regarding the delivery status of the image.).

Sentry 342 typically does not performed the function of managing the relationship between images and their derivatives, as this is the function of the file save software associated with the image editing application. However, in one embodiment, sentry 342 is combined with an application interface. Sentry 342 can be deployed in a number of ways. In one embodiment, sentry 342 is integrated

(or stored, or connected) to a workstation or server in such a way that all image data must first pass through the sentry 342. In another embodiment, sentry 342 includes a separate hardware (or hardware/software) device inserted between a network (or network connection) and a user terminal. As such, sentry 342 decodes watermarks and intercepts passwords from image traffic before the user terminal receives the image, or directly after transmitting an image.

In another embodiment, e.g., in a TCP/IP environment, a sentry 342 is deployed as software within a TCP/IP stack in the user station or server. In yet another embodiment, a sentry is incorporated in (or called by) an image-handling program's open, save and close operations. When used in connection with a database history or other record, sentry 342 provides efficient tracking and tracing. Since the history file reveals each use (and printing, transmission, etc.) of a watermarked image, the image can be efficiently tracked as it passes from terminal to terminal, or from database to terminal, etc.

Some images may include at least two watermarks. A first watermark includes a unique identifier, as discussed above. This identifier allows database inquires and association as discussed above. A second watermark can be applied prior to printing, faxing, etc. This second watermark preferably includes a so-called fragile watermark. As earlier noted, a fragile watermark is designed to be lost, or to degrade predictably, when the data set into which it is embedded is processed in some manner.

Once an image is printed, it then includes both the first and second watermarks. If the image is subsequently scanned back into a digital form, e.g., via a scanner, photocopier, web cam, digital camera, etc., the fragile watermark is corrupted (or destroyed) in a foreseeable manner. Printed copies can be tracked and traced accordingly. For example, a photocopied image is scanned into a digital form. The first watermark is used to identify the image and retrieve an image history (e.g., as created by a sentry or other logging method). Since the fragile watermark is destroyed (or predictably degraded) in the copy process, the photocopy is determined to be an unauthorized copy. The history log can be used to determine which user, or user terminal, printed the copy.

It will be recognized that watermarks can be applied to any data set (e.g., an image, map, picture, document, audio clip, video program, etc.) for forensic tracking purposes. This is particularly useful where several copies of the same data set are distributed through different channels (e.g., provided to different users). Each can be "serialized" with a different identifier, and a record can be kept of which numbered data set was provided to which distribution channel. Thereafter, if one of the data sets appears in an unexpected context, it can be tracked back to the distribution channel from which it originated.

In an alternative embodiment, with reference to Fig. 5, a digital watermark embedder is included in aerial platform 311. The aerial embedder embeds images (e.g., after or during capture) and relays such to ground station 312. In yet another embodiment, an image is digitally watermarked downstream from ground station 312, such as in a user terminal, or an embedder associated with the databases 314 and/or 315.

Although not belabored, artisans will understand that the systems described herein can be implemented using a variety of hardware and software systems. One embodiment employs a computer or workstation with a large disk library, and capable database software (such as is available from

Microsoft, Oracle, etc.). The watermarking and database operations can be performed in accordance with software instructions stored in the disk library or on other storage media, and executed by a processor in the computer as needed. (Alternatively, dedicated hardware, or programmable logic circuits, can be employed for such operations.) It should be appreciated that embodiments according to the present invention are not limited to managing satellite and other aerial imagery. Indeed, other imagery may be so managed. Also, the techniques disclosed herein can be used in "non-secure" systems. In one such embodiment, watermark identifiers are used to link images and/or related information. A security or permission level is not required in such a system.

Fifth Set of Embodiments

Fig. 11 shows a representative (but blank) image 510 (composed of rows and columns of pixels, not particularly shown). For purposes of discussion, image 510 may be an aerial photo of land, but it should be recognized that imagery 510 is not so limited.

Due to various factors (e.g., camera lens artifacts, and the perspective from which the photo was taken, etc.), the rectangular photo typically does not depict a rectangular area of land. Instead, the area of land depicted may actually be trapezoidal, or of other shape. Overlaid on the Fig. 11 image are sample latitude and longitude lines 512, 514. These are virtual and do not appear in the actual image. (The straightness of the lines is unusual. In most landscape images, the receding horizon tends to curve any latitude or longitude projections that are not parallel to the image boundaries.)

Each point depicted in the image 510 has a unique position that may be expressed by latitude and longitude (and, if 3D accuracy is desired, elevation) coordinates. In accordance with an illustrative embodiment of the invention, such position data for a single location depicted in the image is determined. (Various techniques can be employed, e.g., reference to a pre-existing map or database, ground-truth measurements using GPS equipment, etc.) This location, and the pixel 516 corresponding thereto in the image, are termed the "arbitrary origin" in the discussion that follows. (For expository convenience, the arbitrary origin in this discussion is the upper-left-most pixel in the image, and the ground point corresponding thereto.)

The image 510 is digitally watermarked across its extent with a payload that includes the coordinates of the arbitrary origin (e.g., latitude/longitude/elevation). In addition, the watermark payload also includes a parameter (e.g., angle 518) identifying the orientation of a vector pointing from the arbitrary origin to a known direction (e.g., true north). The watermark payload can also include a scale datum, e.g., indicating that 100 pixels to the right (along the row) from the arbitrary origin corresponds to a distance - on land - of 250 yards.

As noted, a rectangular image generally does not depict a rectangular piece of land. Moreover, even if a photo is taken from directly overhead - using a lens that introduces no aberrations — there is the slight complication posed by the fact that longitudinal lines are not parallel, but meet at the poles.

Accordingly, if high accuracy is desired, the watermark can additionally convey coefficients for one or more polynomials (e.g., one for each coordinate axis), which model the apparent warp of the photographic depiction along different axes. (In an exemplary arrangement 5 coefficients of 8 bits each are provided for the latitude and longitude polynomials, and 6 coefficients of 8 bits each are provided for the elevation polynomial.)

Thus, in an exemplary embodiment, the watermark payload may comprise the following (196 bits total):

Digital watermarking is not belabored in this specification because such technology is well understood by artisans in the field of steganography. Briefly, however, watermarking typically works by making subtle changes to the brightness of image pixels, conveying message payloads that can be detected by suitable detector software or hardware. The embedding process generally adjusts to the unique characteristics of the image, placing a stronger watermark signal in areas with rich detail and a weaker watermark signal in areas with little detail. Because the payload is carried by the image's pixels, it is file-format independent. The payload can survive most normal processing operations, such as compression, edits, file format transformations, copying, scanning and printing. Some watermarking techniques are also robust against rotation and scaling, e.g., through use of embedded calibration data, or auto-correlation techniques.

Any watermarking technique can be employed in the present invention, provided the requisite number of watermark payload bits can be embedded without introducing objectionable corruption into the image. Examples of suitable watermarking techniques are found in the patent documents cited above. In a particular embodiment, the watermark payload is represented in a single 128 x 128 pixel patch 520, which is then tiled across the image (with local scaling to reduce visibility). Each patch comprises 16,384 pixels. In Fig. 11, one of the patches 520 is indicated by cross-hatching for ease of identification.

To enhance robustness, the watermark payload may be processed, e.g., by BCH, Reed-Solomon, convolutional, or turbo coding, or the like, to provide error detecting/correcting capability. Such coding has the effect of transforming the 196 bit payload bits into, e.g., 320 bits ("raw bits"). Each of the 16,384 pixels in the patch is encoded with one of these raw bits, so that each such bit is represented about 50 times per patch. The pixels corresponding to a single raw bit are desirably distributed across the patch, so that severe corruption of a small area of the watermarked image does not irretrievably lose certain raw bits.

On the detection side, the image is processed to retrieve the 320 raw bits, and then the 196 payload bits are determined from the raw bits. From these payload bits, a user of the image knows the geographical coordinates of the point at the arbitrary origin and, through use of the other encoded parameters, can deduce the geographical location of any other point depicted in the image.

In other embodiments, more elaborate watermark encoding can be used. For example, instead of tiling the identical watermark patch over and over across the image, each patch can be slightly different, e.g., encoding the position of that tile within the array of tiles. In one arrangement the tile position data is a pair of numbers indicating tile-row/tile-column offsets from the tile containing the arbitrary origin. Referring to Fig. 11, the tile containing the arbitrary origin 516 may be designated

{0,0}. The tile next to it in the row may be designated {0, 1 }, etc. These index values may be encoded as

5 bits each, which bits are included in the watermark payload. This arrangement offers advantages in environments in which image cropping, rotation, or other image transformations may occur. By decoding the payload from a watermark tile, its location relative to the arbitrary origin can be determined, and the location of the arbitrary origin 516 can thus be inferred (even if that point has been cropped out of the image).

In still other embodiments, information about the image perspective can be conveyed through a watermark. Various forms of representation are possible. In one, the image perspective data can comprise the compass angle at which the camera is pointing (?), and the elevation angle between the arbitrary origin point and the camera (f ). The former may be represented, e.g., by 10 bits, the latter by 8. Additionally or alternatively, the perspective data can identify the lens or its attributes, so that optical distortion of the image can be characterized. In an index-based system, a six-bit code can be used to identify one of 64 different lenses.

With different types of imaging systems, different forms of perspective information may be appropriate. For example, in so-called "whisk broom" cameras (i.e., those that repeatedly acquire line scans from a moving viewpoint), the perspective information may additionally include the starting and ending positions (the latter may be expressed as an offset from the former, allowing some payload conservation).

In yet other embodiments, elevation data for different points in the image can be encoded through watermarks. In one such arrangement, elevation data is determined for points at 64-pixel gridded spacings across the image. These points are designated in Fig. 11 by the stars labeled 522. (Only a few such stars are shown in Fig. 11. The arbitrary origin 16 is also such a point.) The elevation may be expressed in absolute terms (e.g., feet above sea level), or relative to another reference (e.g., the elevation of the arbitrary origin). Again, 16 bits per elevation may be used. (Or if difference in elevation from the arbitrary origin is used, then 8-12 bits may suffice.)

In one such arrangement, the elevation data for each starred point is watermark-encoded in a 64 pixel by 64 pixel subpatch 524 centered around the star. Again, one such sub-patch 524 is shown in Fig. 11 by cross-hatching for ease of identification. More generally, these sub-patches 524 are the regions bounded by the fine, dotted lines in Fig. 11. Again, sub-patches 524 are tiled across the image, but each one conveys a (typically) different elevation payload.

(In the Fig. 11 arrangement, each patch 520 encompasses one full sub-patch 524, and parts of eight others. By this arrangement, elevation data is encoded for the points at each comer of each patch 20, as well as for the point at the center of the patch, and at points mid-way along each patch side boundary. In other embodiments, of course, sub-patches 524 can be sized and positioned differently relative to patches 520.)

The elevation watermark, based on patches 524, may be simply overlaid on the main watermark, based on patches 520. Desirably, however, there is some coordination between the two watermarks, so as to avoid extreme changes in any pixel values (as may occur, e.g., if both watermarks try to change a pixel by a maximum amount permitted by the respective watermarking technique). In one such coordination arrangement, each pixel in the image is assigned to one of the two watermarks. For example, 50-80% of the pixels in the image may be assigned to the main watermark, and 50-20 % may be assigned to the elevation watermark. The assignment may be done based on a regular array, or a stochastic assignment may be used. In some cases, it may be prudent to allocate extra pixels to carry the elevation payload where - as in the upper left - the sub-patch 524 extends beyond the boundary of the image, limiting the number of pixels to convey elevation data (e.g., for arbitrary origin point 516).

(This coordination technique has applicability beyond the present context, and is generally applicable to a variety of watermarking applications in which plural watermarks are used.)

By techniques such as the foregoing, an image can be provided with extensive photogrammetric information that travels with the image, notwithstanding distortion, cropping, format conversion, etc.

This data can be exploited in various ways. One utilizes a computer system on which the image is displayed, e.g., on a monitor or screen. An operator uses an input device, such as a mouse, light pen, graphics tablet, or the like, to designate a particular point in the displayed image. In response to selection of the point (by clicking or other known selection technique), the computer processes the embedded watermark information and displays to the operator the precise latitude, longitude and elevation of the selected point.

Using the elevation data, the computer system can also generate, and render, a 3D view of the depicted landscape, from an arbitrary viewing angle. Surfaces that are hidden in the original image may be extrapolated using known techniques, and presented in a different color or texture to indicate their synthetic basis.

In embodiments in which the camera perspective is known, the projections of latitude and longitude lines on the depicted terrain can be adjusted, e.g., in accordance with variations in elevation. If the camera perspective is such that it is viewing down a downwardly-inclined slope, for example, the latitude or longitude lines that traverse this slope can be virtually placed more closely spaced together than would be the case if the camera view were orthogonal to the slope.

The mathematical manipulations associated with such operation are somewhat complex, but well within the skills of those working in the photogrammetric and mapping arts. To determine elevation at an arbitrary point and to generate 3D models, for example, the elevations at the starred points 522 are provided to an algorithm that applies a bi-cubic spline-fitting model so as to estimate the elevation at any point on the image.

(The computer system can take various forms, but most include conventional computer components such as one or more CPUs, volatile storage (e.g., RAM), non-volatile storage (e.g., ROM, fixed and removable magnetic disks, fixed and removable optical disks), interfaces (e.g., WAN, LAN,

USB, modem, serial), input/output devices (e.g., monitor, keyboard, mouse, tablet, joystick, light pen), etc. Associated with the computer system is various software, including operating system software and applications software — the latter being programmed to perform the data processing and presentation operations detailed above. Naturally, such programming can be stored on fixed or removable computer storage media. In some embodiments, parallel or distributed computer architectures may be employed, e.g., with different components of the computer system being located remote from each other.)

A variety of aerial mapping and associated image database techniques can be used in conjunction with the present invention. Representative systems are shown, e.g., in patents 5,608,405, 5,926,581, 5,974,423, 6,023,278, 6,177,943, 5,995,681, 5,550,937 and 6,150,972.

While the "arbitrary origin" was the pixel in the upper-left corner of the image in the illustrative example, this placement is not critical. The arbitrary origin can be moved to any location, with relative measurements being adjusted accordingly.

Likewise, while the detailed embodiment contemplated that the coordinates of the arbitrary origin are literally encoded as part of the digital watermark payload, in other embodiments this need not be the case. Instead, e.g., the watermark payload can be an arbitrary identifier that identifies an entry in a data structure (e.g., table or database) in which the coordinate data is stored. The same index-a-remote- store approach can be used with any of the other payload data.

Although latitude/longitude/elevation were used as exemplary coordinates, it will be recognized that other coordinate geometries can alternatively be employed.

The main watermark payload is described as including coordinate, orientation, scale, and polynomial correction data. Depending on the application, certain of this data may be omitted, and/or certain additional information may be included in the watermark payload. The payload length is exemplary. Some embodiments can employ a payload that is considerably shorter (e.g., by abbreviating the bits dedicated to each data and/or omitting certain data). Other embodiments may employ a payload that is longer.

While elevation may be expressed in height above sea level, this need not be the case. Height relative to any other measure can alternatively be employed.

The illustrative embodiments' encoding of plural data (e.g., coordinate data, and lens data) in a single watermark payload is not essential. In other embodiments, the different elements of embedded information can be conveyed through distinct watermarks, e.g., layered over each other, interspersed between each other, coordinated with each other in the manner of the elevation watermark, etc.

In many embodiments, lossless data compression techniques (e.g., Lempel-Ziv based) can be employed to reduce the number of payload bits that are encoded in a watermark. Although described in the context of watermarking for mapping and photogrammetric purposes, the principles detailed herein find application in many other watermarking applications, not limited to the purposes particularly detailed. Concluding Remarks

To provide a comprehensive disclosure without unduly lengthening this specification, the above-mentioned patents and patent applications are hereby incorporated by reference. The particular combinations of elements and features in the above-detailed embodiments are exemplary only; the interchanging and substitution of these teachings with other teachings in this application and the incorporated-by-reference patents/applications are also contemplated. The features found in one section may be readily combined with those features in another section.

As will be apparent, the technology detailed herein may be employed in reconnaissance and remote sensing systems, as well as in applications such as guidance of piloted or remotely piloted vehicles.

It should be understood that the technology detailed herein can be applied in the applications detailed in the cited DEM patents, as well as in other mapping and image (or audio or video or other content) asset management contexts. (Likewise, the technologies detailed in the cited patents can be advantageously used in embodiments according to the present invention.) While particular reference was made to Digital Elevation Models and albedo maps, the same principles are likewise applicable to other forms of maps, e.g., vegetative, population, thermal, etc., etc.

While certain of the illustrated embodiments correlated the incoming imagery with a projective image based on the master DEM/map, in other embodiments a reference other than the master DEM/map may be used. For example, a projection based just on part of the historical data from which the DEM/map was compiled can be used (e.g., one or more component data sets that are regarded as having the highest accuracy, such as based directly on ground truths).

Likewise, while certain of the foregoing embodiments contemplate a handheld reading device, the invention is not so limited. An input device may be tethered to a desktop or laptop computer or compliant kiosk, an input device may also communicate with the computer via a wireless channel, etc. As an alternative, a handheld device (or an input device) may scan a map area to capture embedded data. The handheld device wirelessly communicates the captured data to a networked computer. The computer decodes the captured data (e.g., data including a watermark). The computer can then access networked information via the decoded watermark. The data can be wirelessly communicated to the handheld device for display or may even be communicated to a separated device for display. Still further, while the figures illustrate the map areas as rectangular-shaped, this is naturally not essential. Indeed, other area shapes may be advantageously employed. Moreover, in other embodiments, individual map locations may be watermarked, instead of watermarking individual blocks. For example, on a state map, all city locations may be watermarked according to respective locations. In a city, streets and building locations are watermarked. In another example, areas corresponding to roads, streams, attractions can also be watermarked.

Although not belabored, artisans will understand that the systems described above can be implemented using a variety of hardware and software systems. Some embodiments may employ a computer or workstation with a large disk library, and capable database software (such as is available from Microsoft, Oracle, etc.). The registration, watermarking, and other operations can be performed in accordance with software instructions stored in the disk library or on other storage media, and executed by a processor in the computer as needed. (Alternatively, dedicated hardware, or programmable logic circuits, can be employed for such operations.)

Others of the embodiments use handheld devices with built-in processors. It will be recognized that client-server architectures can be used, with the processing divided between one computing device at the user's location, and one or more others at a remote location. The watermark encoding/decoding may be performed at the user location, or at a remote location to which the user's computer passes data.

Certain of the techniques detailed above find far application beyond the context in which they are illustrated. For example, equipping an imaging instrument with an optical shutter that imparts a watermark to an image finds application in digital cinema (e.g., in watermarking a theatrical movie with information indicating the theatre, date, time, and auditorium of screening).

In view of the wide variety of embodiments to which the principles and features discussed above can be applied, it should be apparent that the detailed embodiments are illustrative only and should not be taken as limiting the scope of the invention. Rather, we claim as our invention all such modifications as may come within the scope and spirit of the following claims and equivalents thereof. (For expository convenience, the term "map" as used in the claim should be construed to encompass terrain models, such as DEMs.)

Claims

WE CLAIM

1. In a method of compiling satellite imagery and generating a map therefrom, an improvement comprising: watermarking image data acquired by a satellite; storing the watermarked image data in a database; generating a map from the database; and watermarking the map.

2. In a method of generating a digital map from a database containing data from a plurality of aerial sources, an improvement comprising watermarking the map.

3. A database storing plural sets of component map data, from which a composite map formed using said component map data can be formed, characterized in that said plural sets of component map data each are encoded with a different watermark, said watermark encoding, or linking to, meta data associated with each said component map data.

4. A map divided into a plurality of areas, with each area comprising at least one embedded digital watermark including location information for the respective map area.

5. An apparatus to read digital watermarks embedded within a map, the map being divided into a plurality of areas, with each area comprising at least one embedded digital watermark including location information for the respective map area, said apparatus comprising: a global positioning system receiver to determine a location of said apparatus; an input device to capture an image of at least a portion of the respective map area; memory including executable software instructions stored therein, the instructions to extract the location information from the at least one embedded digital watermark from the captured image of at least a portion of the respective map area, and to correlate the location of the apparatus with the extracted location information; electronic processing circuitry to execute the software instructions; and an output device to indicate the correlation of the apparatus location and the captured watermark location information.

6. A method of making a map comprising an improvement of: dividing a map into a plurality of areas; steganographically encoding plural-bit location data within each of the plurality of areas, wherein the location data is unique per each of the plurality of areas.

7. A method of navigating with a map embedded with digital watermarks comprising the steps of: extracting a digital watermark from the map, the digital watermark including location information which uniquely identifies the respective map watermark extraction area; comparing the location information to a physical location; and providing feedback to correlate the location information and the physical location.

8. A method of correlating a physical location to a map location, the map being divided into a plurality of areas, with each area comprising at least one embedded digital watermark including location information for the respective area, the method comprising the steps of: extracting the location information from the watermark at the map location; comparing the extracted location information to global positioning system (GPS) received coordinates of the physical location; providing feedback based on the comparison of the physical location and the map location.

9. A sign having plural bit data encoded thereon in the form of a digital watermark, the data comprising a unique identifier.

10. A method comprising the steps of: capturing an image of a sign; extracting a digital watermark from the captured image, the watermark including plural-bit data; and outputting a response in accordance with the plural-bit data.

11. An apparatus to read digital watermarks embedded within a map, the map being divided into a plurality of areas, with each area comprising at least one embedded digital watermark including location information for the respective map area, said apparatus comprising: a global positioning system receiving means for determining a location of said apparatus; input means for inputting an image of at least a portion of the respective map area; memory means for maintaining executable software instructions stored therein, the instructions to extract the location information from the at least one embedded digital watermark from the captured image of at least a portion of the respective map area, and to correlate the location of the apparatus with the extracted location information; processing means for processing the software instructions; and output means for outputting a correlation of the apparatus location and the watermark location information.

12. An apparatus to read digital watermarks embedded within a map, the digital watermarks including location information for respective map locations, said apparatus comprising: a global positioning system receiving means for determining a physical location of said apparatus; input means for inputting data corresponding to at least a portion of the respective map area; processing means for extracting the location information from the input data and for correlating the physical location with the extracted location information; and output means for outputting an indication of the relative correlation between the apparatus location and the watermark location information.

13. A method comprising the steps of: accessing a database comprising information; retrieving a subset of the database information; storing the retrieved subset of database information in a handheld computing device, the handheld device including an input device; capturing a portion of a digitally watermarked map by the input device, the portion including at least one watermark comprising map location information; in the handheld computing device, determining which of the retrieved subset database information corresponds to the map location information; and providing the corresponding retrieved subset database information as feedback.

14. A method comprising the steps of: inputting a map location to a computing device; determining a current location; in the computing device, determining a relationship between the input map location and the current location; and providing directions from the current location to the input map location.

15. A method of maintaining images in a database, wherein a first image includes a digital watermark embedded therein, the digital watermark comprising a first unique identifier, said method comprising the steps of: storing the first image to be indexed by the first unique identifier; storing information related to the first image; and linking the first image and the related information by the first unique identifier.

16. A method for managing images, the images including a first image comprising a first identifier steganographically embedded in the first image in the form of a digital watermark, said method comprising the steps of: retrieving the first image from a database; altering the first image to create a second image; steganographically embedding a second identifier in the second image in the form of a digital watermark; and associating the second image in the database with the first identifier.

17. A method to monitor images in a system, the system comprising at least a first user terminal to communicate with a second user terminal and with a database, the images comprising at least a first image digitally watermarked to include a first identifier, said method comprising the steps of: determining a security level associated with the first image; comparing the first image security level with a user security level; and allowing access to the first image based on a result of said comparison step.

18. A system comprising: a first user terminal; a second user terminal; a database, wherein the first user terminal and the second user terminal are in communication, and the first user terminal and the second user terminal are each in communication with the database; and a gatekeeper to regulate the flow of an least a first image between the first user terminal and the second user terminal, wherein the first image comprises at least a first digital watermark including a first identifier, said gatekeeper to determine a security level associated with the first image, compare the first image security level with a user security level, and to allow access by the second user terminal to the first image based on a result of the comparison.

19. A module for use in a network comprising at least a first terminal in communication with a database, said module to monitor the flow of an least a first image at a first network location, the first image comprising at least a first digital watermark including a first identifier, said module comprising: means for determining a security level associated with the first image; means for comparing a first image security level with a user security level; and means for allowing access to the first image based on a result of said comparing means.

20. A method comprising digitally watermarking an image with a payload that represents at least two position coordinates for a point depicted within the image.

21. A method comprising digitally watermarking an image with a payload that includes first and second portions, said watermarking using a tiled approach, wherein uniformly-sized patches of the image are processed in accordance with the payload, wherein the first portion is unchanging across all of said tiles, but the second portion changes between tiles, so that position information about each tile can be determined therefrom.

22. A method comprising digitally watermarking different regions of an image with different watermark payload data, wherein a first region of the image is watermarked with payload data relating to an elevation of terrain depicted in said first region, and a second region of the image is watermarked with payload data relating to an elevation of terrain depicted in said second region.

23. A method of compiling aerial imagery and generating a map there from comprising the steps of: digitally watermarking image data to include imagery characteristics corresponding to the image data, the image data acquired by an aerial platform; correlating the image data based on the imagery characteristics; and generating a map from the correlated image data.

24. A method of managing aerial imagery comprising the steps of: watermarking patches of the aerial imagery, wherein each patch includes at least one watermark, the at least one watermark including an index; storing in a database a plurality of data records corresponding to a range of watermark indexes, wherein the data records comprise imagery characteristics.

25. A method of generating a map comprising the steps of: steganographically encoding data in the form of a digital watermark in each of a plurality of patches, said encoded data including a location indicator; and piecing together the plurality of image patches based at least in part on the location indicator.

26. A method of correlating imagery data generated under a plurality of different conditions, said method comprising the step of: embedding imagery characteristics in the imagery data; and modifying the imagery data based on the embedded imagery characteristics so as to standardize at least some of the imagery data.

27. A data structure stored on a computer readable medium, the data structure comprising an aerial image including an embedded watermark, said watermark including a geographic locator.

28. A data structure stored on a computer readable medium, the data structure comprising an aerial image including embedded data in the form of a digital watermark, said digital watermark including imagery characteristics.

29. A method of marking a photograph comprising the steps of: obtaining geovector information corresponding to a location depicted in the photograph; and

digitally watermarking the geovector information in the photograph.

30. A method comprising the steps of: digitally watermarking location information in an object; and linking the information to at least one data record.

31. An article of manufacture comprising steganographically embedded data in the form of a digital watermark, the watermark comprising location information.

32. A method of notarizing a document, comprising the steps of obtaining a geovector and embedding the geovector in the document in the form of a digital watermark, the geovector comprising date and location information.

33. A method of making a map comprising the steps of: obtaining first geovector information corresponding to at least a first region to be depicted by the map; and digitally watermarking the first geovector information in the map.