WO1995007000A1 - Video camera for recording labelled items - Google Patents

Abstract

An electronic camera back (1604) functions as an in-camera frame grabber. It may be attached to a camera body (1601) that focuses an image on a supported (1612) CCD chip (1607). The image is then read out and digitised, then compressed in a JPEG processor (1611) before being stored in a buffer (1605) ready for onwards transmission. Multiple images may be stored. Substantial compression of for example 20:1 is used for the purpose of capturing multiple machine-readable labels as compressed images from collections of merchandise (like tree trunks being loaded onto a ship) for later decoding by image analysis methods. An example 4096 x 4096 pixel CCD chip image is read out, compressed, and stored as about 600K bytes of data in about 6 seconds. Range finding an exposure compensating equipment such as lighting and neutral-density filters are options. Other applications include remote surveillance such as of aircraft at airfields using telecommunications to convey compressed image data.

Description

VIDEO CAMERA FOR RECORDING LABELLED ITEMS

FIELD

This invention relates in general to information gathering means for locating and identifying marked or labelled objects during storage and transport. It relates to apparatus for electronic image acquisition, computer vision, and image analysis. More specifically the invention provides high-resolution image acquisition means adapted for use in a variable and uncontrolled environment and may be applied to the reliable recovery of inventory data from one or more labels included within each image.

BACKGROUND

The timber industry, like many others, has a particular need for the convenient and preferably automatic recording of individual commodity items (logs and processed timber in this case), noting in particular the location of each item at a recorded time. Often the recording act must be carried out in outdoors environments during transportation, processing, and storage.

Logs may be initially identified at a weighing station by ownership, location, handler, type, or quality. Many other items of goods apart from logs, even containers of goods may similarly need to be identified and located from time to time and at one place or another. This information should help with the management of stocks and resources (particularly important with the "just-in-time" management practice), in optimising the flow of transport and movement of goods, and in maintaining a watch over property. Information may be used to categorise shipments, to prepare processing stations, or to pinpoint the whereabouts of each of many logs during movements. Accurate identification can aid in efficient operation and production management and can help reduce stock loss. The nature of the forestry industry, where bulk handling is now commonplace and where each log may have a value of the order of NZ $1000, imposes particular requirements. A recording process should be fast, for wasted time is expensive. It should not involve danger to operators and therefore a form of remote sensing is preferable to (for example) having a person make a close approach to read and record labelling data, as when the logs are on a transporter, cradle, crane sling, or the like.

Bulk handling, where entire logs are often picked up by transporters in bundles, implies a need to record items in bulk and accordingly labels as described in our co-pending New Zealand patent application, No. 250406 have been developed for machine recognition and decoding in situations where a single captured image showing labels on items in bulk might contain 10, 40 or more labels.

One aspect where conventional technology is inadequate is that of image capture. A conventional solid-state video camera according to NTSC, PAL, or CCIR standards may record a total of from 0.06 to 0.3 million pixels. Although these cameras can be used on single labels virtually filling the image area, their resolution is insufficient to adequately resolve the patterns printed on each of a group of many labels in the one image. In order to gather information speedily from an accumulation of labelled items there is a need for a higher resolution than can be provided by conventional video cameras. Similar considerations apply if a label occupies only a small part of an image. CCD chips having a higher pixel count - up to or over 4096 x 4096 pixels in X and Y directions are known, and a 6K x 6K array would have over 100 times as many pixels to be read as are in a standard video camera.

Yet the commercial usefulness of a multiple-image identification system is largely a function of its speed. For example there is a requirement to be able to move all the pixels from one captured image to a downstream collector (typically involving the techniques of digital computation) and thereby clear the data from within the camera before it is time to take the next image. In many industries, the image capture and transfer speed may have to be related to external timings, such as the passage of items on a conveyer belt or the speed of a moving camera platform passing along storage racks. Previous applications such as astronomy involving the use of high-resolution CCD arrays have generally been able to tolerate long readout times of perhaps 50 to 250 seconds per image.

There is therefore a clear need to provide a high-resolution camera with a modular data acquisition system having a short readout time or readout cycle time so that high-resolution optical data capture and computer analysis of the resulting image can be extended to commercial applications requiring rapid, repeated data capture.

A further need is for a high-resolution surveillance camera, preferably having good performance under low lighting conditions, optionally equipped with means for remote operation.

An identification system should be modular so that it can be reconfigured from standard building blocks or modules for different applications or for different operating modes.

OBJECT

It is an object of this invention to provide an electronic camera, or parts therefor, for label detection and identification, or at least to provide the public with a useful choice.

STATEMENT OF INVENTION

In a first broad aspect the invention provides a frame grabber capable of receiving a first digital signal representing an image, comprising means to compress the first digital signal by at least 4: 1 in real time to produce a compressed digital image having a reduced number of bytes compared to the total number of bytes of the first digital signal, and buffer means to store at least one compressed digital image.

In a subsidiary aspect the invention provides a frame grabber wherein said means to compress the digital signal is capable of compressing the digital signal in the range of 20:1 to 30:1.

In another subsidiary aspect the invention provides a frame grabber wherein said buffer means is capable of storing at least two images. In a further subsidiary aspect the invention provides a frame grabber wherein said means to compress the digital signal has at least two input channels capable of processing the information in parallel.

In a yet further subsidiary aspect the invention provides a frame grabber wherein said means to compress the digital signal comprises a JPEG processor.

In a second broad aspect the invention provides image capture means for a camera comprising at least one solid-state CCD light-sensitive array capable of being positioned at an image plane of a camera, at least one analogue output channel from said CCD array, one or more analogue to digital converter(s) for converting analogue signals from the or each analogue output channel to one or more digital signals, and a frame grabber connected to the output(s) of the analogue to digital converters).

In a subsidiary aspect the invention provides image capture means wherein the solid-state CCD light-sensitive array is an X-Y array having at least 2048 pixels along each edge (i.e. a total of 2 ^ pixels)

In another subsidiary aspect the invention provides image capture means wherein the solid-state CCD light-sensitive array is an X-Y array having at least 4096 pixels along each edge (i.e. a total of 2 ^u pixels).

In a further subsidiary aspect the invention provides image capture means further comprising means to communicate the one or more compressed digital image(s) to an external image processing unit or other receiver of image data.

In a third broad aspect the invention provides a camera for recording labelled items comprising image capture means and optics including image focusing means capable of focusing an image on the CCD array, aperture control means, and shutter means.

In a subsidiary aspect the invention provides a camera wherein the camera is mounted on a self-powered mobile platform so that the camera can be positioned relative to one or more labelled items to initiate a data capture process. Iz ά further subsidiary aspect the invention provides a camera wherein means are provided to laterally move the camera along a guide rail, and to tilt and pan the camera when at selected positions in order to repeat the process of capturing an image from more than one viewpoint.

DRAWINGS

The drawings and the associated descriptions of preferred embodiments of this invention are provided by way of example, and are not intended to be in any way limiting as to the scope or extent of the invention.

Fig 1: This is a diagram showing a vehicle with a modular camera and recording system mounted on the front, image interpretation devices within the vehicle which is equipped with a radio link, and a pile of logs with identification labels on the ends in position for recording the encoded labels displayed on the ends of the logs.

Fig 2: This block diagram shows data and signal flow between major components of the modular recording system of Fig 1.

Fig 3: This diagram shows a front view of a modified vehicle which has the camera and ancillary equipment mounted in front.

Fig 4: This diagram shows a side view of the vehicle with the attached camera and ancillary equipment.

Fig 5: This diagram depicts details of the camera carriage and the guide rail with attached motor and sensors, in a side view.

Fig 6: This diagram shows details from the top of the camera support mounted on the guide rail.

Fig 7: This diagram shows details from the front of the camera support mounted on the guide rail. This diagram shows details of a spring-loaded CCD mount support allowing a CCD to be used within a photographic camera body.

This diagram shows details of the camera back replacement which locates the CCD at the film plane of a standard photographic camera.

This diagram shows the operator's console which includes rangefinder and positioning information.

This diagram shows in plan view a preferred hydraulic powering system for use on the camera housing to control flaps, filters, and the like.

This diagram shows a side view a preferred hydraulic powering system for use on the camera housing to control flaps, filters, and the like.

This diagram shows the front view of a preferred hydraulic powering system for use on the camera housing to control flaps, filters, and the like.

This diagram shows a portable version of the camera system optionally attached either to a portable computer carried in a backpack, or attached from time to time to a docking station. The operator is shown taking a picture of a pile of logs.

This diagram shows a fixed version of the camera system attached to an adjacent image processing computer, both mounted on a mast or pole. The camera is aimed at the rear of a trailer loaded with labelled logs.

Shows a plan view of a camera including outlines of internal data-reduction circuit units.

Shows a block diagram of a camera including internal data-reduction circuit units, a battery, and a transmitter. PREFERRED EMBODIMENTS

In summary this invention offers the particular advantages of being able to remotely record and decode one or more labels on merchandise being transported or in storage.

Previous image recorders are restricted (by resolution) to considering single labels when under relatively ideal viewing conditions. This invention uses a high-resolution electronic image capture camera to generate a detailed digital version of an image including possibly many labels. The image so captured may be scoured by an image analysis computer hunting for labels of a type adapted for computerised recognition, that decodes and presents the information within each label. Each label is preferably unique to that particular item of merchandise. The camera may in fact be a back adapted to fit an existing camera. The invention may be regarded as a kind of CCD array/frame grabber/image storage combination, for use at the film plane, which compresses the image data as soon as the image is taken, so that its digital representation takes up considerably less storage space than if it was not compressed.

A particular application for which this invention has been developed is that of monitoring the flow of timber; principally whole logs, from the forest to the final destination. In recent times the value of a single log of Pinus radiata has risen to be of the order of NZ $1000. There is a need for a device capable of reliably and economically showing that a certain labelled log was at a certain place at a certain time. One embodiment of this invention is intended to be moved about a dockside area, for example, capturing images of labels on bundles of logs stacked on cradles ready for shipping, extracting the data within the labels and forwarding it to an administrative computer facility which keeps track of the progress of the goods in question.

This invention has increasing relevance as the value of the items it can record rises, as the volume of material in transport rises, and as the "just-in-time" approach to the movement and delivery of goods becomes more widespread.

The modularity of this invention resides in the manner in which a functional assembly can be put together using components of different capacities yet of similar function. Thus it may be vehicle-mounted (Fig 1), mounted on a recording mast within a goods yard, (Fig 15), or carried by an operator (Fig 14). Associated with the modular identification application of this invention is the use of a preferred identification label which generally comprises an encoded or text-character label having a distinctive outer frame, which helps to make the label recognisable against a background of any realistic degree of image complexity.

The preferred image locating means of this invention comprises solid-state, charged-coupled devices (CCD) as image capture devices, preferably with automatic control of orientation, focus, light levels, and viewing quality. The label identification application uses image analysis software that has been developed to search out and decode any labels that may occur within an image. A preferred CCD array has 4096 x 4096 pixels although the apparatus can function with a lesser or a greater resolution, and 2048 x 2048 pixel cameras have been used. Trials have been conducted with scanning linear arrays (eg 3084 x 1) but XY arrays (eg 4096 x 4096 or greater) are preferred. 6144 x 6144 pixel arrays have recently been announced.

The perceived disadvantage of such large arrays is the huge volume of digital data that they generate. Our invention provides means to manage this data by substantially reducing the number of bytes required to adequately represent any one image. "Adequate" relates to the application. Where (as in label identification) edge recognition software subsequently determines label content, adequate images result from compression ratios of as much as from 20:1, 30:1 and to more than 40:1, using the JPEG procedure described later. On the other hand images of peoples' faces may deteriorate noticeably if compression of more than about 4:1 is used. The compression ratio can be preset to a desired value. Thus a 6144 x 6144 array, digitised to 8 bits per pixels then compressed by 40:1 requires just under one Mb of storage instead of the 37 Mb as first extracted from the CCD array, and a 2048 x 2048 array image can be compressed to 100K from the original 4 Mb. (These compressed sizes are approximate; they depend somewhat on picture content and on the control information stored with the image so that it can be used in decompression.

Fig 16 illustrates a compact camera having several data reduction circuits included within its back. While this camera has been developed for use with the scan vehicle, (see below) it may be used in other types of mounts. In one preferred embodiment a scan system will be mounted on a small vehicle 101, 301, 401 (as shown in Figs 1, 3, and 4) and is for use in identifying objects during handling, transport, or in storage, such as the ends of rods, pipes, bottles or logs 102 which have no normal single positional orientation about their axis of rotation.

Another embodiment 1400 of the modular scan system is shown in fig 14. It will preferably be "person-mounted" and battery-powered in the same manner as a news video cameraman - with a hand-held camera 1402 and viewer 1403. A cable 1406 may connect via a plug 1407 into a backpack 1404 containing preferably miniaturised electronic equipment 1405 to transmit label contents to a remote station. Alternatively the cable 1406 and the plug 1407 may connect the camera to a docking station for downloading the stored information onto a local-area network; preferably fibre-optic based or alternatively using copper wires or radio. This version might be used to take stock of stacks of timber in storage in a yard. Preferably the camera employs a high-resolution CCD array as described elsewhere in this specification.

A further embodiment 1500 (see Fig 15) can be mounted on a mast 1505, or wooden pole or on a building or some other fixed structure. A camera 1502 is connected by cable to a control box 1503 containing preferably miniaturised electronic equipment to compress, store, and transmit label contents from an aerial 1504 to a remote station. Both the camera equipment and the data compression apparatus may be incorporated in the same box. Alternatively, fibre-optic cable may be used to transmit information.

Preferably mains power is provided to this installation although a solar-powered version might be feasible. Lighting means - either flash lamps or floodlights 1506 - and rangefinder means 1507 (to aid in focusing) may be provided. This embodiment may be used to automatically record logs 1501 loaded on a transporter, forced to stop with the labelled area in position at a predetermined spot (see below - "Example application for automatic single-scan system"). Preferably the camera employs a high-resolution CCD array as described elsewhere in this specification.

EXAMPLE APPLICATION USING SCAN VEHICLE

An application (Fig 1) for such a mobile scan vehicle is in the timber export industry; in the dockside capture of pictures of labeled logs 102 for later automatic recognition of the logs from the labels in the pictures. The advantages of this invention are: simultaneous capture - from a safe distance - of the data pertaining to an entire cradle of logs, and high reading accuracy, substantially independent of the ambient light and other conditions. The resulting information defines the presence of an identified log at an identified site at an identified time. (Global positioning satellite receivers or the like may be used in conjunction with the recording apparatus in order to indicate position and time).

As each log arrives at a marshalling yard (perhaps within or adjacent to the forest) a nine character identification is assigned to the log. A computer generates two copies of a (for example) 18 cm diameter or diagonal plastic label incorporating the assigned identification, preferably in a form providing duplication and redundancy in case of partial defacement of the label, and preferably in a form providing a distinctive border for computer recognition of the label. The plastic sheets are then stapled or otherwise affixed to the opposite ends of the log.

At a dockside area, the usual procedure is for a grab to pick a load of logs and carry them across to a cradle where a crane is then used to load the cradle load onto a ship for shipment. A bunting machine bumps the ends of the logs to align them for lifting with the crane.

Once the logs are aligned, the scan vehicle is driven to the scanning position facing one end of the cradle of logs and takes preferably three overlapping pictures of the log ends, including the labels identifying the cradle. The three pictures are obtained, in the preferred embodiment, by moving the camera to each of three assigned positions along the guide rail (106, 302, 402, 502, 602, 702). Each position is predetermined by placement of a fixed "target" (519, 313, 314) holding a pattern of metal plates in alignment with the path of an array of three proximity detectors 518 on the moving platform (536, 520, 620, 720) supporting the camera, and control means (within the box 311/411) follows a sequence in order to capture sets of images.

The log-holding cradles will preferably have permanent labels attached which uniquely identify the cradle and the side of the cradle. Optionally the scan vehicle may carry Global Positioning Satellite receivers to automatically record position and time along with the pictorial data. Pictures (which may initially be up to 16 Mb in size) are preferably compressed as part of the capture process (see later). Image analysis preferably occurs on board the vehicle or other image capture device (as indicated in Fig 2, for the sake of immediately indicating the occurrence of unreadable captured images, to avoid unnecessary volumes of data storage or data transmission) but may be arranged to occur at a base station or a base computer facility after transmission by (for example) radio.

The vehicle then moves around to the other side and takes a similar picture of the log ends on the other side of the cradle, in order to provide further duplication of data, in case particular log ends are visible at only one end, or in case only one end of one or more logs received or retained an at least partly intact label.

The cradle of logs is then strapped and lifted onto the ship. The scan vehicle may thereupon move about the dockside area to record from another cradle which has been loaded and bunted in the meantime.

Ultimately a record is created which includes information that certain logs were situated at a certain site at a certain time. This is communicated to a base station for storage, later viewing, and analysis to identify the logs in the cradle and correlate that information with (for example) ownership, weight/volume, or type.

Multiple vehicles of the type shown in Fig 1 can be used together on the wharf and each vehicle can handle scanning for multiple cradles depending on the timing of the operations.

EXAMPLE APPLICATION FOR AUTOMATIC SINGLE-SCAN SYSTEM

Another application for such a scan system is the capture of pictures of labeled logs at or soon after their initial collation in a marshalling yard, for later automatic recognition of the logs from the labels in the pictures.

As each log is entered to a marshaling yard a nine character identification is assigned to the log. A computer generates two copies of a 18 cm diameter plastic label incorporating the assigned identification. The plastic sheets are then stapled to the opposite ends of the log. Whenever logs loaded on trucks enter or leave the marshaling yard or other controlled areas, they are routed past a recording area. Fig 15 illustrates the recording arrangement that may be used in such an area. Mechanical devices about the recoiding area will preferably positively force the truck into proper alignment at a fixed position with the fixed-mount scan system. The scan system then causes the light levels to be at an appropriate level by energising floodlights (like those as 303, 304) and automatically takes a picture of the visible ends of the logs.

The image is then compressed and transmitted by radio or by fibre-optic or high-speed data cable along with the truck number to a base station for storage, optional later viewing of the image, and analysis to identify the logs on the truck.

In the event of stocktaking it may be more convenient to use apparatus as indicated in Fig 14 to move about a timber yard and collect images of merchandise (logs) bearing labels.

EXAMPLE APPLICATION FOR PORTABLE SCAN SYSTEM

An application for the portable scan version of this system is the capture of labels when the use of vehicles is inconvenient or not feasible.

In one example of such an application, labelled warehouse goods are stacked in bins along a wall with labels visible from a distance, or a stack of logs may lie within a yard.

The operator takes the portable camera 1402 (as per Fig 14) and takes a picture of at least some of the merchandise, adjusting zoom and focus and camera light controls in accordance with readings given in the viewfinder 1403. The picture is automatically compressed and stored in the portable computer 1405. A sequence of such pictures is taken which encompass all visible labels of the total of the merchandise. The camera system is then taken back to a docking station in an office or tally shed and the camera pictures transferred to an analysis computer for label recognition and transfer of label numbers to an inventory tracking system. (Alternatively a radio link may be used to send compressed pictures during the procedure, so enabling the operator to be quickly alerted of any inadequacy in the images. MECHANICAL COMPONENTS

The recording equipment in the preferred embodiment of Fig 1 is more clearly displayed in Figs 3 to 13. Fig 3 shows a front view of a vehicle 301 equipped for log recording. The structure bolted to the front of the vehicle may be regarded as (a) support means for a camera guide or traversing rail 302, (b) the camera assembly 310 and (c) support means for ancillary devices used to enhance the captured image (such as floodlights (303, 304) preferably ultrasonic range finders for automatic focus purposes (307, 308) and a TV camera 309).

The scan or image-capture camera (105, 310, 410, 515, 902) and guide rail (106, 302, 402, 502, 602, 702 will be mounted (as by bolts 434) on the front of the vehicle 401 along with dual rangefinders (107-108, 305-306) and floodlights (109-110, 303-304). Protective bars (not shown) will provide crash protection for the scan equipment. Power 436 and signal lines 435 (Fig 4) will attach the camera and electronics to the vehicle power (430) and computer equipment (431, 432, 433) inside the vehicle. A TV camera (111, 309) mounted in front and below the image-capture camera will be connected to a monitor (112, 321, 421) inside the vehicle positioned next to the driver alongside the operator console (112, 320, 420, 1010) used to control camera capture actions and control the onboard computers (431, 432, 433). A motor-driven alternator and inverter (114, 430) may be attached to the vehicle to provide additional power for the camera system. An antenna (113, 324, 424) will be mounted on or under the vehicle roof to relay data between the mobile scan vehicle and a base station.

Inside the vehicle there is a TV monitor 320, a control console 321, a computer 323, 432, 433, such as a 386 PC clone, and interfacing boards which are adapted to function with the camera control circuitry in a box at 311. (See also the block diagram of Fig 2.)

In use the camera 310 is caused to take preferably three pictures of the labelled goods (e. g. logs) from both ends and the centre of the traversing rail 302. The rail is preferably a square section rail mounted on a horizontal axis, with its walls at 45 degrees to the vertical.

The camera is supported by running wheels (503, 504, 603, 703, 704) within a frame 536 and provided with compression springs 530 to bear against the rail. Preferably brushes or the like are provided to keep the running surfaces clean during use in dusty or muddy conditions. A second frame 520 can swivel (about bearings 531 under control of a swivel motor 532) so as to cause the camera to face slightly towards the centre line of the vehicle 101 when at either end of the guide rail. Preferably the camera 310 is also provided with means for tilting on its optical axis.

A preferred mechanism to control this traversing movement comprises (a) an array of proximity sensors 518, (b) three inset plates 519, placed at each end (313, 314) and at the midpoint of the rail (obscured in Fig 3). Each inset plate has a distinct arrangement of metal strips so that the proximity sensors can determine camera position. Two limit switches (315, 316, 533, 733) are provided. A motor 535 drives, a long toothed belt (317, 534, 634, 734) via a reduction gear, attached to the camera platform, in order to cause traversing motion. In addition, motion of the belt is detected by means of an attached toothed wheel and a proximity sensor capable of detecting passage of a tooth. Preferably the toothed wheel is remote from the driving mechanism so as to better sense actual belt motion.

In a recent embodiment the motor section is controlled from within one enclosure; this being made responsible for the camera's position along the rail, range flnder flaps, and various motors and pneumatic actuators within the camera assembly. Conveniently this controller also manages the digitisation of the rangefinder data. Preferably it also includes a digital output to drive various illuminating floodlights.

POWER SUPPLIES AND BUSES

Taking the vehicle-mounted version as an example, there is a 12 volt DC supply bus for actuators; on which switching transients and the like are tolerated. Control connections to this bus are made via opto-isolators. There is a positive and a negative DC supply for the CCD circuits (to be split into a number of closely controlled DC supplies fed to the clocking pulse generators). There is a 9V DC supply bus to each processor from which a regulated +5 volt supply is generated locally. Pole or backpack versions will be scaled down appropriately.

A fibre-optic link is used to convey compressed pixel data from the camera unit to the PC computer, and a CAN bus (or the like) link (a two-wire omnidirectional serial bus using open drain wired-or drivers similar to but more powerful in terms of speed, addressing, and instances than the I²C bus, interfaced through microprocessors of the 80XX51 family) is used for all other data. The CAN bus may be conventionally driven from a board within the PC, with CAN addresses mapped so that various remote boards appear to the PC to be at separately addressable I/O ports.

IDENTIFICATION

Each module is preferably given a unique computer-readable serial number so that the controlling computer can test whether any module of a given system has been changed, indicating that calibration may be in jeopardy. Preferably this serial number is embodied in a one-time-programmable memory chip as a 24-bit number. As an example, the motor enclosure contains a 256 byte EEprom (e.g. type Philips PCF8574) within a totally sealed box. This chip includes a serial number and calibration information for the rail. In the case of the camera identifier module, it would include the standard settings for that camera head.

THE CAMERA

Fig 5 also shows the interior of the camera housing 510. We built this camera around a "Pentax" (TM) large-format body although we now prefer a "Mamiya" (TM) body as it may be split between front and back at the film plane. Thus our invention (high resolution camera with inbuilt data compression) could be made and sold as a camera back to be fitted onto an existing camera.

The modified "Pentax" camera 515 is shrouded by housing 510 from water, foreign matter, and from stray light. There is a motor-driven front flap which swings up when an image is about to be captured. Neutral-density filters 512, 513 may similarly be swung into or out of position in order to reduce the intensity of strong light. Circuit boards 508, 516, and 517 correspond to "CCD control board", "S&H, A->D board", "PC Header + JPEG Compression" in Fig 2. Preferably a fibre-optic cable is used to convey image data as photon streams to the "PC peripheral" board incorporated in or adjacent to the image-processing computer. Optical coupling has been found to minimise induced noise, yet has a high signal-carrying capacity. A hydraulic or pneumatic version of this camera housing is described later. Fig 16 shows an alternative sealed camera module, intended to be opened up for service in only a controlled laboratory environment. Various electronic modules are now included in it, to cause CCD chip readout to occur and then to accept the analogue "video" signals from the CCD chip, digitise, compensate, and compress the signals and transmit them - preferably as serial data in a fibre-optic cable - ready for passage to a base station or other image analysis device.

In the section of the camera, Fig 16, 1600 is the entire assembly, 1601 is the camera lens, 1603 is the image plane formed by the lens of an object at the predetermined distance, 1604 is the environmentally secure case, 1607 is a CCD array, 1608 is a circuit board carrying sample and hold chips and also serving as a base for the four

CCD array supports 1609 pressing the CCD against the film plane of the camera by spring-loaded screws 1612 (or as per Figs 8 and 9), 1606 is a light-tight and electrically shielded barrier separating various circuit boards 1611 (and the like - such as the video compression board(s)) from the sensitive areas around the CCD chip, 1605 is a series of connectors used for a CAN bus used for control (including adjustment, calibration, and test purposes) and also power connections, and 1610 is a fibre-optics connector used to transmit the compressed picture data from this assembly at high speed. In general high-speed electronic circuitry radiates much more electromagnetic interference than low-speed ones and CCD chip input amplifiers which work at millivolt levels yet at a wide bandwidth are likely to pick up this interference.

One of the main advantages of bringing the JPEG etc boards into the same module as the camera itself is that local earth loops and the like can be "designed out" and removed from a camera installation. Thus this assembly approximates a "black box" unit; its input is an optical image and its output is a compressed digitised video signal - conveniently transmitted to an analysing computer as an optically coupled signal.

The preferred boards provide for remote adjustment of all parameters such as CCD voltage levels. They preferably include a 32K byte RAM, for which one use is in accepting various JPEG tables from the host computer at startup time, then passing them to the JPEG board (with the help of a local microcontroller) in response to a single instruction. We have used jgraphic camera - a large-format singie-lens reflex camera body and lens, with automatic exposure means driving the lens aperture, as a convenient optical section of the entire camera. The film back is here replaced with a spring-loaded assembly extending from the rear of the camera body that holds the active area of the CCD chip in the focal plane of the lens.

The camera includes at least one solid-state CCD light-sensitive array. The camera may have a shutter and this may be used either or both to protect the CCD array from inadvertent strong lights (such as the sun) and to define the actual exposure, although for the latter purpose the actual exposure can also be determined electrically by means of bias voltages applied to the chip.

Typical CCD chips require a number of electrical inputs; some are steady and are used to provide suitable biases across semiconductor junctions for light (i.e. photoelectron) storage, and others may from time to time oscillate between states so as to transfer the resulting charges out from each pixel and into an output connector. High-resolution chips commonly provide typically two or four output connectors connected to different areas of the image-sensitive area, so that several sectors of the image may be read out at the same time.

The (or each) output signal, which is an analogue or "video" signal representing the charges in each pixel read out in a sequence, is preferably buffered and is then fed to a device which converts it to a digital format. Preferred devices are known as flash A to D converters because they offer a suitably high conversion rate, which is preferably at a rate such that each pixel of the CCD is also digitally represented as one byte (or other word). Preferably the parallel streams of analogue information leaving the CCD array are first "dissected" to extract the video information from DC offsets and pulses, then digitised as parallel streams by converters clocked to work in sequence, the digitised pixel values are passed to individual inputs of a JPEG board, where each is handled separately by a process including a DCT algorithm, the output of which is quantised and Hoffman-compressed, and combined into one stream at the output of the preferred JPEG hardware. This preferred hardware is provided with several inputs - originally intended for use as colour inputs (RGB or CMYK) and we find it convenient to tie the separately digitised camera outputs because compressing several streams of information in parallel, as it flows out of the CCD chip, saves a great deal of time. Examples of effects that may require compensation measures to be applied include

(a) dark current,

(b) static background current (reflecting individual pixel sensitivity, which may also be colour-dependent) and because thermal cooling is generally not practiced in our applications it includes

(c) a thermal current component, which is partially dependent on the time taken to read any particular pixel (pixels accumulate dark current even after exposure), leading to a shading effect.

Compensatory values may be stored in a non-volatile memory beside the camera and read back at the time of compression, or they may be stored in the host computer and subtracted prior to computation, though this approach may lead to extra work for the compression utility and it may reduce the working range of the camera (256 levels, without gamma correction).

Thermal cooling or temperature control, for example using a Peltier thermo-electric device, may be used as an option, especially in hot environments. Although there is little need in this application to cool the CCD to the astronomers preferred values of -60 degrees C or below, to nullify dark currents, it may be useful to maintain it at a known and substantially constant temperature.

One preferred method for compensating for the baseline drift is by dynamic baseline correction. This method holds compensatory values in a lookup table (2 x 8K byte RAM or ROM or EEProm) which is addressed simultaneously with the column addresses for the CCD readout. Memory data is passed through a digital-to-analogue converter and summed with the video data in an operational amplifier so that after compensation the video signal has a substantially constant baseline.

Preferably, good construction practices are followed in order to minimise artifacts in the video signal. For example both the CCD video amplifier/buffer chips and the clock chips (that generate sequences of waveforms having closely controlled high and low levels) should be close to the actual CCD. Sample construction practices are the use of surface-mounted devices on printed-circuit boards which include an internal ground plane, good power supply decoupling, and folding the boards so as to pack the integrated circuits into a small space. The raw stream of digitised data is preferably compressed so that the total number of pixels required to adequately represent the image is significantly less that the original number of pixels. This compression hardware, used with a 4096 x 4096 pixel array reduces the initial 16 megabytes stream of data to typically about 600 Kbytes depending on image content. It is then relatively simple to handle the reliable transfer of 600K bytes over a distance in about 6 seconds; a preferred time between shots.

We prefer a compression algorithm which is a standard procedure because decompression may be accomplished using off-the-shelf hardware or software, although if the data is to be accepted at our own image analysis station the compression algorithm should match whatever decompression algorithm we choose to use, if any. Software that carries out image analysis within an array of compressed data may exploit the advantage of a smaller data space.

One general-purpose algorithm used for this purpose is the known "JPEG" or "Joint Photographic Expert Group" algorithm. This algorithm is defined for example in "JPEG Still Picture Compression Algorithm", October 15th 1991, available from C-Cube Microsystems marketing Department, 1778 McCarthy Boulevard Milpitas CA 95035, USA.

There are some dedicated integrated circuits available which perform this type of compression, which represents an acceptable balance of compression against speed. A newer version of this is the "Wavelets" algorithm, and many others are under development around the world as a result of interest in broadcasting HDTV (high definition Television). Other forms of compression, such as those based on fractals, may be useful. At present they are slow, although the degree of data reduction may be considerable.

The provision of memory allows one image to be stored temporarily while an earlier image undergoes compression or transmission, thus allowing the camera to handle several images in quick succession. Preferably the memory unit can store several compressed images and acts as a kind of pre-transmission buffer which can hold a number of images. A suitable memory or buffer is made up of dynamic random-access memory (D-RAM) in one or four megabyte modules, as are used in personal computers; and 4 Mb of storage can store at least five or six typical compressed images. Use of memory modules avoids disadvantages of storage on magnetic media such as physical size, slow data transfer speed, inherently lower reliability, lower capacity, and higher power consumption. Hash memory is a useful alternative to D-RAM.

A further application for a memory unit may be at the communications end of the processing chain, because the preferred use of a handshake protocol requires that the sending of data may be delayed momentarily or for some time, until conditions in the transmission medium or at the receiving station are suitable for transmission.

The camera may also include a suitable communications adapter, capable of bidirectional communications in the chosen medium. One preferred medium is fibre-optic cable because of the high speeds (i,e, wide bandwidth) yet excellent electrical isolation that may be achieved with this medium. In fact, one of our proposed modes of use involves a portable (backpack) type of camera with storage for about six captured and compressed images, and a docking station at which the stored images may be dumped into a tap made into a fibre-optic "bus" running the length of a wharf or dock. This docking station may also include means for recharging the batteries of the portable camera.

Alternative data transmission means include radio (preferably at a high frequency as the link may be required to have a bandwidth of the order of 1 MHz), and ^:'Wavelan" (TM) is one example of a product adapted to transmit digital data over moderate distances, or infra-red beam communications, or conventional conducting cables. If direct connection is used, interference due to earth loops and the like (such as from a high-speed computer) should be minimised.

Preferably the entire assembly is contained within an environmentally resistant housing, and preferably this housing is compact. Examples of environmental hazards include mechanical damage through collision or abrasion, heat damage through solar or floodlight radiation, and electrical damage through accidental contacts or electromagnetic radiation. Additional protection to shield the sensor itself from excessive light is also preferred.

In use, the speed of the process of communicating an originally 16 million pixel image from the CCD chip within the camera through the JPEG compression device and into its internal storage means as an about 600K version of the image is at the present time about 6 seconds although further circuit improvements may render this time much shorter. The JPEG processor operates in real time; that is, there is no perceptible time delay imposed on the transfer of information from the CCD chip into memory as a result of compression.

ADVANTAGES

Regarding this camera back as a kind of frame grabber, the invention has a number of advantages over the usual frame grabber which is generally an accessory board plugged into a bus within a computer that creates a mapped representation of an image in digital form in a memory plane, the advantages including:

(a) because digitisation and image storage occurs adjacent to the CCD chip within the camera there is considerably less possibility of image degradation than would be the case if a video lead took an analogue signal to a distant computer,

(b) because data compression is applied to the first version of the digital image, prior to storage as a second compressed version, the capacity requirements for storage and onwards transmission path bandwidth are reduced, (c) the time taken to handle the data is reduced as a result of real-time compression within the CCD readout path, and

(d) subsequent data analysis (image analysis) may be faster and require fewer computer resources if applied to a more compact data structure (the compressed version of the image).

Producing this frame grabber in the form of a camera back so that any useful modular camera may be fitted with an electronic back has the advantages of providing a "black box" solution which can accept one or more focused images, then store and deliver a conveniently compressed digital version of the or each image, which version can later be uncompressed if required during a delayed appreciation or analysis procedure by reversing the compression procedure.

Regarding the invention as a combination of a machine- vision camera and compression unit, it has further advantages including that it brings high-resolution digital image capture into almost the real-time environment; that is, it provides for the repetitive capture and handling of high-resolution images at rates compatible with a number of industrial and transport procedures. For example it converts a 4096 x 4096 pixel camera image into transmittable data in about 6 seconds, or at least it is ready to take the next picture in about 6 seconds.

This speed allows test exposures (for example to determine light levels, focus, or subject matter ) to be taken and evaluated before conditions change.

It also allows groups of labels to be re-photographed before they are lost to view if an initial image was not readable, or became corrupted before acceptance, for any reason.

It also has the advantage that it is adapted for use in environmentally harsh and unpredictable applications where the lighting and/or mechanical conditions are incompatible with earlier cameras.

Figs 6 and 7 illustrate the attachment of the camera platform 730 to the guide rail (602, 702). The use of a toothed belt 634, 734 provides a positive drive to the camera platform.

Figs 8 and 9 illustrate means to adapt the CCD chip 909 to a conventional camera 901. The CCD chip is mounted on a metal plate 904, which is pressed into contact with the film guides 903 by two or more spring-loaded legs 800. Each leg includes a compression spring 802 which presses a telescoping peg 801 from within the hollow cylindrical shaft 803. Peg 804 serves to limit the maximum movement of the parts 803 and 801. In use, pegs 800 are installed in the camera as at 905. Wires 907 from the CCD chip socket are taken to connectors 906. Optional CCD chip cooling means may be installed; for example a miniature fan or a Peltier cooling module may be placed behind the chip.

Fig 2 illustrates in block form the path of the signals representing an image. The CCD 4K² (4096 x 4096) block represents the camera chip, mounted within a camera body as in Fig 9. These devices require a number of steady voltages or sequenced arrays of pulses in order to clear the array, to retrieve image signals, or to put the array into a light-sensitive mode. The CCD control board generates the required voltages. In order to convert the analogue levels into computer-compatible form two analogue-to-digital converters are provided, as there are two output lines from the preferred type of chip. Image compression, preferably according to the JPEG format but optionally in another format is preferred in order to reduce redundancy and volume in the potentially large image. The signal is passed, preferably along a fibre-optics link 435 (Fig 4) to a mini-computer, preferably of the PC type, via a peripheral or board adapted to receive fibre-optics data. After (optionally) further data reduction which may extend as far as recovering the information encoded within imaged labels, the image data is optionally stored and optionally transmitted by radio or possibly a cellular telephone modem to a remote base station.

Fig 16 illustrates a physical arrangement for placing such boards in close proximity to a CCD camera chip.

The operator's control display is illustrated at 1010 in Fig 10. 1011 is a LCD display indicating the rangefinder readings, 1012 is a bar LED display to indicate computer progress, 1013 is a steering guide indicating off-centre states, and 1015, 1016, and 1017 are pushbuttons to start lining up, to capture an image, or to abort an operation. An optional key switch 1014 is also provided.

In one preferred embodiment the operator's console makes use of one or two integrated circuits linked by the I²C bus to a microprocessor. One of these chips is capable of either generating an informative video overlay, electrically mixed on the TV monitor which forms the rangefinder, and the other (or both in a different embodiment) can activate lamps under button switches and detect any switch depression, in order to lead the operator to a suitable action.

Details of an optional fluid powered camera case are shown in Figs 11, 12, and 13. (Although electric actuators are one option it is thought that hydraulic actuators may better resist the working environment). 1100 shows a top view of a camera case, with an inspection aperture 1104, an outer flap 1102, an inner flap 1106, a pneumatic/hydraulic cylinder 1103 to drive the outer flap, another pneumatic/hydraulic cylinder 1105 to drive the inner flap, fluid reservoirs 1101, and linking pipes. 1107 is a sequencing control valve.

In Fig 12, 1200 shows the case from one side. 1202 is the pneumatic/hydraulic cylinder to drive the inner flap 1205, 1201 indicates the position of the inspection flap, 1211 is a cylinder to move the neutral density filter 1206, 1210 is an upper limit switch for the movement of 1211, and 1207 is a lower limit switch. 1212 indicates two fluid reservoirs. The arc indicates the travel of the protective outer flap 1204.

In Fig 13, 1300 is the front view of the hydraulic camera assembly. 1302 is the position of the internal cylinder for operating the neutral density filter. 1301 is the end of the outer flap cylinder (1202). 1303 is the end of the inner flap cylinder. 1304 is a fluid reservoir.

CONTROLMETHODSANDSOFTWARE

The following example shows a typical interactive instruction sequence for control instructions for the log identification when a dual-scan vehicle is available.

Control Instruction Sequence:

1. Wait for grab to load cradle and bunting machine to square logs.

2. Driver moves vehicle to one side of cradle and presses the start button on the operator display (1015 - Fig 10).

3. Scan unit continually measures distance of two rangefinders from cradle and reports results on the operator display. A TV display is also operational.

4. The driver maneuvers the vehicle using rangefinder measurements and the TV image to align the assembly on the front of the vehicle perpendicular to the cradle of log ends and at substantially the required distance.

5. Computer turns on the green light (1013) indicating the vehicle is correctly positioned.

6. Operator brakes, and presses the Capture button 1016.

7. The computer opens the cover flap(s) over the scan module, measures light, adjusts the camera lens aperture and if necessary turns off or on the lights, or inserts or removes a neutral density filter, to ensure that the light level as seen at the CCD is appropriate. These exposure adjustment means may be automatic. The camera is also moved to one end of the guide rail.

8. The computer takes the first picture.

9. The computer then closes the flaps and begins moving the camera to the central position on the guide rail while the image is being compressed and sent to the computer in the vehicle.

10. When the compressed image is complete, the computer begins to transmit the image or extracted data over radio back to a base station, optionally also storing it in case of poor reception.

11. The camera flaps are opened and the second picture is taken from the central position in the same way as the first picture.

12. The third picture is taken from the third guide rail position the same way as previously.

13. The driver (who may have aborted the sequence by way of the Abort button 1017) is notified of capture completion by the operator console lights going off.

14. The last picture is compressed and information is transmitted to the base station while the camera is returned to the central guide rail position and also while the driver drives on to the next cradle position.

15. Steps 1 to 14 are repeated for both sides of each cradle and for each new cradle full of logs.

For single-camera automatic scanning:

1. Wait for truck to pass over sensors and reach the required scan position.

2. Automatically raise the blocks to lock the truck in position and signal the driver. 3. Automatically adjust the light level as required to ensure that required level is set for the scan.

4. Automatically read any separate truck identification tag.

5. Automatically take a picture of the ends of the logs and the truck identification tag if it is present

6. Then automatically compress and transmit the pictures or extracted data back to the base station.

For portable camera scanning:

1. Operator lines up target in view-finder and adjusts zoom, focus, and f-stop, and presses computer-shutter button.

2. Computer fires camera shutter, downloads, compresses and files the image or an extract thereof into a disk file.

3. Computer waits for next depression of computer shutter button.

A variation of the above involves the use of a backpack camera, which is carried about the wharf by a person who photographs first one end of the cradled logs and then the other, preferably with three photographs of each end, and then returns to a docking station in a tally hut to download the compressed images to an analysis station while the logs from the cradle are being slung and then hoisted into the ship. We prefer a fibre-optic cable run along a slit cut into the wharf and brought out to a docking point inside each tally hut.

EXAMPLE APPLICATION FOR REMOTE SURVEILLANCE

The design of the self-contained camera in the form described above having as its output an image in compressed format for relaying by optical fibre to a nearby radio transmitter may be altered for certain applications by including the radio transmitter within the self-contained camera. A block diagram of this embodiment is shown in Fig 17. Surveillance of remote areas with a high-resolution camera (e.g. 16 million pixels resolution) may be useful in a number of applications such as borders surveillance, wild-life studies, and the like. The camera may survey remote airfields to record the identification markings on aircraft, where the high sensitivity of this type of camera allows it to work under conditions of low light intensity.

In such cases the compression of the video signal by a built-in compression algorithm in hardware or software is an advantage, to help overcome the volume of traffic to be transmitted. The wide bandwidth of the compressed signal (a consequence of the large amount of data within it) suggests use of UHF radio links in order to accommodate the bandwidth and in order to carry UHF links over a distance a satellite link may be preferred if there is no nearby microwave relaying means. On the other hand, use of extreme forms of compression (such as fractals) may compress a high-resolution picture to such an extent that it can be transmitted on a short-wave radio link or an ordinary telephone line. Fractal compression as it is presently applied tends to be rather a lengthy process for the materials-handling applications that form the bulk of this specification, but in other applications a processing time of some minutes may be acceptable if a transmission duration of perhaps an hour is to be avoided.

In Fig 17, 1700 indicates the entire module which is preferably provided in a weathertight and damage-resistant enclosure. 1701 is a lens, which for this purpose may be motorised and indeed the entire module may be provided with a pan and tilt mechanism, driven locally or remotely - though the high resolution of the preferred CCD devices 1702 may render movement, and the use of lenses of longer focal lengths relatively unnecessary. Optionally the module may be provided with means to check a newly acquired image against an older image, and then automatically (a) transmit the new image (or just the altered part or parts of it) to a remote station, and/or (b) close in on the altered parts and send a more detailed image of the altered parts. The data compression module 1703 is preferably a board implementing the JPEG compression procedure, though other compression modules (if any) are alternatives. Recent developments in video-on-demand (VOD) via telephone lines may provide further alternative effective data compression systems. The transmitter module 1704 is conveniently a UHF transmitter though other radio frequencies and even microwaves may be used. 1705 is an aerial here shown as a directional Yagi type with a fixed mounting though if a low-orbiting satellite is used as a relay station a more omnidirectional aerial may be preferred. As one means for providing electric power over a long period, 1706 represents a storage battery (preferably including power management) and 1707 represents for example a solar cell array to maintain a charge in the battery.

VARIATIONS

The design of the camera may be varied in a number of ways depending on the requirements of the particular application.

The limiting resolution of the system may be varied to suit the application; for example by using CCD arrays of between 512 x 512 and 4096 x 4096 pixels (the range of sizes commercially available and under test at the time of preparing this specification) or a greater number.

Range finding features may be removed if not required by the application environment.

Internal light control may be removed if not required by the application environment.

Finally, it should be noted that a number of other variations upon the principles of this invention or other embodiments of it still lie within the scope of this patent.

VARIATIONS IN SCAN SYSTEM DESIGN

The scan system design may be varied in a number of ways depending on the requirements of the particular application.

Different makes or models of camera may be used if convenient

Standard video cameras and signals may be used instead of the preferred high-resolution CCD devices and resultant readout signals.

Alternatively the CCD resolution may be reduced by defocusing, image averaging, combining pixels, or other means. Light control may be removed if not required by the application environment

Instead of floodlights, flashlights may be used. Typical exposure durations are of the order of 2-3 mS.

The response of the camera may be restricted to one or more specified spectral bandwidths, to take advantage of object reflectivities, lens performance, or the like.

Range finding may be eliminated if not required by the application environment. The sensitivity of the CCD devices allows small lens apertures to be used, although diffraction limiting of allowable apertures should be considered. For this reason a neutral-density filter is a preferred method for reducing the incoming light as it permits the lens aperture to remain wider.

Identification of containers for groups of items may use the same system used for the items being scanned and be scanned at the same time.

Alternatively identification of containers may be done by another means or not at all depending on the application requirements.

Controlled cover flaps for the scan units may be eliminated if the unit is not to be used in adverse environmental conditions.

More cameras may be used to simultaneously scan larger areas.

Rotational alignment of cameras may be eliminated if not required by the application.

Identification of location of the container for groups of items may be predetermined and stored with the container identification so that only the container identification is required to locate the container.

Scan units may be scaled up or down as required to accommodate the containers and items being recorded.

The guide rail may be eliminated if not required. The bumper support frame may be shaped differently if required by the application.

The modular unit may be attached to different kinds of platforms such as poles, boats, planes, cars, and robot arms.

Applications of this device are by no means limited to felled timber or even to labelled merchandise in various containers. It could be used to monitor rolling stock on a railway. It could be used for rolling inventory in a warehouse or for monitoring the usage of supermarket trolleys in a supermarket Advances in the automated recognition of the human face, for example, may allow an automated roll call to be made at a school.

Finally, it should be noted that a number of other variations upon the principles of this invention or other embodiments of it still lie within the scope of this invention as set forth in the following claims.

Claims

1. A frame grabber capable of receiving a first digital signal representing an image, comprising means to compress the first digital signal by at least 4:1 in real time to produce a compressed digital image having a reduced number of bytes compared to the total number of bytes of the first digital signal, and buffer means to store at least one compressed digital image.

2. A frame grabber as claimed in claim 1 wherein said means to compress the digital signal is capable of compressing the digital signal in the range of 20:1 to 30:1.

3. A frame grabber as claimed in claim 1 wherein said buffer means is capable of storing at least two images.

4. A frame grabber as claimed in claim 1 wherein said means to compress the digital signal has at least two input channels capable of processing the information in parallel.

5. A frame grabber as claimed in claim 1 wherein said means to compress the digital signal comprises a JPEG processor.

6. Image capture means for a camera comprising at least one solid-state CCD light-sensitive array capable of being positioned at an image plane of a camera, at least one more analogue output channel from said CCD array, one or more analogue to digital converter(s) for converting analogue signals from the or each analogue output channel to one or more digital signals, and a frame grabber as claimed in any one of claims 1 to 5 connected to the output(s) of the analogue to digital converter(s).

7. Image capture means as claimed in claim 6, wherein the solid-state CCD light-sensitive array is an X-Y array having at least 2048 pixels along each edge (i.e. a total of 2 ^ pixels)

8. Image capture means as claimed in claim 6, wherein the solid-state CCD light-sensitive array is an X-Y array having at least 4096 pixels along each edge (i.e. a total of 2 ^M pixels).

9. Image capture means as claimed in claim 8 and further comprising means to communicate the one or more compressed digital image(s) to an external image processing unit or other receiver of image data.

10. A camera for recording labelled items comprising image capture means as claimed in any one of claims 6 to 9, and optics including image focusing means capable of focusing an image on the CCD array, aperture control means, and shutter means.

11. A camera as claimed in claim 10, wherein the camera is mounted on a self-powered mobile platform so that the camera can be positioned relative to one or more labelled items to initiate a data capture process.

12. A camera as claimed in claim 11, wherein means are provided to laterally move the camera along a guide rail, and to tilt and pan the camera when at selected positions in order to repeat the process of capturing an image from more than one viewpoint.