WO1997014253A1

WO1997014253A1 - Apparatus and method for generating synthetic and absolute real time environments

Info

Publication number: WO1997014253A1
Application number: PCT/US1996/016386
Authority: WO
Inventors: Frederick W. Von Tagen
Original assignee: Von Tagen Frederick W
Priority date: 1995-10-10
Filing date: 1996-10-10
Publication date: 1997-04-17
Also published as: EP0860083A4; EP0860083A1

Abstract

An apparatus for generating synthetic and absolute real time environments within an enclosed real environment such as in a cyclorama (3, 4). The apparatus includes a plurality of transmitters (62) mounted in a fixed known, and spaced-apart locations outside of the real environment. Each of the transmitters are capable of transmitting an electromagnetic signal into the real environment. A camera (2), capable of moving in a plurality of dimensions within the environment, is located within the real environment for capturing a real image of the real environment. A receiver movable with said camera is capable of receiving the electromagnetic signals from the plurality of transmitters. Finally, a camera positioning processing device is linked to the receiver for calculating the location of the camera within the real environment in response to the electromagnetic signals received from the plurality of transmitters. A computer generation system can then utilize this location data to synchronize the real image captured by the camera with a synthetic image generated by the computer.

Description

APPARATUSANDMETHODFORGENERATINGSYNTHETIC ANDABSOLUTEREALTIMEENVIRONMENTS

BACKGROUNDOFTHEINVENTION This invention relates generally to cycloramas used in the fields of photography and video to provide a horizon-less backdrop for the images being produced and more particularly to a method and apparatus for seaming together synthetic and real images within a cyclorama environment. The use of computer-generated images to create synthetic environments has dramatically increased in recent years. The movie and video industry has achieved heretofore impossible depictions and presentations of real and imaginary characters in synthetic environments, adding tremendously to the entertainment value of their products. Advertisers now realize the significant impact on consumers which can be achieved by the use of synthetic environments as settings for the delivery of their commercial messages. Television studios routinely use computers in conjunction with their camera equipment to generate composite real-time images for broadcast. Computerized compositing camera equipment is used to create a synthetic environment as a backdrop for subjects in a studio. Computerized compositing camera equipment can also be used to display the real-time image of a distant person or event on a synthesized screen to achieve the appearance of an interactive "teleconference" which is taking place in the studio. Computerized compositing camera equipment is commonly used in the creation of the ubiquitous weather report and forecast wherein the real-time image of the weather reporter is displayed along with stationary or dynamic satellite images of the earth and its weather systems of interest, or with some other form of "weather map" or graphic. The camera is used to capture the image of the reporter (foreground image); and the maps and graphics (background images), which are stored as computer data files. The foreground image is generated and layered onto the background image by a process known in the art a "compositing". One such video compositing tool is the Ultimatte-7 produced by Ultimatte Corporation of Chatsworth, California.

In perhaps its highest form, the technology for generating synthetic environments can permit a person to become part of, and interact with, a synthetic environment. The ability to generate an interactive synthetic environment, or a "virtual reality", offers limitless opportunities for education and entertainment.

In many applications, the generation of a synthetic environment requires a horizonless chamber as either a medium onto which synthetic images can be projected, or as a featureless background to allow the image of a character or actor to be isolated and captured. Such a structure, commonly referred to as a "cyclorama", can be constructed as a permanent studio, or alternatively, can be a structure assembled within a larger space. I earlier described a freestanding cyclorama in U.S. Patent No. 4,893,477, which is expressly incorporated herein by reference.

Further development in this technology is limited in part by the ability to determine the precise location of a camera or projection device within a cyclorama. Precise location of the camera or projector within the cyclorama is key to high image quality, particularly at the interface or border between portions of composited images which are supplied from different sources. Where the camera or projector is stationary, its location can now be determined with adequate precision, and current technology permits a fixed camera to be rotated and perhaps moved translationally in one direction, however, producers are now demanding the freedom to mover the camera or projector along any or all of the three translational axes within the cyclorama. It is not possible to do so now, while at the same time monitor the precise location of the camera or projector without compromising the visual quality ofthe cyclorama. For example, U.S. Patent No. 5,255,211 discloses a method and apparatus for generating a synthetic environment which employs infra-red sources and sensors or video receivers mounted on the interior of a chamber wall to determine the location of a user in the chamber. The need to mount the detection devices on the interior surface of the cyclorama wall necessarily disrupts the horizonless, featureless appearance ofthe cyclorama, and is at best a less than ideal compromise.

A need therefore remains for a cyclorama which can be readily assembled in any location, and which provides for the precise determination of the position of a camera or projector within the cyclorama while preserving the visual integrity of the cyclorama inner surface.

SUMMARY OF THE INVENTION The present invention is embodied in a cyclorama comprising a horizonless, featureless inner surface, a camera or projector

(hereinafter referred to collectively as "camera" for brevity) mounted within the cyclorama which is capable of translational movement along one or more axes within the cyclorama, and means for determining the translational position ofthe camera within the cyclorama. The means for determining the translational position of the camera includes at least one, and preferably five transmitters mounted on the exterior of the cyclorama at predetermined locations. The transmitters are configured to transmit a beam of electromagnetic radiation, preferably RF radiation, through the cyclorama wall. The camera includes one receptor corresponding to each external transmitter which detects the electromagnetic radiation therefrom, and which generates a data output from which the three-dimensional angular displacement of the receptor (and therefore the camera) relative to the external transmitter. In alternative embodiments, a single receptor detects the transmissions from a plurality of transmitters. The camera or projector also includes a processor for receiving and compiling the angular displacement data from each receptor, and for using that data, along with the data describing the predetermined location of each transmitter, to precisely determine the location of the camera within the cyclorama. The precise location of the camera within the cyclorama is then transmitted to a location for use in generating the synthetic environment or composited image.

The cyclorama includes means for externally mounting the RF transmitters at any of a plurality of locations on the exterior of the cyclorama. In the preferred embodiment, the cyclorama comprises a plurality of preformed, abutting panels mounted on an external framework. The external framework includes numerous predetermined mounting points for transmitters, and provides for the mounting of a transmitter at positions other than those predetermined locations if required. To minimize environmentally-induced errors in determining the location ofthe camera position within the cyclorama, the entire cyclorama and external framework is constructed of non- ferrous and non-magnetic materials. Only in this way can distortion- inducing interaction between the transmitted electromagnetic radiation and the cyclorama be minimized.

The foregoing and other objects, features and advantages of the invention will become more readily apparent from the following detailed description of a preferred embodiment of the invention which proceeds with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a perspective view of a cyclorama with the cove sections shown outlined for purposes of illustration only;

FIG. 2 is a perspective view of a floor-to- wall or floor cove with fragments broken away for purposes of illustration;

FIG. 3 is a sectional view taken along line 3-3 of FIG. 2; FIG. 4 is a sectional view taken along line 4-4 of FIG. 2; FIG. 5 is a rear elevational view of the floor cove shown in FIG. 2; FIG. 6 is a sectional view taken along line 6-6 of FIG. 1 and showing abutting side flanges and overlying material;

FIG. 7 is a sectional view taken along line 7-7 of FIG. 1 and showing a ceiling cove;

FIG. 8 is a front elevational view ofthe ceiling cove shown in FIG. 7 but removed from supporting structure; FIG. 9 is a perspective view of a corner cove; and

FIG. 10 is a horizontal sectional view taken along line 10-10 of FIG. 9.

FIG. 11 is a perspective view of a cyclorama showing transmitters mounted on the exterior thereof. FIG. 12 is a top plan view of the cyclorama of FIG. 11.

DETAILED DESCRIPTION Referring now to the drawings, the reference numeral 1 indicates a subject being photographed or video taped or otherwise reduced to an image by use of a camera at 2. A floor is at F and a ceiling is at C. Cycloramas, as earlier noted, serve to provide a background or backdrop for the subject being photographed which appears in the photo or other image rendered as an uninterrupted expanse. A lower or floor cove of the present cyclorama is indicated generally at 3 and is comprised of a molded curved sheet 4, preferably of resin reinforced fiberglass, and having an upper edge 5 and a lower edge 6 parallel to one another. Side edges at 7 and 8 are curved through approximately a 90 degree arc. Side flanges are indicated at 9 and 10 each being integral with a side edge. Contiguous with the side edges are recessed curved side margin 11 and 12 as best shown in FIG. 7. Floor cove 3 includes an upper flange 13 integral with cove upper edge 5. With attention to FIG. 4, sheet 4 is of increased section at 14 along its lower margin which terminates in lower edge 6. The increased section or thickness of lower portion 14 serves to reinforce the sheet adjacent a floor surface at F and also enables the use of a fillet 15 to continue the cove arc for mergence with the floor surface. Edge 6 rests on floor surface F by reason of side flanges 9 and 10 being beveled at 9A and terminating in an offset manner from edge 6.

The side flanges, as well as upper flange 15, define spaced apart apertures to receive fasteners joining the floor cove side flanges of adjacent coves and the upper edge flange of the cove to a supporting structure such as wall structure at W.

A ceiling cove is indicated generally at 20 and includes a curved sheet 21 having side edges 22 and 23 provided with curved side flanges 24 and 25. In similarity to the floor cove, the ceiling cove is provided with recessed curved marginal side areas at 26 and 27 to receive material overlying the marginal areas and the abutting side edges of adjacent ceiling coves to conceal said edge. Upper and lower flanges 30 and 31 are integral with upper and lower edges 28 and 29 of the ceiling cove.

A corner cove structure generally at 32 utilizes upper and lower coves of substantially identical construction curved along two axes through 90 degrees. As the upper and lower corner coves are almost identical, the following description of one is believed sufficient. The lower corner cove 32 includes a molded sheet 33, preferably of fiberglass, ABS, or molded polymeric material, having curved side edges at 34 and 35 integral with which are side flanges 36 and 37. A horizontal edge 38 carries a horizontal flange 39. The bottom apex of a lower cove is in the shape of an edge formed at 41 on a radius of about four inches or so and is of a thickness at 40 greater than the remainder of the cove to receive an anchoring or fastening device. The flanges 36, 37 and 39 are apertured at intervals to receive fasteners.

An upright corner cove is indicated generally at 43 and may be a ceiling cove 20 rotated through 90 degrees so as to reorientate same to the vertical whereat its side flanges 24 and 25 will abut the horizontal flanges 39 of the upper and lower corner coves. Upon such reorientation, the flanges 30 and 31 ofthe ceiling cove will each abut wall structure W.

In a typical cyclorama installation, the wall components W would be wallboard or the like supported by wall studs 45. A horizontal stringer 46 provides a horizontal surface on its underside to which is attached the upper flange 13 of each floor cove. Ceiling coves 20 are attached to the wall components in a similar fashion with a second stringer at 47 serving to receive flange 31 of said ceiling coves. To provide an uninterrupted surface between the wall component W and the floor and ceiling coves, the latter are offset somewhat inwardly from the surface ofthe wall component to permit a quantity of plaster or the like to conceal the interface between cove and wall component. Desirably the wall component may be slightly recessed along its upper and lower edges to receive the plaster and provide a smooth surface.

With attention to the floor and ceiling corner coves 33, the interface or joints between the side flanges ofthe corner coves and the side flanges of the floor and ceiling coves are concealed by the application of plaster or the like which occupies the side marginal areas 11-12 and 26-27 of the floor and ceiling coves and extends over the joint and merges with the curved surface of the floor and ceiling corner coves in a feather- edged manner. Accordingly, the surface ofthe cyclorama is characterized by being continuous without irregularities or interruption which would cause shadows to appear to destroy the desired image of infinity behind the subject being photographed or filmed. The surface could, if required, be coated with a masking paint such as that manufactured by Ultimatte® or Chromeakey® .

While FIG. 1 shows the cove components visible, it will be understood that all joints in the cyclorama will be concealed. The lower corner coves 32 by reason of being of increased thickness at 40, as well as lower portion 14 of lower coves 3, permit anchoring devices to be recessed therein.

The floor and ceiling coves may be used mdependently, that is, a series of lower coves alone or lower and upper coves with corner coves at the intersection of wall components depending on the size ofthe cyclorama.

Turning now to FIG. 11, the external framework of one embodiment is shown generally at 50. Framework 50 includes upright 52, base 54, angled support 56, and connectors 58 and nut and bolt assemblies 60 for attaching the cyclorama panels to the framework. Uprights 52 include a plurality of preformed holes for use in mounting transmitters 62 outside the walls ofthe cyclorama, which are preferably made from ABS or fiberglass.

Transmitter 62 emits an electromagnetic radiation, preferably an RF frequency, although a spread frequency could also be employed.

The signal is transmitted through the cyclorama panels. In this way, the visual integrity of the cyclorama inner surface is maintained. In one embodiment as shown in FIG. 12, five RF transmitters are mounted on a rectilinear framework which is used to support a generally rectilinear cyclorama. One transmitter 62 is positioned outside each vertical wall W, and one (not shown) above the ceiling. A camera 2 is movably located within the cyclorama. More or fewer transmitters can be employed if preferred.

In another preferred embodiment, the transmitters are placed in known locations far above the camera and cyclorama, for instance the upper corners of a studio. In this way, three transmitters can be used to pinpoint the precise spacial location of the camera since three transmitters yields a solution for two possible locations, one above the transmitters and one below, with the logical elimination of the point located above the transmitters.

As discussed above, while the invention is described in terms of a camera for purposes of illustration, the present invention is equally applicable for precisely determining the location of a movable projector or a head-mounted display unit for use in a virtual reality system. The camera is equipped with one or more RF receivers, each of which is capable of determining and conveying the relative position of a remote RF transmitter in terms of three-dimensional angular data. Such equipment is commonly used in global positioning and navigational systems wherein one or more receivers identify the positions of one or several transmitting satellites. A processing unit then uses the satellite location data to determine one's geographical position with a high degree of accuracy.

Similar principals are employed in the present invention. The precise location of each transmitter is determined relative to a fixed point within the cyclorama, preferably the centerpoint of the cyclorama. This "absolute transmitter location data" is stored as a data file which can be accessed by a position processing unit such as a PC or other micro-processor based device. One or more RF antennae or receivers are mounted on or immediately adjacent the camera assembly, and receive the RF transmission from a transmitter outside the cyclorama, and determine the precise location of each transmitter relative to the camera as a three-dimensional angular displacement (relative transmitter location). Antennae and receivers such as those used for global positioning systems, available from HMorrow, Inc. of Salem, Oregon for example, are modified to accommodate the much shorter distances involved. Relative transmitter location data is gathered for each transmitter and is transmitted to the camera position processing unit (CPPU). The CPPU then accesses the absolute transmitter location data file and, using the relative transmitter location data, calculates the precise camera location within the cyclorama relative to the centerpoint of the cyclorama. The camera position data is then transmitted to a processing unit or graphics computer for use in creating a dynamic synthetic image for subsequent transmission or display according to well-known principles. Pan and tilt parameters can be determined relative to a known location and calibration point (determined by the above method) to determine the direction in which the camera is pointing. Additionally, zoom and focus parameters can be detemined. These parameters are gathered using mount systems such as those available from Ultimatte®, Vinten®, or Rademec®. This information is then transmitted to a receiver linked to the camera position processing means for generation of the dynamic synthetic image.

In this way, it is possible to quickly and precisely monitor the position of a moving camera, and use that information, together with the pan, tilt, zoom and focus information provided by the camera and mount system, to continuously alter a synthetic environment or composited image responsive to movements of the camera, and to do so with essentially no time delay. Applicant has also come to understand for the first time that in order to ensure the highest degree of accuracy in determining the camera position within the cyclorama, the panels, framework 50, upright 52, base 54, angled support 56, connectors 58, and nut and bolt assemblies 60 must, without exception, be made of non-ferrous, non-magnetic materials. The use of ferrous or other magnetic materials in the cyclorama or framework introduces distortions in the RF fields generated by the transmitters, and thereby introduces small, but significant errors in the relative transmitter location data provided by the RF receivers. These errors can manifest as offset or overlap errors in a composited image.

Having described my invention in terms of the foregoing examples, those skilled in the art will understand that electromagnetic frequencies other than RF can be substituted, and that the cyclorama can take alternate forms, and can be formed from alternative materials without departing from the scope ofthe my invention. I claim all modifications and variation coming within the spirit and scope of the following claims.

Claims

I claim:

1. An apparatus for generating synthetic and absolute real time environments comprising: an enclosed real environment; a plurality of transmitters mounted in fixed, known, and spaced- apart locations outside of said real environment, each of said transmitters capable of transmitting an electromagnetic signal into said real environment; a camera located within said real environment for capturing a real image of the real environment at a focal point within the camera, said camera being capable of moving in a plurality of dimensions within the environment; a receiver movable with said camera and capable of receiving the electromagnetic signals from the plurality of transmitters; and camera positioning processing means linked to said receiver for calculating the location of the camera within the real environment in response to the electromagnetic signals received from said plurality of transmitters.

2. The apparatus of claim 1, further including: a detector located on the camera for detecting the pan of the camera within the real environment; a pan signal transmitter located on the camera for generating a pan signal in response to said detected pan and transmitting said pan signal; and a receiver, linked to said camera positioning processing means, capable of receiving said pan signal.

3. The apparatus of claim 1, further including: a detector located on the camera for detecting the tilt of the camera within the real environment; a tilt signal transmitter located on the camera for generating a tilt signal in response to said detected tilt and transmitting said tilt signal; and a receiver, linked to said camera positioning processing means, capable of receiving said tilt signal.

4. The apparatus of claim 1, further including: a detector located on the camera for detecting the zoom of the camera within the real environment; a zoom signal transmitter located on the camera for generating a zoom signal in response to said detected zoom and transmitting said zoom signal; and a receiver, linked to said camera positioning processing means, capable of receiving said zoom signal.

5. The apparatus of claim 1, further including: a detector located on the camera for detecting the focus ofthe camera within the real environment; a focus signal transmitter located on the camera for generating a focus signal in response to said detected focus and transmitting said focus signal; and a receiver, linked to said camera positioning processing means, capable of receiving said focus signal.

6. The apparatus of claim 1, further including: a detector located on the camera for detecting the roll of the camera within the real environment; a roll signal transmitter located on the camera for generating a roll signal in response to said detected roll and transmitting said roll signal; and a receiver, linked to said camera positioning processing means, capable of receiving said roll signal.

7. The apparatus of claim 1, further including: an image processor capable of receiving said captured image from said camera; and computer image generation means linked to said image processor and said camera positioning processing means for generating a synthetic image and transposing said synthetic image on said captured image in response to said calculated location of said camera.

8. The apparatus of claim 1, further including a cyclorama containing the real environment therein.

9. The apparatus of claim 8 wherein the transmitters are mounted on the cyclorama.

10. The apparatus of claim 9 wherein the cyclorama is made substantially of non-ferrous materials

11. The apparatus of claim 1 wherein the transmitters and receiver are RF-based.

12. A method for synchronizing a captured image taken within a real environment with a synthetic image generated by a computer comprising: capturing a real image of a real environment with a camera positioned at a location within the real environment; determining the location of the camera within the real environment; determining other parameters of the camera taken from the set consisting of pan, tilt, zoom, focus and roll; generating a synthetic image in response to said determined location and said determined parameters; and superimposing said synthetic image on said real image.

13. The method of claim 12 wherein the step of determining the location of the camera includes: providing a plurality of transmitters; mounting said transmitters at spaced apart locations relative to the real environment; fixing a location of each of said transmitters relative to said fixed point in said real environment; transmitting a signal from each of said transmitters to a receiver movably mounted with said camera; and calculating the location of the camera in response to the signals received by said receiver.

14. The method of claim 13 wherein the step of fixing a location of each of said transmitters includes: using a global positioning system to determine the location of each transmitter.

15. An apparatus for generating synthetic and absolute real time environments comprising: an enclosed real environment; a camera located within said real environment for capturing a real image of the real environment at a focal point within the camera; a camera communication device positioned at a preselected location offset relative to the focal point of the camera; a plurality of data communication devices mounted in fixed, known, and spaced-apart locations outside of said real environment, said data communication devices being in wireless communication with said camera communication device to enable detection of the distance of the camera communication device from the data communication devices; data generation means for generating location data in response to said wireless communication; and camera positioning processing means linked to said data generation means for calculating the location of the camera within the real environment from data received from the data generation means.