WO2002009445A2

WO2002009445A2 - System and method for interactive television

Info

Publication number: WO2002009445A2
Application number: PCT/US2001/023215
Authority: WO
Inventors: Venkati Peri; Scott Nurmi; Jeb Hurley
Original assignee: Cyclovision Technologies, Inc.
Priority date: 2000-07-24
Filing date: 2001-07-24
Publication date: 2002-01-31
Also published as: WO2002009445A3; AU2001280723A1

Abstract

An apparatus for interactive television that employs immersive video is disclosed. In a preferred embodiment, a camera and immersive camera is used to capture a scene of interest. Video signals from both cameras are modulated and broadcast by a television station. Ultimately, a video is presented as the major picture on a television screen, while an immersive video is presented as an inset, and vice versa.

Description

SYSTEM AND METHOD FOR INTERACTIVE TELEVISION

FIELD OF INVENTION The present invention is generally related to the field of immersive video, or television, imaging. More particularly, the present invention relates to interactive television employing immersive video.

BACKGROUND OF THE INVENTION

Television today allows a user a limited experience. Limitations in viewing video and television is based primarily on a single video stream transmitted to a passive viewer, where the camera's viewing angle is restricted. A viewer generally has the two very limited options of watching the particular video stream and/or viewing the recorded video of the particular video stream.

The following prior patents represent the state of the art of immersive video, or television, images, and are all hereby incorporated by reference:

U. S. Patent No. 5,850,352 to Moezzi et al . discloses a system and method of synthesizing diverse spatially and temporally coherent and consistent virtual video cameras, and a virtual video images, from multiple real video images that are obtained by multiple real video cameras. It further concerns the creation of three-dimensional video image models, and the location and dynamic tracking of video images of selected objects depicted in the models for, among other purposes, the selection of a real camera or image, or the synthesis of a virtual camera or image, best showing the object selected.

U. S. Patent No. 5,760,826 to Nayar discloses an omni-directional imaging system that uses a convex paraboloidal mirror, telecentric relay objective lens and a standard camera lens which projects an annular image of a substantially hemispherical scene captured from a single viewpoint onto a planar sensing device, such as a CCD .

U. S. Patent No. 5,745,126 to Jain et al . discloses a method and system of synthesizing a virtual video camera, and a virtual video image, from multiple real video images obtained by multiple real video cameras . U. S. Patent No. 5,729,471 to Jain et al . discloses the automated dynamic selection of one video camera/image from multiple real video cameras/images in accordance with a particular perspective, an object in the scene, or an event in the video scene . With the proliferation of personal computers and such advances in video and imaging technology, there has been a growing demand and interest for interactive movies, video-on-demand, interactive television, and the like. Thus, it has become desirable to provide viewers with the freedom to choose the viewing direction.

SUMMARY OF THE INVENTION A feature and advantage of the present invention over the above-mentioned prior art, in interactive television is achieving interactivity by transmitting content data, such as audio and/or video data, over one or more television channels, such that at least one channel comprises immersive video.

The term 'television' herein refers generally to the ability to transmit and receive audio and/or video data over media, such as modulated electromagnetic signals (broadcast TV), cable networks, and digital networks, which includes a local and/or global network, such as the Internet . The term 'interactive' generally refers to the ability to change, on demand, a variety of parameters, including viewing parameters, such as the viewing direction or orientation of the camera.

The term "immersive video' generally refers to the imaging, from a reference viewpoint, a section of a sphere of view substantially larger than conventional video .

Advantageously, the present invention allows viewer interaction and/or editing of the video and television in a way that facilitates user-specified parameters of presentation .

In accordance with one embodiment of the present invention, the system comprises a panoramic video camera that transmits immersive video over a single television channel. The television receiver is suitably adapted to receive this immersive video, and convert it to conventional video, which is presented on a television screen. The remote control is also suitably modified to transmit view orientation commands that alter the conventional view of the immersive video, on demand.

In accordance with another exemplary embodiment of the present invention, the system comprises a conventional video camera and a panoramic video camera, both transmitting video of the same or related events over two separate television channels simultaneously. In this embodiment also, the television receiver is suitable adapted to receive, decode and present both channels on a television screen using picture-in-picture technology, where the immersive video is displayed as the inset picture. The remote control is also suitably modified to transmit view orientation commands, for example, that is capable of altering the conventional viewing of the immersive video, on demand. A computer or processor driven system, tangible medium including instructions thereon, and process is also provided.

There has thus been outlined, rather broadly, the important features of the invention in order that the detailed description thereof that follows may be better understood, and in order that the present contribution to the art may be better appreciated. There are, of course, additional features of the invention that will be described hereinafter and which will form the subject matter of the claims appended hereto.

In this respect, before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting.

As such, those skilled in the art will appreciate that the conception, upon which this disclosure is based, may readily be used as a basis for the designing of other structures, methods and systems for carrying out the several purposes of the present invention. It is important, therefore, that the claims be regarded as including such equivalent constructions insofar as they do not depart from the spirit and scope of the present invention .

Further, the purpose of the foregoing abstract is to enable the U. S. Patent and Trademark Office and the public generally, and especially scientists, engineers and practitioners in the art, who are not familiar with patent or legal terms or phraseology, to determine quickly from a cursory inspection, the nature and essence of the technical disclosure of the application. The abstract is neither intended to define the invention of the application, which is measured by the claims, nor is it intended to be limiting as to the scope of the invention in any way.

The above objects of the invention, together with other apparent objects of the invention, along with the various features of novelty that characterize the invention, are pointed out with particularity in the claims annexed to and forming a part of this disclosure. For a better understanding of the invention, its operating advantages and the specific objects attained by its uses, reference should be had to the accompanying drawings and descriptive matter, which illustrates preferred embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates a preferred embodiment of the present invention, showing modifications for immersive channels .

Figure 2 shows a block diagram of the receiving process . The same reference numerals refer to the same parts through the various figures .

NOTATIONS AND NOMENCLATURES The detailed description that follows may be presented in terms of program procedures executed on a computer or network of computers . These procedural descriptions and representations are the means used by those skilled in the art to most effectively convey the substance of their work to others skilled in the art. A procedure is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. These steps are those requiring physical manipulations of physical quantities. Usually, though no necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared and otherwise manipulated. It proves convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. It should be noted, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.

Further, the manipulations performed are often referred to in terms, such as adding or comparing, which are commonly associated with mental operations performed by a human operator. No such capability of a human operator is necessary, or desirable in most cases, in any of the operations described herein which form part of the present invention; the operations are machine operations.

Useful machines for performing the operation of the present invention include general purpose digital computers or similar devices.

The present invention also relates to apparatus for performing these operations . This apparatus may be specially constructed for the required purpose or it may comprise a general purpose computer as selectively activated or reconfigured by a computer program stored in a computer. The procedures presented herein are not inherently related to a particular computer or other apparatus . Various general purpose machines may be used with programs written in accordance with the teachings herein, or it may prove more convenient to construct more specialized apparatus to perform the required method steps . The required structure for a variety of these machines will appear from the description given.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS According to the present invention, an immersive video broadcast system is provided, which comprises: (a) panoramic video camera capable of capturing an immersive video of a scene of interest; (b) a video transmitter capable of transmitting said immersive video; and, optionally, (c) one or more non-panoramic video cameras. Moreover, the present invention relates to an immersive video broadcast and receiving system comprising: (a) panoramic video camera capable of capturing an immersive video of a scene of interest; (b) optionally, one or more non-panoramic video cameras each capable of capturing a conventional video of at least one perspective of the scene of interest; (c) one or more video transmitters capable of transmitting said immersive video and, optionally, said conventional video; and (d) a receiver capable of receiving at least said immersive video and, optionally, said conventional video. In a preferred embodiment of the invention, the immersive video broadcast and receiving system further comprises a converter capable of converting said immersive video to a conventional video perspective for display. Details of how the converter is configured can be found in Nayar , as it discusses geometric mapping. More preferably, the conventional video perspective is displayed on a television or computer monitor. The controller can be a remote control, or a mouse, or other pointing device.

According to a specific embodiment of the present invention, an immersive video broadcast system is contemplated, which comprises: (a) panoramic video camera capable of capturing an immersive video of a scene of interest; (b) one or more non-panoramic video cameras each capable of capturing a conventional video of at least one perspective of the scene of interest; and (c) one or more video transmitters capable of transmitting said immersive video and said conventional video. In yet another embodiment of the invention, a interactive television system is described, which comprises: (a) panoramic video camera capable of capturing an immersive video of a scene of interest; (b) optionally, one or more non-panoramic video cameras each capable of capturing a conventional video of at least one perspective of the scene of interest; (c) one or more video transmitters capable of transmitting said immersive video and, optionally, said conventional video; (d) a receiver capable of receiving at least said immersive video and, optionally, said conventional video; (e) a converter capable of converting said immersive video to a conventional video perspective for display; and (f) a view orientation controller for altering the conventional video perspective on demand. Referring now to Figure 1, a television camera 11 captures a scene of interest in conjunction with an immersive camera 12. Video signals from both cameras are modulated and broadcast by the television station 13 and antenna 14. At the receiving end, television antenna 15 receives the modulated electromagnetic signals. A picture-in- picture tuner in the television receiver 16 decodes regular television and immersive channels. For instance, a video or movie is presented as a motion picture 17 on the television screen, while an immersive video is presented as the inset picture 18 of the picture-in- picture display.

Here, the television is suitably modified to perform geometric transformations on the immersive video, to present the immersive video in a user-desired perspective, prior to display. A television remote 19, suitably modified, is used to control the viewing orientation, for example, of the immersive video.

Figure 2 shows the details of the receiving process . Signals from the antenna 15 are decoded by the picture- in-picture tuner of the television 21. The conventional data stream 23 is rendered as a major picture, while the immersive data stream is passed through a geometric mapping engine 22, which performs geometric mappings on the immersive video to generate a conventional representation of the perspective video, rendered thereafter as a minor picture or inset 18.

A remote 19 transmits user-specified commands, such as viewing orientation commands, to the mapping engine 22, which manipulates the command signals to produce the desired representation. Alternatively and optionally, it is possible to reverse the sized viewing of the major and minor pictures, such that the immersive video is shown as a major picture, and the conventional data stream is shown as a minor picture or inset .

The above embodiments are only to be construed as examples of the various different types of audio and/or video scene imaging systems that may be utilized in connection with the computer assisted and/or implemented process of the present invention. The many features and advantages of the invention are apparent from the detailed specification. Thus, it is intended by the appended claims to cover all such features and advantages of the invention that fall within the true spirit and scope of the invention. Further, since numerous modifications and variations will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation illustrated and described. Accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope of the invention.

Claims

CLAIMS What is claimed is:

1. An immersive video broadcast system comprising:

(a) panoramic video camera capable of capturing an immersive video of a scene of interest;

(b) a video transmitter capable of transmitting said immersive video; and, optionally,

(c) one or more non-panoramic video cameras.

2. An immersive video broadcast and receiving system comprising:

(a) panoramic video camera capable of capturing an immersive video of a scene of interest; (b) optionally, one or more non-panoramic video cameras each capable of capturing a conventional video of at least one perspective of the scene of interest;

(c) one or more video transmitters capable of transmitting said immersive video and, optionally, said conventional video; and

(d) a receiver capable of receiving at least said immersive video and, optionally, said conventional video.

3. The immersive video broadcast and receiving system of claim 2 which further comprises a converter capable of converting said immersive video to a conventional video perspective for display.

4. The immersive video broadcast and receiving system of claim 3 in which said conventional video perspective is displayed on a television or computer monitor.

5. An immersive video broadcast system comprising: (a) panoramic video camera capable of capturing an immersive video of a scene of interest;

(b) one or more non-panoramic video cameras each capable of capturing a conventional video of at least one perspective of the scene of interest; and (c) one or more video transmitters capable of transmitting said immersive video and said conventional video .

6. An interactive television system comprising: (a) panoramic video camera capable of capturing an immersive video of a scene of interest;

(b) optionally, one or more non-panoramic video cameras each capable of capturing a conventional video of at least one perspective of the scene of interest; (c) one or more video transmitters capable of transmitting said immersive video and, optionally, said conventional video;

(d) a receiver capable of receiving at least said immersive video and, optionally, said conventional video;

(e) a converter capable of converting said immersive video to a conventional video perspective for display; and

(f) a view orientation controller for altering the conventional video perspective on demand.

7. An interactive computerized system for presenting or manipulating immersive images, said system comprising: (a) panoramic camera means for capturing an image in order to transmit video signals representative of immersive video;

(b) receiver means for receiving and converting said video signals into a desired viewing format; and (c) control means for transmitting commands capable of altering viewing of said immersive video on demand.

8. The system according to claim 7, wherein video signals are transmitted over a plurality of inter- connected computer networks comprising at least one of a local network, a global network, and the Internet.

9. The system according to claim 7, wherein video signals are transmitted over modulated electromagnetic media.

10. An immersive video system for interactive television, said system comprising: (a) a panoramic video camera configured for transmitting video signals representative of immersive video ;

(b) a television camera configured for capturing a scene of interest; and (c) a television station.

11. A method for processing immersive video signals for interactive television viewing, said method comprising the steps of: (a) capturing a scene of interest using one or more of a video camera or immersive camera;

(b) modulating video signals received from one or more cameras ;

(c) decoding the modulated signals into one or more television or immersive videos or channels.

12. The method according to claim 11, further including the step of performing one or more geometric transformations on each video, prior to display.

13. The method according to claim 11, wherein the step of capturing includes a step of capturing a video of the same or related event .

14. The method according to claim 11, further including the step of decoding one or more television or immersive videos or channels simultaneously on a television screen.

15. The method according to claim 11, further including a step of transmitting viewing orientation commands that alter viewing of the immersive video.