WO1997012314A1 - A lap-top device and method for accessing and controlling communication services - Google Patents

Abstract

An interfacing device and method for controlling and accessing information and communication services and applications are described (10, 11, 12, 17, 18). The method relates to a common virtual canvas linking the display space of a large, central public display (17) with the private work and display spaces of ambulatory, high-resolution touch-activated devices, such as lap-top controllers (10). Instead of actually transferring information from one lap-top controller (10) to another or to the public space, with a lap-top controller (10) the users are actually controlling the permission of others to access certain information transmitting the information to the set-top box (10, 11, 12, 17, 18). Therefore, the user is actually controlling the state of their collective virtual canvas by manipulating permissions which determine each user's view (17) of multimedia information stored in central network-based servers.

Description

A LAP-TOP DEVICE AND METHOD FOR ACCESSING AND CONTROLLING COMMUNICATION SERVICES

RELATED APPLICATION This application is a continuation of provisional U.S. Patent Application Serial No. 60/004,519 filed on September 29, 1995, having the same title, inventor, and assignee as the present application.

HELD OF THE INVENTION My invention relates to an interface device and method specifically for accessing and controlling information and communication services and applications.

BACKGROUND OF THE INVENTION

As the vision for a powerful National Information Infrastructure (Nil) comprising an interconnection of intelligent, broadband digital networks is more clearly defined, it becomes apparent that user interfaces to control these all-purpose networks will have to satisfy a number of conflicting requirements imposed by the wide range of high-quality, interactive multimedia applications. Users will need to read textual information from screens without fatigue, much as people do now with today's books, magazines and other documents. Large screens will display high-resolution, full-motion entertainment-quality multimedia applications, allowing users to immerse themselves fully in the visual imagery. But these users will need simple and easily remembered methods for controlling information access, and these methods will need to remain fairly constant across multiple applications. Finally, user interfaces that inherently support cooperative and collaborative entertainment, work, and play will be advantaged in the new information age. Therefore, a relatively small set of powerful, easy to learn, and consistent methods will have to be developed if users are to access a broad range of Nil services successfully. If such uniform methods are not established, there could be a number of tools with diverse human interfaces and methods that may curtail the deployment and success of the NIL

To these ends, objectives of my invention are a new interface device and method for controlling information and communication services and applications on different networks.

SUMMARY OF THE INVENTION My invention relates to an interface device and method for accessing and controlling information and communication services and applications on different networks. My method is based upon the concept of a common virtual canvas linking the display space of a large, central public display with the private work and display spaces of ambulatory, high-resolution, touch-activated devices, such as a lap-top controllers Instead of actually transferring information from one lap-top controller to another or to a public space, with my lap-top controller the users are actually controlling the permission of others to access certain information By pushing information into a public space, the user signals a controlling set top box to permit the information to be transmitted to a public display, without the user actually transmitting the information to the set top box Therefore, the user is actually controlling the state of their collective virtual canvas by manipulating permissions which determine each user"s view of multimedia information stored in central network-based servers

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 A depicts an illustrative embodiment of a system comprising two lap¬ top controllers linked to a set top box and a large television set in accordance with an aspect of my invention Figure IB depicts an illustrative embodiment of my lap-top controller in accordance with an aspect of my invention

Figure 2 depicts the transfer of objects from the user's private space on my lap-top controller to a shared public view, in accordance with an aspect of my invention Figure 3 depicts the lap-top controller being used to zoom into details of buttons on a dress from a home shopping application in accordance with an aspect of

invention

Figure 4 depicts two lap-top controllers at the same location used to work collaboratively on the shared public space in accordance with an aspect of my invention Figure 5 illustratively demonstrates the transfer of multimedia directly between lap-top controllers in accordance with an aspect of my invention

Figure 6 illustratively demonstrates the transfer of multimedia information over a network using lap-top controllers in accordance with an aspect of my invention

Figure 7 illustratively depicts the video streamer technology used to access or preview video to be displayed on shared public space in accordance with an aspect of my invention

Figure 8 depicts illustrative embodiments of display screens for lap-top controllers in accordance with an aspect of my invention.

Figure 9 depicts an illustrative embodiment of a display screen of a lap-top controller with a TV listings interface in accordance with an aspect of my invention Figure 10 depicts an illustrative embodiment of a display screen for lap-top controllers being used to preview still frames from several TV channels before making a selection for programming on a large, shared, display in accordance with an aspect of my invention.

The organization and operation of the invention will be better understood from consideration of the detailed description of the illustrative embodiments thereof, which follows, when taken in conjunction with the accompanying drawings.

DETAILED DESCRIPTION OF THE INVENTION

My invention relates to an interface device and method for controlling information and communication services and applications on different networks. My method is based on the concept of a common virtual canvas which links the display space of a large, central public display with the private work space and display space of ambulatory, high-resolution, touch-activated devices.

My user interface system solves some immediate problems for emerging broadband video distribution channels, while positioning for future full-service networks and ultimately for the Nil. While this system is suitable for all market segments, this application focuses illustratively on residential applications for ease of discussion. Most users of home television services currently have a television receiver with a remote control device. Currently, most providers of advanced information networks expect that some sort of set-top box will be used to access the advanced offerings of full service networks (FSNs) and that the television receiver will be the only output device for most of these services. Therefore, a single display device must handle technically conflicting requirements, namely displaying bright and colorful full motion entertainment video along with the high-resolution text and graphics that will be needed to support user interfaces for services such as video browsing and information searching.a

I take a different approach to resolving the problem of uniform information access and display. Herein, I disclose using a set top box in conjunction with a new consumer interface device that would replace the remote control commonly used with most television receivers. This new device, which I refer to herein as a lap-top controller, has some aspects of the emerging personal digital assistants (PDAs) but is designed to be an integral part of the FSN interface system.

Illustratively as shown in Fig. IB and in accordance with my invention, the lap-top controller 10 comprises an infrared (IR) transceiver 11 and a radio frequency (RF) transceiver 12 for communicating with a set top box 14 which is connected to a shared-area large screen "public" display 15, as seen in Fig. IA. My lap-top controller 10 also comprises a display 17, having a large area for the user to view information located in his or her public and private spaces. Furthermore, my lap-top controller comprises a central control processor 18 which controls and coordinates the transfer, receipt, and display of information into and out of the controller based upon signalling from the user and other users and based upon software used to operate the lap-top controller

My lap-top controller is characterized as having a large display area, in display 17 the contents of which can be manipulated by the user by providing direct physical input via touching or use of a stylus, and might advantageously be approximately similar in size to the children's toy, "Etch-A-Sketch ' A display resolution of 75 pixels/inch or more would make graphics and text readable at convenient hand-held distances The device is capable of displaying full color video at 30 frames per second over its entire surface Although full motion video is important for some of the more advanced uses of the lap-top controller, many powerful new capabilities can be enabled with slower displays My experience with large screen video-conferencing systems indicates that a display area with an 8x3 aspect ratio (twice the 4x3 aspect ratio of NTSC television) would be most desirable as a preferred embodiment In addition to allowing more area to accommodate public and private viewing and work spaces, this wider aspect ratio would also allow the device to emulate a full-width keyboard if needed My lap-top controller is tightly coupled with the shared public display 15 (and other private displays) by RF and IR linking technologies and by operation software running in the network, the set top box, and in the lap-top controller itself

My lap-top controller is linked to the set top box with two channels One channel supports the high speed delivery of full-motion video and high resolution graphics to the controller The other channel is a bidirectional data link The two channels work cooperatively to allow the flow of multimedia information between the lap-top controller and the set top box By using the data link to control flow on the high speed link, both graphics and video could be accommodated

For user input, the lap-top controller includes a touch screen over the display surface With appropriate software in the lap-top controller, its user could use a fingertip (or stylus) for wπtten input or for image manipulation on the face of the display An experimental research prototype of an initial version of a lap-top controller was built around the Datalux Touch LCD Monitor The 640 X 480 pixel, 256 color, AMLCD flat panel display is adequate for the presentation of easily readable text and moderate resolution images It is mated with the Elo TouchSystems Accutouch® touchscreen which provides fingertip data access and control

Figure 1 A depicts an illustrative embodiment of my lap-top control system comprising two lap-top controllers 10-1 and 10-2 linked to a set top box 14 and a large television set 15 The set top box handles the interface with the FSN and provides wireless links to the lap-top controllers 10 and a wired connection to the television set 15 The television display is optimized for full-motion, full-color entertainment video and is essentially the same as present-day TVs This TV might have a very large (projection) display area, and might support enhanced resolution signals The display on the lap-top controllers can support small video windows, but are primarily designed for high resolution graphics and text

By providing separate viewing spaces the lap-top control system can support new and novel information manipulation methods These methods will facilitate access to a wide variety of information and communications applications and allow multimedia browsing in a powerful and versatile manner

An outstanding aspect of my lap-top controller and method is the linking of the pπvate display area of a lap-top controller with the larger, common public display area (the television set) and, under certain conditions the linking of one lap-top controller's display area, to the display areas of other lap-top controllers As shown in Figure IA, the display areas of the TV set 15 and of each of the lap-top controllers 10 are part of a single, large virtual canvas Objects can be moved with great facility within a space and from one space to another in a manner that simulates moving physical objects My lap- top controller 10 provides users a powerful visual representation of what is actually happening as they manipulate information on their screens

All objects that can be depicted on the surface of my lap-top controller can be manipulated by pushing them around on the display surface As with using a mouse, objects can be selected by touching them in a specified way They can be activated with a set of easily recalled pressure or touch sequences and can be pushed or dragged to new locations by simply touching them with a finger This includes repositioning video, graphic, or textual data as well as activating functions that operate on this data My approach can be applied even for data that has been slid onto public areas or other private areas on the user's display In one mode of operation, depicted in Figures 1 and 2, a fixed portion 20 of the lap-top controller's display space is used to represent the common space Users can assemble some needed combination of objects in their pπvate spaces 21 and, when satisfied with the result, slide them into the area of the public space Other users in the same physical area (e.g. living-room) can now see the final visual result of an effort that may have taken a good deal of local manipulation to generate Once the information is in the public space, other users can slide a copy into their own pπvate spaces on their lap¬ top controllers for their personal modification. If the information represents objects (in both the physical and software sense of the word) then these users can add these objects to their stock of executable objects In a related method, as illustratively depicted in Figure 3, the user may have a local display of the "common area" that can be shrunk or enlarged at will to envelop all, or any part of the user's private work space In one example, the common area could be used to expand a rectangular area that occupies a small (too small to view comfortably) area of the lap-top controller display Users could push this fixed-sized box around on their private space, causing the common space to act as a giant magnifying glass The area of the user's screen within the rectangle would blow up to fill the canvas of the shared screen In a related and more powerful capability, the movable common-screen representation area on the lap-top controller could be scaled up or down on the screen bv stretching it with the fingers If it is made bigger, the magnification going from lap-top controller to common area is reduced If the window is made smaller, the magnifying effect is made larger As a special case of the relationship between private and public spaces, the user's entire lap-top controller workspace could be dragged around on the virtual canvas as shown in Figure 4 If several users dragged their private spaces into the common space, the contents of their pπvate spaces could be simultaneously displayed for comparison on the large common area screen Their work could be easily reviewed, superimposed for comparison, or connected to form a larger common work

The transfer of multimedia objects between private spaces, and in conjunction with the public space, represents an important use of this new method Here, in addition to its own private area, each lap-top controller could call up a portion of its display area representing the pπvate space of another lap-top controller as shown in Figure 5 Passing information from one collaborator to another would then be as simple as sliding it into the other user's private space As it is moved into this space on the local controller, the information would appear in the pπvate space of the colleague's controller This sharing operation is analogous to the public/private area sharing discussed earlier A powerful corollary to the public/private space paradigm is that the network could be used to interconnect users at separate physical locations. Geographically separated users could then share multimedia information For example, as shown in Figure 6, a user in one location could push a picture of a zoo animal into his public space and have it appear in the public space of a colleague (or colleagues) across the country Instead of actually transferring information from one lap-top controller to another or to the public space, with my lap-top controller the user is actually controlling the permission of others to access certain information By pushing information into the public space, the user signals the set top box to permit the information to be transmitted to the public display, without the user actually transmitting the information to the set top box Therefore, the user is actually controlling the state of their collective virtual canvas by manipulating permissions which determine each user's view of multimedia information stored in central network-based servers In order to augment the lap-top controller's ability to access, as well as share multimedia information, there may advantageously be employed a multimedia play back mechanism, such as that discussed in U.S. Patent Application Serial No. 08/300,824, filed on September 2, 1994 for inventors Cruz, Rohall, Rosenberg, and Smoot which mechanism is referred to herein as Video Streamer. Video Streamer is a way to represent and display large amounts of video information in a two dimensional space as shown in Figure 7. Here, video frames 30 are sampled, at intervals, out of a stored video clip, and arranged along a time-line like an overlapping deck of playing cards (with the edges exposed). In order to use the interface, one assumes that a streamed version of, for instance, a movie one is interested in, is displayed on the lap-top controller's surface. A finger tip is dragged along the time axis 31 of the stack to locate key scenes in the movie. As the finger is moved, the nearest sampled still frame is displayed. A 90-minute movie could be perused in seconds, with only a small sub-sampling of actual visual information needing to be sent to the lap-top controller from a central, network-linked media server. If the user finds that the clip appeals to them, then the entire document could be pulled from storage with its play-out commencing at a selected frame.

A Streamer time axis can also represent nonlinear temporal relationships. For instance, preprocessing a video stream could determine major scene changes and thus extract the more important aspects of a movie or other presentation. When the stack of visual information is displayed, the individual samples could be spaced by distances on the time line "virtual Z axis" that represented their actual occurrences.

The "Z" axis of a Streamed output could represent parameters, other than time, that are related to various descriptors of the material being displayed. For instance, the individual samples could be residential homes in a continuum of increasing or decreasing prices. Houses which were similar in another dimension (e.g. floor space) could be shown in close proximity on a Streamer axis. Similarly, in a shopping application, men's suits could be lined up by color, material, or cost in separate Streamer stacks that were automatically compiled from a network data base. Although many home entertainment applications will not require a keyboard, there are other applications where the flexibility and precision of a keyboard are required. In these cases the lap-top controller could emulate a keyboard by using the touch screen capability in conjunction with a graphic or photo showing the desired keys. Audible feedback could emulate key "clicks." The power of this approach lies in its adaptability to the task at hand. User interfaces could be drastically simplified by limiting each keyboard to those keys needed at each stage of an application. An example of such a "soft keyboard" is shown in Figure 8 where a numeric keypad 40 is provided for the direct entry of TV channel numbers in a TV channel controller application.

My method, using the lap-top controller for controlling network applications has considerable flexibility. One can envisage lap-top controllers with advanced capabilities. In addition to the basic display and touch-sensor described above, a lap-top controller might allow for biometric access (e.g., hand print or shape ID), or might provide a microphone for use with voice recognition. Additional input devices might include a scanner for documents or a video camera for video-conferencing.

Application of my lap-top controller to movie browsing and to home shopping has been disclosed above, but there are some other application scenarios to further illustrate the versatility of the interface. A near-term, but powerful, use of a lap-top controller is for controlling access to TV programming. As cable and hybrid fiber-coax systems expand from 50 to a 100 or more channels, it becomes difficult for users to remember what is available to them. In the simplest case, each channel could be graphically represented by an icon 41 depicting the station logo as shown in Figure 8. These icons could be grouped according to classes (off the air, pay TV, personal favorites, etc.) The user would tune his or her TV by touching a button for the class of programming desired, then touching the icon of the desired service provider. Figure 8 shows "screen dumps" from my laboratory prototype of such a user interface. The right screen assumes that a "FAVORITES" button has been pressed to show a previously- designated group of channels. The user can then, for example, touch the CNN logo 41 to tune to that channel. The left screen shows how a "soft keypad" 40 could be provided to allow the user to tune to a channel by direct entry of its number. This example assumes that the user has touched first 5, then 6, to tune channel 56.

The TV access application detailed in Figure 8 has been implemented in our experimental research prototype and researchers are able to move between several screens simulating moving through a Television browsing session. Researchers can change channels by touching station logos, or by pressing the "buttons" on a pop-up keyboard.

Another powerful way to access TV channels would be to use the lap-top controller to display a program-listing grid as shown in Figure 9 (much like those found in TV Guide or newspaper listings). Although listings services have been proposed and trialed in the past, the use of the normal TV display has severely limited the scope and depth of their listings. The high resolution of the lap-top controller coupled with its proximity would allow the user to browse the entire TV listing for a particular night. A grid that overflowed the display could be simply scrolled by fingertip pressure. The user could tune to a desired program by touching the appropriate grid section in a definitive way. For example, the first touch could highlight the grid box 50 as shown in Figure 9 and the second touch could select it. A tentative selection could be previewed locally (as a live video inset) on the controller before it was selected for viewing on the large public display.

Many extra information features could be built into a service supplying the lap-top controller. For example, selecting a particular movie from a Video Dialtone network could initiate the display of a movie trailer on the lap-top controller screen.

In a scenario extrapolated from the growth of today's Picture-In-Picture television capabilities, the lap-top controller 10 can simultaneously display several TV channels as shown in Figure 10. Live video or freeze -frame -sampled selections could be used. The user can switch to a program of interest by simply touching the desired displayed channel image.

Advertisers might use the lap-remote to display advertisements without the need to disrupt programming on the public display (of course this would depend on whether they could be certain that their Ads would be viewed at all). The lap-top controller 10 provides an excellent interface to multimedia games, especially those where there is a notion of a private games area and a public one. Recent work at Bellcore led to an experimental research prototype of a multimedia, multi-player electronic card game environment. Players of an electronic card game used an electronic platform which supported an application simulating a card deck that could be dealt into private areas and then played on public areas of linked CRT screens.

The lap-top controller system supports true, physically separate, public and private areas. Service providers could construct realistic and compelling games in which players maintained cards or tokens in truly private areas, and played them on simulated (public display) tables or game boards. With secure network technology, one could support gambling and games of chance.

Action games such as a "Tank War" game could be supported by showing the common battle terrain on the public display while keeping the users' controls and gauges (e.g. fuel level) private on the lap-top controller screen.

In a tele-teaching scenario, each student might have a lap-top controller placed on a desktop. Private spaces would let each work at their own pace, possibly on different assignments. The instructor could have access to all of the individual private spaces on his/her own private display, either automatically or when granted by a student. Students could share work by simply sliding it onto the public board. The entire object-oriented student environment could be available to the instructor so he or she could examine the history of the student's work and offer aid when needed. Most of the lap-top controller devices descπbed here can be assembled from existing subsystems The envisioned computing power is within the range of current lap-top computers Although display technology is not yet able to provide the envisioned wide screen color display at a moderate price, the populanty of lap-top computers and other personal electronic devices is dπving the market in that direction Low cost touch screen technology is already available

Low cost consumer devices are available that use RF links to transmit television signals from one VCR to several TV sets within the home These can form the basis for the RF links discussed herein In addiuon, digital IR linked controllers have become nearly ubiquitous devices in many homes They provide wireless control of even low cost electronic devices Their technology would be suited to the lap-top controller A major concern for any non-tethered device is sufficient battery capacity at low cost Portable camcorders and cellular telephones are dπving this market This application would also be well matched to the capabilities of Bellcore's patented lithium ion battery technology, and in any event, the controller can always be returned to a charging cradle when not being used.

CONCLUSION I have descπbed a lap-top controller, a new interface for accessing and controlling FSN and ultimately Nil services This interface (in conjunction with a large. shared display satisfies the conflicting requirements of large displays for entertainment along with the need for closeup, high-resolution displays for the presentation of graphics and text The display combination is especially useful for information shaπng applications and where applications require some notion of public and private access to multimedia information A number of applications ranging from better access to television programming to home shopping have been disclosed Finally, the above- descnbed embodiments of the invention are intended to be illustrative only Numerous alternauve embodiments of the present invention may be devised without departure from the spirit and scope of the following claims

Claims

What is claimed is 1. A system for allowing transfer of information between multiple users, said system comprising a large display device, a plurality of individual controllers, each of said individual controllers having a display surface which defines a shared public display area and a private work display area, and a set-top box which communicates with each of said individual controllers and with said large display device 2 A system in accordance with claim 1 wherein said set set-top box has wireless communication with said individual controllers and wire connection to said display device 3 A system in accordance with claim 2 wherein said wireless communication compπses an IR video link and an RF data link between said set-top box and each of said controllers 4 A system in accordance with claim 3 wherein each of said individual controllers further comprises an IR transceiver, an RF transceiver, and a central processor 5 A system in accordance with claim 4 wherein each of said individual controllers further comprises a touch screen over the surface of said display surface 6 A system in accordance with claim 1 wherein said set-top box provides interconnection to an external communication network 7 A method for transfemng infoπnation between individual controllers and a common display device comprising the steps of assembling information in a private display area of an individual controller and transferring said information from said private display area of sid controller to said common display device. 8. The method in accordance with claim 7 wherein said individual controller has a display surface and a touch screen over said display surface, said step of transferring comprising touch dragging the information from said private display area to said public shared display area 9 The method in accordance with claim 7 wherein said step of forwarding comprises communicating by wireless from said individual controller to a set-top box and communicating by a wired connection from said set-top box to said common display device