CN100454220C - Control framework with a zoomable graphical user interface for organizing,selecting and launching media items - Google Patents

Abstract

Systems and methods provide a control framework for organizing, selecting and launching media items including graphical user interfaces coupled with an optional free space control device for collection of the basic primitives of point, click, scroll, hover and zoom, which permit for easy and rapid selection of media items, e.g., movies, songs etc., from larger or small collections. The remote control maps natural hand movements and gestures into relevant commands while the graphical display uses images, zooming for increased/decreased levels of detail and continuity of GUI objects to provide easy organization, selection and navigation among the media items by a user.

Description

Be used to organize, select and start the control architecture of media item

Technical field

The present invention relates to a kind of control architecture that is used to organize, select and start media item, relate in particular to the control architecture that is used to organize, select and start having of media item of scalable graphic user interface.

Background technology

The invention describes a kind of framework (framework) that is used to organize, select and start media item.The part of described framework comprise have location (point), the design and the operation of the graphical user interface of the convergent-divergent basic building block piece (building block) of click, rolling, hover (hover), more particularly, relate to can with the related graphic user interface of media item that uses with free space location telechiric device.

The technology relevant with the communication of information had development rapidly in the past few decades.TV, cell phone, internet and optical communication technology (only listing a few here) combine the consumer are submerged in during available information and amusement select.With the TV is example, has introduced CATV (cable television) service, satellite TV, pay-per-use film and video request program over nearest 30 years.Yet perhaps the televiewer of the sixties in 20th century is merely able to receive on their televisor usually is four or five wireless TV channels, and the televiewer of today has an opportunity to select in hundreds of and potential thousands of program and info-channel.At present mainly the video on demand techniques that uses in hotel etc. provides a kind of potential that is used for carrying out from thousands of movie program lists the home entertaining selection.For example digital video record (DVR) equipment of TiVo incorporated company (Ao Weisuojin street, California 2160,95002) release has further enlarged available selection.

For providing the technology of the ability of so much information and content, the terminal user provides chance and challenge to system designer and service supplier.Although a challenge is that the terminal user more is ready to have more selection usually and the selection of being unwilling diminishes, it is opposing that institute is handled in the promptly simple again fast selection that this hobby and they wish.Unfortunately, the terminal user by its development that visits the system of media program and interface make select to handle not only unhappy but also remarkable.Consider the example of TV programme once more.When TV is in the initial stage, determine to see which program is relative simple thing, this is because selectable number is less.People can consult the program guide of printing, this program guide is formatted as for example a series of row and column, and these row and columns have demonstrated at (1) contiguous television channel, program that (2) are transferred to these channels and the corresponding relation between (3) date and time.By regulating tuner knob, TV is tuned to required channel, and spectators watch selected program.After this, allow the remote control unit of spectators' distant tuning TV to be introduced into.This mutual increasing produced the existing picture that is called as " channel surfing " to user-TV, and whereby, spectators can promptly watch the short-movie section of broadcasting in a large amount of channels, thereby can learn fast at any given what program of time and can see.

Although the fact that exists the quantity of channel and the content that can watch sharply to increase, available usually user interface, the opertaing device that is used for TV are selected and in the past 30 years of framework in do not change too many.The program guide of printing remains the most popular mechanism that is used to transmit programme information.Having up and down, many button remote control of arrow remain most popular channel/content selecting apparatus.The people of design and realization TV user interface have become the existing selection processing and the direct extension of interface object to the reaction of the increase of available media content.Like this, the line number in the program guide of printing is increased to hold more channel.The number of the button in the telechiric device also is increased with the function and content supporting to add and handles, for example, shown in the image pattern 1 like that.Yet this method has increased spectators greatly and has browsed the required time of the information that can obtain and carry out the complexity of selecting required action.Controversially be, the essence of the trouble at existing interface has hindered the commerce of some services (for example video request programs) to be used, and this is because the consumer opposes to being the new service that too slow and too complicated interface has increased complexity again In the view of them.

Except increasing aspect bandwidth and the content, the integrated bottleneck problem of also aggravating user interface of technology.The consumer is making a response energetically and will select to buy integrated system rather than a large amount of separable modules.A combination that example is TV/VCR/DVD of this trend, three ingredients wherein were three independently assemblies in the past, were nowadays then frequently bought as an integrated unit.This trend can continue, potential net result is that major part (if not the whole) communication facilities that can find in the family at present will be combined as integrated unit, for example, TV/VCR/DVD/ internet access/radio/stereo unit.Even if those people that continue to buy separation assemblies also can expect mutual mutually between the seamless control of above-mentioned independent assembly and these discrete assemblies.Along with the growth of this integrated level, produced the potential possibility that makes user interface complicated more.For example, when being suggested, the number of the button on these general remote-control units is normally more than the number of buttons on single TV remote control unit or the VCR remote control unit in so-called " general " remote control unit (for example combination of TV remote control unit and VCR remote control unit function).If can not find the correct button in this telechiric device exactly, these have increased the button of number and function makes the people except can only be difficult to control other anything aspect control TV or VCR the simplest.In the time of many, these general telechiric devices can not provide enough button to visit control or the characteristic on the peculiar many aspects of some TV.In this case, still need the remote control unit of original device, and since the user interface issues that causes of integrated complexity still exist so handle the initial arguement of many remote controls.Some remote control unit have solved this problem by " soft " button that increases available expert order establishment.These soft key have subsidiary LCD display device sometimes and indicate their action.This mode also has defective, that is, if sight line is not transferred to telechiric device from TV, they just are difficult to use.Another defective of these remote control unit is to have adopted multi-mode to attempt to reduce number of buttons.In the general remote-control unit of these " medellings ", exist special button to select this telechiric device whether to communicate by letter with TV, DVD player, cable set top box and VCR etc.The problem that this has produced many usability comprises sending a command to wrong device, forcing the user to be sure of by watching telechiric device whether it is in correct pattern, and it can not give many devices integratedly provide any simplification.The largest benefit of these general remote-control units is that it can be organized into telechiric device by the command sequence that allows the user will be used for multiple arrangement certain integrated level is provided.This is task of difficulty, to such an extent as to many users employ the installation personnel of occupation to come their general remote-control unit is carried out program composition.

People have also made some and have attempted making the screen interface between terminal user and the media system to adapt to modern needs.Electronic program guides (EPG) is developed and is implemented to replace aforesaid media guidance.Early stage EPG provides the reprography product of the media guidance of printing substantially.For example, the cable service operator provides the EPG of simulation, and wherein, special-purpose channel has shown the slow rolling grid (grid) of channel and the program that they are correlated with in some time ranges (for example, following two hours).Even if it also is slowly tediously long utilizing this mode to roll by 100 channels, and can not upgrade to comprise a large amount of additional content arrangements (deployment) feasiblely, for example, video request program.The digital EPG of more profound complexity also is developed.In digital EPG, listing information (and optional application/system software) is passed to special-purpose EPG equipment, for example, and top box of digital machine (STB).The EPG of numeral has greater flexibility and comes to be the media system design user interface, and this is because they can provide local mutual, and can insert one or more contact beds the user with between with viewed media program.Can find an example at above-mentioned interface in the 6th, 421, No. 067 United States Patent (USP) of toll bar people such as (Kamen), the disclosed content of this article is merged in this paper as a reference.A kind of GUI (graphic user interface) that describes in ' No. 067 patent that Fig. 2 shows.In Fig. 2, according to people's such as toll bar patent, program channel has been listed on first hurdle 190, the current in progress program of second hurdle, 191 expressions, hurdle 192 is illustrated in the program of broadcast of following half an hour, and the 4th hurdle 193 is illustrated in the program of playing after half an hour half an hour.Baseball bat icon 121 represents to estimate that with this baseball game proceeds to and hurdle 192 time corresponding shelves across hurdle 191 and 192.Yet text box 111 does not extend feeding hurdle 192.This expression estimates that the rugby match can not extend to and hurdle 192 time corresponding shelves.As shown in the figure, pictograph icon 194 is illustrated in after the rugby match, and ABC will broadcast the horse racing program.Icon among Fig. 2 can use the vernier (not shown) to start (actuate) to carry out various operations, for example downloads the information relevant with selected program.In the U.S. the 6th, 314,575,6,412,110 and 6,557, other digital EPG and related interfaces also described in No. 350 patents, its disclosed content also is merged in this paper as a reference.

Yet, above-mentioned interface have (in other shortcoming) can not be easily between the small set of the big collection of media item and media item the problem of resize ratio (scale).For example, can work well for little collections of media items in the interface that depends on the rendition list, but is tedious for browsing big collections of media items still.Compare with the list interface that is used for media giant program set, the interface that depends on graduate navigation (navigation) (for example tree construction) can have and travels through speed faster, but is unsuitable for little collections of media items.In addition, the user trends towards losing user therein have to move through the interest that three layers or more multi-layered selection are handled in tree construction.For these all situations, present remote control unit travels through above-mentioned tabulation or hierarchy and carries out these with tedium ground more and select to handle by forcing the user to press button and knob down repeatedly.Can use (for example going up page turning and following page turning) when select jumping control, the user often has to see this telechiric device finding these concrete buttons, or the process of having to is trained to know that they exist just.

Therefore, expectation obtains simplifying organizational structure, technology and the system that control between user and the media system and screen interface and accelerated selection are handled, and allows the service provider by supplying the benefit that broad medium program and new service utilize the available bandwidth that end user device is increased to the user expediently simultaneously.In addition, also expectation provide can be easily and select apace to handle and no matter the interface of size of viewed collections of media items.

Summary of the invention

System and a method according to the invention has satisfied above-mentioned needs and other needs by the overall control architecture that is provided for organizing, select and start media item, and described general frame comprises the user interface framework that can select media item convenient and fast.Can use free space pointing devices to control above-mentioned framework, described free space pointing devices comprises the button and the scroll wheel of minimal set, is used for locating, clicking and roll with the selection of traversal relational graph user interface.The graphic animations of this exemplary graphic user interface (GUI) by using indicator in the screen, producing in the choice the time at described indicator hover, thereby and to select to amplify and dwindle with between global view and detailed view smoothly navigation be that the user provides feedback.Exemplary of the present invention uses image so that the details and the successional grade of increase/minimizing GUI object (allowing the user to navigate expediently) are carried out convergent-divergent.Graphic user interface according to the present invention organizing media item on virtual face is selected.Similar selection can be combined in together.Initially, the face view that this interface display is dwindled, and under most situation actual be chosen in this amplification stage and can not show its ball of wax.Along with user's amplification gradually, be revealed about the more details of groups of media items or selection.On different zoom level, can use different control, thereby make the user can play the selection group, select separately, or arrive virtual other parts to browse other relevant media item.

For this reason, the invention provides a kind of like this control architecture that is used to organize, select and start a plurality of media item, comprising: be used to organize the device of described a plurality of media item, described a plurality of media item are represented by a plurality of different images; Be used to navigate to the device of one of described a plurality of media item, one of described a plurality of media item are represented by one of described a plurality of different images; Be used to select one of described a plurality of media item to be used for different semantic grade device shown; And be used at the device that is converted to described different semantic grade by the current semantic grade that will show one of described a plurality of media item when carrying out following operation, the described size that changes one of described a plurality of different images that is operating as, and its primary importance from described current semantic grade is converted to the second place of described different semantic grade.

According to an exemplary of the present invention, a kind of method that is used in the graphic user interface display message comprises: show first group of image with first amplification stage; Receive first and select indication, described first selects the subclass of the described first group of image of indication identification; First zoom version that shows the described selection subclass of described first group of image with second amplification stage; Receive second and select indication, the image in the subclass of the described image of the described second selection indication identification; And second zoom version that shows the described image that is identified, wherein, but second zoom version of the described image that is identified comprises at least one user's start-up control element.Thereby can comprising, control in this embodiment use gyrostat or other mechanism to limit the free space pointing devices that screen position and displacement vector are determined required special command.Can also comprise one group of button and " retreating " button that is used for " click " or the incident of selection.

According to another exemplary of the present invention, a kind of graphic user interface comprises: a GUI screen, and it shows first group of image with first amplification stage; Be used to receive the device of the first selection indication, described first selects the subclass of the described first group of image of indication identification; The 2nd GUI screen, it shows first zoom version of subclass of the described selection of described first group of image with second amplification stage; Be used to receive the device of the second selection indication, described second selects the image in the described image subset of indication identification; And the 3rd GUI screen, it shows second zoom version of the image of described identification, wherein, described second zoom version of the described image that is identified comprises the bootable control element of at least one user.Can comprise the image kind that is nested into any degree of depth and the kind of kind according to the convergent-divergent graphic user interface of illustrative embodiments of the invention.

Description of drawings

Corresponding accompanying drawing is described exemplary of the present invention, wherein:

Fig. 1 shows the traditional remote control unit that is used for entertainment systems;

Fig. 2 shows traditional graphic user interface that is used for entertainment systems;

Fig. 3 shows an exemplary media system that can realize illustrative embodiments of the invention (display and telepilot) therein;

Fig. 4 has at length shown the system controller among Fig. 3;

Fig. 5-8 shows the graphic user interface that is used for media system of the exemplary according to the present invention;

Fig. 9 has described according to the present invention the example data structure of an exemplary;

Figure 10 (a) and 10 (b) have described dwindling and amplified version of a part of utilizing the exemplary GUI that the data structure of an exemplary is set up according to the present invention shown in Figure 9;

Figure 11 shows and is used to produce the ordered list that two strands that the GUI according to illustrative embodiments of the invention shows connects;

Figure 12 (a) and 12 (b) have shown the dwindling and amplified version an of part of another exemplary GUI, and it is used to describe the operation according to the node monitoring algorithms of illustrative embodiments of the invention;

Figure 13 (a) and 13 (b) show the example data structure according to illustrative embodiments of the invention, the operation of node monitoring algorithms when it is used to be described in the view conversion of the view of GUI from Figure 12 (a) in Figure 12 (b);

Figure 14 shows according to illustrative data structure illustrative embodiments of the invention, that comprise the virtual camera of the convergent-divergent of realizing the resolution unanimity;

Figure 15 (a) and 15 (b) have shown the dwindling and amplified version an of part of exemplary GUI, and it shows the semantic convergent-divergent according to illustrative embodiments of the invention;

Figure 16-20 shows the scalable graphic user interface according to illustrative embodiments of the invention;

One group of coverage control exemplary, that can be provided according to illustrative embodiments of the invention (overlay control) is provided Figure 21; And

Figure 22 has described the schematic framework that is used to realize according to scalable graphic user interface of the present invention.

Embodiment

The present invention will be described in detail with reference to the accompanying drawings.In different accompanying drawings, same or analogous element is represented with same reference number.In addition, following detailed description of the invention is not a limitation of the present invention.On the contrary, scope of the present invention is limited by claims.

For being provided, context so that discuss, at first can realize wherein that with reference to 3 couples in figure an exemplary integrated medium system 200 of the present invention is described in detail.But those of ordinary skills can predict, and the present invention is not limited in and realizes this class media system, but can comprise more or less assembly therein.In native system, I/O (I/O) bus 210 links together each assembly in the media system 200.210 representatives of this I/O bus are multiple to be used for any one of mechanism that the signal between the media system assembly is transmitted and technology.For example, this I/O bus 210 can comprise the independent audio that is used for transmitting audio signal " plug " cable of right quantity, the concentric cable that is used for transmission video signal, the two-wire system serial transmission line that is used for transmission of control signals or infrared ray or radio-frequency (RF) transceiver, optical fiber or any other transmission mechanism that is used to transmit the signal of other type.

In this exemplary embodiment, media system 200 comprises TV/monitor 212, video cassette register (VCR) 214, digitized video dish (DVD) record/playback apparatus 216, audio/video tuner 218 and small-sized dish player 220, and these devices all are connected on the I/O bus 210.VCR 214, DVD 216 and small-sized dish player 220 can be monochromatic light dish or single magnetic tape type equipment, also can be multiple optical disk or many magnetic tape types equipment.They can be unit or be integrated in together independently.In addition, media system 200 also comprises a microphone/speaker system 222, video camera 224 and wireless I/O control device 226.According to exemplary of the present invention, wireless I/O control device 226 is media system remote control unit of supporting the free space location, and the button that it has minimal amount navigates to be used for support, and communicates by RF signal and entertainment systems 200.For example, wireless I/O control device 226 can be utilize gyrostat (gyroscope) thereby or other mechanism limit the free space pointing devices that screen position and displacement vector are determined required special command.On wireless I/O control device 226, also can comprise and to be enabled in one group of button of " click " primitive (primitive) that describes below, and comprise " retreating " button.In another exemplary embodiment, wireless I/O control device 226 is media system remote control unit that the assembly by IR signal and media system 200 communicates.In another embodiment, wireless I/O control device 134 can be similar to exemplary media system telechiric device in appearance, and increasing simultaneously has the characteristic that can allow the user that tracking ball or other navigation mechanism of mouse position are set in the display of media system 100.

Media system 200 also comprises system controller 228.An exemplary embodiment according to the present invention is described, and system controller 228 can be operated with storage and show the entertainment system data that can obtain from a plurality of entertainment system datas source, and can control and the relevant various features of each assembly of system.As shown in Figure 3, in case of necessity, system controller 228 can be connected to each assembly of system directly or indirectly by I/O bus 210.In an exemplary embodiment, except that I/O bus 210, perhaps be used for substituting I/O bus 210, system controller 228 is equipped with wireless communication transmitter (or transceiver), and it can communicate by IR signal or RF Signals ﹠ Systems assembly.No matter controlling medium is what, and system controller 228 all is configured to control each media component in the media system 200 by graphic user interface hereinafter described.

Shown in Fig. 3 was further, media system 200 can be set for the media program of reception from a plurality of source of media and service provider.In this exemplary embodiment, media system 200 from following data source any or all reception information and selectively to its transmission information: electrophone 230, satellite broadcasting 232 (for example antenna) via satellite, very high frequency(VHF) of broadcasting and television network 234 (VHF) or ultrahigh frequency (UHF) radio communication (for example by aerial antenna), telephone network 236 and wire line MODEM 238 (perhaps other internet content source).Those skilled in the art will envision that media component shown in Figure 3 and source of media only are examples, media system 200 can comprise these assemblies more or still less.For example, the input for other form of said system comprises AM/FM wireless device and satellite radio device.

Fig. 4 is the block scheme that illustrates according to exemplary system controller 228 of the present invention.System controller 228 for example can be embodied as set-top box, and for example comprise processor 300, storer 302, display controller 304, miscellaneous equipment controller (for example, relevant device), one or more data storage device 308 and I/O interface 310 with other assembly of system 200.These assemblies communicate by bus 312 and processor 300.Those skilled in the art will envision that, can use one or more processing units to realize processor 300.Memory device 302 can for example comprise DRAM or SRM, ROM, in these memory devices some can be designated as the cache memory that the data used by software and/or these programs of processor 300 operation are treated in storage, and these softwares and data comprise software and/or the data that are associated with the graphic user interface that is described below.Display controller 304 can be presented at GUI screen and the object that describes below to be communicated with other thing together by the demonstration of processor 300 operations with control monitor 212.Scalable GUI according to illustrative embodiments of the invention provides and the irrelevant convergent-divergent of resolution (resolution), thereby makes monitor 212 to show with resolution arbitrarily.Device controller 306 provides other assembly of media system 200 and the interface between the processor 300.Data-carrier store 308 can comprise one or more hard disk drives, floppy disk, CD-ROM equipment or other high capacity memory device.Input/output interface 310 can comprise and one or more in the multiple interfaces comprises for example keyboard interface, RF interface, IR interface and microphone/speech interface.According to exemplary embodiment of the subject disclosure, I/O interface 310 will comprise the interface that is used to receive with the positional information of the motion association of radio positioner.

Carry out the instruction sequence that is contained in the storer 302 according to the graphic user interface of illustrative embodiments of the invention by system controller 228 answer processors and generate and control, select information with the display media item.Above-mentioned instruction can be read into the storer 302 from other computer readable medium (for example data storage device 308) or from the computing machine that is external in media system 200.Make processor thing together with other in monitor 212 generate graphical user interface object and control to the execution that is contained in the instruction sequence in the storer 302.In optional embodiment, can use hard-wired circuit to replace or make up and realize the present invention with software instruction.As mentioning in background technology part, traditional interface architecture related with television industry has serious defective aspect the experience ability providing simply for the user and comprehensively select.Therefore, the control architecture of Miao Shuing has overcome these restrictions in this article, and therefore is suitable for using with TV, does not get rid of certainly with other device and uses.In the interface that control architecture, graphic user interface and/or the various algorithm that also should predict the innovation of describing in this article can be applicable to use with computing machine and other non-television equipment.In order to distinguish these various application of exemplary of the present invention, use term " TV " and " TV " to refer to the subclass of display apparatus in this manual, and term " GUI ", " GUI screen " and " display screen " are common, and refer to television indicator, graphoscope and other any display equipment.More particularly, the intention of term " TV " and " TV " is meant and is need not to use adapter just TV signal (for example can be converted to other form, the subclass that can show the display apparatus of TV signal (for example, NTSC signal, PAL signal or SECAM signal) under the situation computer video form).In addition, term " TV " and " TV " refer to usually the display subclass (for example sofa is to the recreation room TV of family) that can watch from several feet or farther distance, and (for example, the display from the chair to the desktop) that graphoscope is normally closely watched.

Described can be used for realizing according to the control architecture that comprises scalable graphical interfaces of the present invention after, several embodiment at these interfaces are described now.According to exemplary of the present invention, but but user interface has shown the options of category grouping.The user points to interested one or more kind with remote control unit, and by select button amplifying, or press " retreating " button convergent-divergent and return original state.The action that user's each amplification or convergent-divergent return original state all makes the amplification stage and/or the content that are shown in the selectable program in the screen by user interface change.According to exemplary embodiment, the change of each amplification stage can be consistent, promptly changes amplification stage with predetermined rank.Exemplary of the present invention also provides to incorporate into has several visualization techniques to realize being amplified to the user interface of significant proportion.These technology comprise building block and can obtain the combination of scalable and wieldy technology, especially adopt user interface to improve user's visual memory with the technology of calling party interface object again apace.

User interface is visual experience to a great extent.In such environment, exemplary of the present invention has used the user can remember the ability of the object's position in this visual environment.This realizes by (dependable) position stable, that be correlated with is provided for the user interface options.Each object all has corresponding position in scalable layout.In case the user finds interested object, will remember the direction that object is set by the light of nature.If should to as if interested especially, user's this project of accessing more than once then, this can strengthen the user to the memory to the path of this object.User interface according to illustrative embodiments of the invention provides visual memory system (visual mnemonics), it can help the user remember the position of interested project.This visual memory system comprises mobile (pan) and convergent-divergent animation, the virtual transitions and consistent zoom function (functionality) of looking unfamiliar into the geography sensation of motion by user interface, based on the embodiment that describes below, they will become more apparent together with other thing.

Organization mechanism is provided to make the user to select from extremely many program set, and shielding simultaneously and a large amount of related details of selection group.According to the present invention, can use various types of organization mechanism, provide embodiment below.

At first arrive Fig. 8, wherein described the use of the exemplary control architecture that comprises scalable graphic user interface in demonstration and selection music media program according to illustrative embodiments of the invention with reference to Fig. 5.Fig. 5 has described to be in the scalable GUI of maximum deflated state.Wherein, interface display one group of shape (shape) 500.Be presented in each shape 500 is to describe text 502 and/or the picture 504 that these parts by GUI can be operated media program selection group.As shown in Figure 5, shape 500 is a rectangle, and text 502 and/or picture 504 described medium types.Yet, those skilled in the art will appreciate that this GUI that at first sees organizes the others that can represent that the available medium of user are selected, for example, artist, produce time, artistical residential area, the length of program or the further feature of selection.And if be used for not necessarily rectangle of shape that in GUI summary describes various groupings.The text 500 and/or picture 504 in shape group 500, or as to the substituting of text 500 and/or picture 504, the down-scaled version of album cover (album cover) and other icon can be used for further browsing prompting for the user provides.The background parts of GUI 506 can be shown as pure color, maybe can be the part of the picture of mapping graph for example, remembers the locus of kind to help the user, thereby needs less reading just can use this interface afterwards.Select steady arm (vernier) 508 to follow the motion of input equipment, and when the user presses button (not shown in Fig. 5) in this equipment, point out the position that to amplify.

According to an exemplary of the present invention, input equipment can be a wireless mouse, for example, by Gyration inc. (revolution company limited), (be positioned at the boulevard, Saratoga 12930 in California, Saratoga mansion C, 95070 (12930 Saratoga Avenue, Bldg.C, Saratoga, CA 95070)) wireless mouse made, it and the graphical user interface coupling of supporting location, click, rolling, hover and convergent-divergent building block (will carry out more detailed description below).Can bring a feature of this exemplary input equipment of beneficial effect to be when using together with the present invention, it only has two buttons and a scroll wheel, and three inputs just start object.One in these buttons can be configured to amplify (selection) button, and another button can be configured to dwindle (retreating) button.Compare with traditional remote control control module for example shown in Figure 1, the present invention waits aspect these that simplify GUI that the user faced when selecting his or her media program by the number that significantly reduces button.Additional preferred (but be not must) feature according to the input equipment of illustrative embodiments of the invention is that they provide the ability of " free space location " for the user.The phrase that uses in this instructions " free space location " refers to that user's aerial three-dimensional (or more) space free before display screen moves the ability of input equipment, and user interface directly is converted to these motions the respective capabilities of the motion of vernier on screen.Therefore, " free space location " is different from traditional computer mouse location technology, traditional computer mouse location technology uses the surface (for example table surface or mouse pad) that is different from display screen as agency (proxy) surface, comes from these surperficial mouse relative motions and is converted to vernier motion in the computer display.Use free space to locate and control the selection course of further having simplified the user according to the framework of illustrative embodiments of the invention, and the chance that posture (gesture) is incorporated into above-mentioned interface as diacritic input is provided simultaneously.Posture can be counted as can time dependently discern motor pattern, and this pattern can be converted into GUI order, for example with x, y, z, yaw (yaw), pitching (pitch) and wave dimension or their functions of the motion of sub-portfolio arbitrarily.Yet those skilled in the art should be realized that the input equipment that is fit to arbitrarily can use with scalable GUI according to the present invention.The example of the input equipment that other is fit to include but not limited to tracking ball, touch pads (touchpad), traditional TV remote control equipment, voice-input device, arbitrarily can be with user's posture transmission/convert the equipment of GUI order to, or their combination in any.Its each aspect that is intended that the GUI function that is described in this article can both use at least one posture and voice command to start in framework according to the present invention.Optionally realize comprising use vernier and/or other teleswitch or or even phonetic entry come the program of identification selection.

If Fig. 6 has shown the figure that the user selects the words of kind 3 (for example, by in display 212 that vernier 508 is mobile on kind 3 region surrounded at straight-flanked ring, and pressing the button) that the kind 3 that is shown is amplified in input equipment from Fig. 5.Interface of the present invention can produce the convergent-divergent from Fig. 5 to Fig. 6, thereby makes the clear convergent-divergent that produced of user.The embodiment of the convergent-divergent/transitions of above-mentioned generation will be described below.Taken screen in the most display 212 in case comprise the shape 516 of kind 3, above-mentioned interface display in mentioned kind, have an artist of special edition.In this embodiment, seven different artists and/or their works have been shown.The unselected kind 515 adjacent with kind 3 (its in Fig. 5 for dwindling view) is still adjacent with the kind 3 of amplifying on view, but it is in the state of partly being sheared at the edge of display 212.These unselected kinds can be carried out fast browsing to their selection by select finger 508.Yet should be realized that other exemplary of the present invention can be ignored is sheared adjacent object, only shows the selection that is not sheared on the contrary.In the artist group (for example, the artist organizes 512) each can both comprise the image that dwindles album cover, artistical photo or can be by the artistic work (if above-mentioned classification comprises the list of artists that the user sets up) of customization.

The user then can select artist's group to be used for further browsing and/or selecting.Fig. 7 has shown that the response user via vernier 508 being positioned and starting input equipment and selected the view of further amplification of the artist 3, wherein, can see the picture of album cover 520.The same as figure from the GUI screen transformation of Fig. 5 and Fig. 6, non-selected, contiguous artist's (being artist # 2,6 and 7 in this embodiment) is shown as the side in the demonstration that is exaggerated, the user can use select finger 508 that they are clicked, thereby sees these artistical whole views.Except the image 520 of album cover, in this part of interface, artist information 524 can show as the program in artist's group.These information can comprise for example artistical photo, biography, trival matters, works catalogue, influence, link network address and other relevant data.The picture of each the comprised photograph album front cover in the album picture 520, and include text data alternatively.Comprise at album picture 520 under the situation of the list of artists that the user creates that graphic user interface can show the picture by this interface is selected automatically or the user selects in advance.

At last, when the user selected album cover picture 520 from organizing 521, above-mentioned interface zoomed to album cover as shown in Figure 8.Continuation along with convergent-divergent, this album cover can move back for or become and comprise the like this view of some projects, for example, the littler version 528 of the artist of special edition 530 and title, song tabulation 532, the further information 536 relevant, album cover with special edition and be used for playing back content, modification is classified, is linked to artist's webpage, or find out and the control information 534 of selecting relevant any out of Memory.Adjacent special edition 538 demonstrates those and can select by using select finger 508, so that their content of interface display.That is as above mentioned is such, and alternate embodiment of the present invention can for example be amplified to and only show selecteed object (for example, special edition 5), and ignores the cutting out section of non-selected object (for example, special edition 4 and 6).This final convergent-divergent provides an example of semantic property convergent-divergent (semantic zooming), and wherein some GUI element is revealed, and it then is sightless in the previous zoom level.Variously be used to realize that the technology according to illustrative embodiments of the invention is described below.

As shown in Fig. 5-8 and the instructions, the graphic user interface of this exemplary provides browsing music collections.Can also be used for the video collection according to interface of the present invention, for example be used for medium, video request program, video segment and the home theater of DVD, VHS band, other record.Other audio frequency uses and comprises browsing radio programming, teaching tape, the historical accounts of past events and sound montage.Can also use the present invention to organize and visit printing or text media, for example News Stories and e-book.

According to the description of front, those skilled in the art will appreciate that scalable graphic user interface according to the present invention provides fast and easily browsed the ability of a large amount of (or less) media item for the user.This ability is attributable to the numerous characteristics at the interface of illustrative embodiments of the invention, these characteristics include but not limited to: (1) uses image as all or part of selection information that is used for the specific media item, (2) use convergent-divergent to provide fast and select required many as far as possible or the least possible information with the user, and (3) use several GUI technology, they are combined and all are positioned at conplane sensation to all interfaces of user, thereby can finish browsing of GUI, and be remembered by user's sense of orientation.Can various GUI screens " reasoningly " be coupled together together with other thing according to the back one side of GUI of the present invention, it keeps realizing continuously from a GUI screen to next GUI screen by making GUI object as much as possible, for example, by showing edge adjacent, that be surrounded on the non-selected object at current GUI screen edge.Alternatively, if expectation obtains view clearly and other GUI technology provides ground rational feedback, then can ignore the object that is sheared.As using in this article kind, phrase " GUI screen " refers to be presented at one time one group of GUI object in one or more display units.The GUI screen can be presented in the same display of output medium item, and perhaps it can be present in the different displays.Display can be the TV display, computer monitor or other suitable GUI output device arbitrarily.

Another improves the user and feel that the GUI screen is that internuncial GUI effect is when carrying out convergent-divergent or mobile animation (panning animation) effect that produces when the contiguous object that the user selects with the zoom level of current selecteed object equates.Turn back to embodiment shown in Figure 5, when the user watched this GUI screen at first, his or her observation point concentrated on a little near 550.Yet when he or she selected kind 3 to amplify, his or her observation point transferred to a little 552.According to illustrative embodiments of the invention, processing and amplifying is energized so that the POV center is transferred to a little 552 from putting 550.This mobile animation can be used to change various GUI, for example from the change of zoom level, or an object is changed to the change of another object in the same GUI amplification stage.Therefore, if (for example) is arranged in the non-selected kind 515 (kind 2) that the user of the GUI screen of Fig. 6 selects the leftmost side, the animation that then will be moved, this will be to the user visually with influence left or west " moving ".It is consistent sensation that exemplary of the present invention uses above-mentioned technology that the directed movement between the GUI screen is provided, thereby makes the user browse GUI between the amplification stage and between the media item at identical amplification stage more apace.

Can use various data structures and algorithm to realize according to scalable GUI of the present invention.For example, be used for that image viewer in display photos moves and the data structure of convergent-divergent and the paper of algorithm below for example in be described: " Quantum Treemaps andBubblemaps for a Zoomable Image Browser ", Benjamin B.Bederson, UIST2001, ACM Symposium on User Interface Software and Technology, CHI Letters, 3 (2), (Benjamin B compares gloomyly to be entitled as " the quantum number figure and the bubbling figure that are used for scalable image viewer " pp.71-80, UIST2001, the ACM forum of user interface software and technology, the CHI journal, 3 (2), the 71-80 page or leaf), it is merged in this paper as a reference.Yet, that medium are selected in order to be provided for, can in a large amount of application, switch with high grade, and can make the user control selected image to carry out the GUI of various medium selection functions, need additional structure and algorithm with lower grade.

Scalable GUI can be conceptualized as and can support to move and the convergent-divergent animation around the scene of the user interface components in the vision area of display device.In order to finish this influence, can use scene graph data structure (scenegraph data structure) to realize according to the scalable GUI of illustrative embodiments of the invention.Each node in scene graph is represented some part of user interface components, for example button or text label or a class boundary face assembly.The child node presentation graphic element (line, text, image etc.) of intra-node.For example, an application in scene graph can be represented as the node of the various child nodes with the graphic element that is used for its interface.The node of two kinds of specific types is called as camera and layering in this article.Camera is the node that vision area is provided to the other parts of scene graph by being conceived to hierarchical node.Below these hierarchical nodes, be user interface element.The steering logic that is used for scalable interface is adjusted the variation of camera sight line by programming, moves or zooming effect to provide.

Fig. 9 has shown the scene graph that comprises scalable substantially interface element, and these elements can be used for realizing illustrative embodiments of the invention, and specifically, this scene graph has comprised a camera node 900 and a hierarchical node 902.Dot-and-dash line between camera node 900 and the hierarchical node 902 represents that camera node 900 is configured to the child node of hierarchical node 902 is drawn (render) in the vision area of camera.The display apparatus 904 that is coupled makes the user see the vision area of camera.Hierarchical node has three child nodes 904 describing circle and a pair of ellipse.This scene graph also further points out, by node 912-918 drawn in above-mentioned the circle rectangle and the rectangle that drawn in three triangles.Scene graph is gone into other scene graph of other data structure by contact by root node 920.Among the node 906-918 each has by using transformation of local coordinates's matrix to make it carry out the ability of proportional zoom and location with respect to its father node.Figure 10 (a) and 10 (b) show respectively by camera and dwindle with first amplification stage and the appearance of the scene graph that amplifies with second amplification stage.

Drawing scene graph can be achieved as follows.When display 904 needed to upgrade, for example the user had triggered the amplification from the view of figure (a) to the view of figure (b), and the incident that then repaints is called the camera node 900 that is attached to display 904 with drafting camera node 900 self.Make camera node 900 notice hierarchical nodes 902 draw the interior zone of camera vision area so conversely.Hierarchical node 902 is drawn these child nodes self by the child node of notifying it and itself is drawn, or the like.The current transformation matrix and the bounding rectangle that are used for the above-mentioned zone renewal are transmitted (pass) in each step, and obtain alternatively revising to notify each node proper proportion that they should be used for drawing and skew.Because the applied scene figure that operates in scalable GUI according to the present invention can comprise thousands of nodes, so the zone that each node can detect transformation matrix and will be updated really can be seen by the user with the drafting operation of guaranteeing them.Although previously described embodiment has described the scene graph that comprises a camera node and a hierarchical node, should predict, exemplary of the present invention can embed a plurality of cameras and layering.These cameras that are embedded into can provide the user node element for the user, for example indicate the little figure that dwindles in the whole scalable interface current sight line of user zone, and can allow user interface components convergent-divergent and moving independently.

When coordinating the operation of use (for example as the exemplary cinematic browser that will describe with reference to Figure 14-18 below) at the scalable interface of use, the storage and the resource requirement that are used for each application may be above the total effectively storages at media system more.Just hint need be with some or all codes and the data unloading of these application when the user no longer sees them for this.Yet, in scalable GUI according to the present invention, provide such outward appearance to expect, that is, make some or all of application always look like activation for the user.In order to satisfy the target of above-mentioned two antagonism, be the state that the application of " curtain outer (off-screen) " can enter into interim hang-up for user's sight line.In order in scalable GUI according to the present invention, to realize incident being sent to this function application when enter and withdraw from vision area to indicate them.A mode that realizes above-mentioned incident is to increase logic in the code of render component, thereby makes and detect when the user enters vision area.Yet this notification logic of this meaning person is called in each drafting incident, and the more important thing is that it can not easily detect when the user browsed vision area away from assembly.The method that another kind is used for incident is sent to application is notification logic to be incorporated into GUI browse element (for example hyperlink and button), thereby make to change the camera vision area so that the interested assembly of institute is included, notice is sent to said modules at them.Yet this needs the programmer to come watchfully (viglantly) that information code is increased in all possible UI of the browsing element.

According to an exemplary, can use counting yield node monitoring algorithms to notify and use the GUI assembly and/or be applied in when enter and withdraw from the camera vision area.On high grade, this node monitoring algorithms has three main levels of handling: (1) initialization, (2) vision area changes assessment, and (3) scene graph changes assessment.Initialisation stage is calculated by vision area and is changed the node amount that evaluation stage is used, and the correct data structure of initialization.Vision area changes evaluation stage and is called when vision area changes, and notifies all to enter or withdraw from the monitor node of vision area.At last, scene graph changes evaluation stage and is updated in the calculating that initial level produces, and these calculate owing to the variation in scene graph becomes invalid.For example, if the ancestor node of monitor node (ancestor node) has changed the position in scene graph, then the calculating of carrying out in initial level need be recomputated.

In these levels, vision area changes the remainder of evaluation stage drive node monitoring algorithms.Enter and withdraw from vision area when in order to describe node, initialization step is determined the border rectangle of expectation node, and it is converted to the local coordinate system of vision area from its local coordinate system.In this manner, detection node does not enter and need carry out a series of coordinate conversions in each vision area change place.Because the father node of this node can have transition matrix, so this initial step need travel through scene graph up to camera node from this node.As described below, if in the scene graph data structure, use the camera node that embeds, then need a plurality of borders rectangle to be contained in the node that a plurality of positions occur.

Calculate the border rectangle that is used for each monitor node in the vision area coordinate system after, initialisation stage is increased to vision area with boundary matrix and changes in the assessment data structure.Above-mentioned node monitoring algorithms uses basic building block to be used for each dimension of screen.In the scalable interface according to some exemplary, this comprises x dimension, y dimension and ratio dimension.Yet as described below, other exemplary can have additional or different dimensions.The ratio dimension has been described the amplification stage at the node of vision area, and describes by following formula:

$s=\frac{d^{\′}}{d}$

Wherein, s is a ratio, and d is the distance of another point from a point of this node to the local coordinate of this node, and d ' be in this vision area above-mentioned point to the distance of another point.

Figure 11 has shown the exemplary elements piece that the scene that is used to detect a dimension enters and withdraws from.Processing in the x dimension is described below, but those skilled in the art should be realized that the dimension that can handle other in a similar manner.Region unit 1100 comprises the benchmark (references) that is converted the border rectangular coordinates.This comprises a left side and the right side (top and bottom or minimum and the maximum ratio) side-play amount of rectangle.Left and right side-play amount is stored in respectively in conversion block 1102 and 1104, conversion block 1102 and 1104 ordered lists that place two strands to connect itself, thus make low side-play amount of numbering towards reference position.Current vision area border is stored in sight line border block 1106.Block 1106 has the pointer that sensing just exceeds the conversion block on the left side of view and right side, and for example, the conversion block that is located immediately at the right side of the block that is pointed to by the view left side appears in the view, unless the above-mentioned block that is pointed to by the view left side is pointed to by the vision area right side.

When vision area changes, for each dimension following processing takes place.At first, detect view left side and view right side pointer to confirm whether they need to move to comprise or to get rid of conversion block.Next, if one or two in the above-mentioned pointer need move, then they skip over (slide over) above-mentioned conversion block to arrive their new positions.Then, for each conversion block by left side and view right side pointer process, above-mentioned node monitoring algorithms is carried out below with the conversion block notice coding of describing.This notice coding is determined the vision area whether its node separately might enter or withdraw from.If have, then this node is added in the aftertreatment tabulation (post processing list).At last, in the ending of this processing that is used for each dimension, whether its vision area state of each node that detects in the above-mentioned aftertreatment tabulation change (becoming again opposite then with change earlier) has taken place truly.If change has really taken place, then above-mentioned algorithm transmission incident is to said modules.Should be noted that if vision area jumps to the new region at scalable interface fast, then above-mentioned algorithm can detect more puppets and enter and withdraw from incident.

Conversion block notice coding can be realized as look-up table, and whether this look-up table determines to move in and out for the node of this dimension detected.Shown exemplary above-mentioned table below.

The node side	The view side	The view moving direction	The local notice that intersects	Intersect notice fully
					A left side	A left side	A left side	Do not have	Enter
A left side	A left side	Right	Do not have	Withdraw from
					Right	A left side	A left side	Enter	Do not have
Right	A left side	Right	Withdraw from	Do not have
					A left side	Right	A left side	Withdraw from	Do not have
A left side	Right	Right	Enter	Do not have
					Right	Right	A left side	Do not have	Withdraw from
Right	Right	Right	Do not have	Enter

Table 1-conversion notification list

1st, 2 and 3 hurdles are inputs of conversion notification list.Whether specifically, the node monitoring algorithms uses the combination of node side, view side and view moving direction to address this table, entered, withdraw from or not influence with the node of determining this assessment.The node side that conversion block characterized that hurdle 1 refers to by vision area pointer process.Hurdle 2 refers to the vision area side, and hurdle 3 refers to the direction of this vision area side shifting when vision area is passed through the conversion block of node.But whether Output bar 4 or 5 should notifiedly be selected according to this node in above-mentioned node section or complete apparent time.For example, in some implementations, ideal situation is only just to notify this application after visual fully using (for example video flowing window), and this is that to be loaded into because of the video window that part is visual visually be fissile among the scalable GUI.

When the output of this table was represented to enter and withdrawed from, this node monitoring algorithms was added this node to the aftertreatment tabulation.Output bar in the table 1 is filled (populate) based on following rule.If this node intersects in all dimensions, then will enter notice and send to above-mentioned post-processing step.If this node in view and present one or more dimension stop to intersect, then will send OPT-OUT NOTICE.In order to reduce the node number in the aftertreatment tabulation, the conversion block notice is coded in this node added to and detects before the tabulation and the intersecting of other dimension.When only having one or two dimension to intersect in this dimension (for example, three or more intersections), removed post-processing step from all numbers.When user interface object (for example, use) was wanted notified its vision area state in GUI, it registered a function (function) with above-mentioned node monitoring algorithms.When above-mentioned application entered or withdraws from view, the node monitoring algorithms was called the function that this application is registered with the parameter of indication what's going on.As a kind of selection, can use the message transmission to come exercise notice.In this case, each application has event queue.Above-mentioned application tells how the node monitoring algorithms communicates by letter with its event queue.For example, it can point out the address of formation.Then, when the node supervision detected conversion, its was set up and comprises the data structure of notifying cause, and is placed in the formation of this application.

Except use is used for the node monitor for notifications that application memory is managed, this algorithm can also be used for other function according to scalable GUI of the present invention.For example, the node monitoring algorithms can be used for changing the application behavior based on the focus of User, for example by the audio frequency input focus is switched in the current visual application.What other was used for the node monitoring algorithms is applied in amplification stage when changing, and will be written into or unload higher resolution, and image is synthesized.This draws object less, the approaching more coupling display of resolution by making drawing implementer (renderer), has reduced its computational load.Except the vision area of node monitoring algorithms monitoring camera, make it monitor that it also is useful browsing coding (navigation code), this is browsed coding and tells the vision area position that this vision area finished after the animation manufacturing.This earlier notification component they will appear in the view, and can make scalable according to embodiments of the present invention GUI avoid notice sent to because mobile animation and by the node that overflowed.

In order to understand the operation of nodal operation monitoring algorithms better, with reference to Figure 12 (a), 12 (b), 13 (a) and 13 (b) embodiment is described below.Figure 12 (a) and 12 (b) show the part of graphical user interface two different amplification stages, scalable.Low amplification stage place at Figure 12 (a) can see three nodes: circle, triangle and ellipse.In Figure 12 (b), this view is exaggerated, thereby makes ellipse and circular portion as seen, and triangle is invisible fully.These nodes can for example characterize application or the user interface components that depends on the validity event notice, and are therefore followed the tracks of by the node monitoring algorithms according to illustrative embodiments of the invention.In this embodiment, the border rectangle in that Figure 12 (a) and 12 (b) clearly show each node typically can not show these border rectangles although those skilled in the art should be realized that in GUI.Each limit to each border rectangle in 12 (a) and 12 (b) marks, and these marks will be used for showing the limit of border rectangle and the corresponding relation between the above-described conversion block data structure.

Figure 13 (a) shows the exemplary nodes monitoring data structure of the horizontal dimensions of dwindling view that is used for figure (12).In the figure, each limit of node boundary rectangle uses conversion block to characterize.The order from left to right that the level conversion piece occurs in GUI with them is shown among Figure 13 (a).For example, Yuan Xing left side C _{A left side}At first occur, follow leg-of-mutton left side T _{A left side}, or the like, up to the right of ellipse E _RightTill the appearance.Mark is come with empty sign conversion block in the two ends of this tabulation.In Figure 13 (a), also shown and be used for each node and point to them accordingly to the region unit of the pointer of the level conversion piece of their border rectangles.In the bottom of Figure 13 (a) is view data boundary structure, and it comprises the pointer that sensing just is positioned at the conversion block outside front view.For this view that dwindles, all nodes are visible fully, and therefore their all conversion blocks between the conversion block that points to by view data boundary structure.

Figure 13 (b) has shown the node monitoring data structure of the zoomed-in view that is used for Figure 12 (b).In the figure, the view boundary member of data structure is changed as can be seen, is used for leg-of-mutton the right T thereby make it point to _RightAnd oval the right E _RightConversion block, this be because above-mentioned two border rectangular edges just in time be positioned at outside current (amplification) view.

After providing above-mentioned example data structure and GUI scene, the relevant treatment when carrying out scale conversion in the node monitoring algorithms is as described below.At first from the left side of view, the node monitoring algorithms moves right view left side pointer, up to arriving the conversion block that just is in outside the left side view.Shown in Figure 13 (b), view left side pointer at first passes through C _{A left side}Conversion block.For example, suppose that circular node characterizes the application related with scalable GUI or other user interface object (but in its apparent time in view fully, needing to notify).After being given to the above-mentioned input of node monitoring algorithms, table 1 illustrates this circular node should receive OPT-OUT NOTICE for horizontal dimensions.Certainly, the node monitoring algorithms can typically be gathered the notice from all dimensions before notifying this node, to avoid sending unnecessary OPT-OUT NOTICE.Next, view left side pointer is by leg-of-mutton left side T _{A left side}If this triangular nodes is asked notice when it leaves view fully, then this node monitoring algorithms goes out not at each table 1 middle finger that notice is necessary.Yet, pass through T at the view pointer _RightThe time, table 1 illustrates this triangle and withdraws from view fully, and should be notified.The view pointer stops at here, here because the right C of circular boundary rectangle _RightIn view still as seen.

The node monitoring algorithms is similar from the processing on this right side.Attempt the right side pointer and move left to oval the right E _RightWhether ask all or part of notice according to ellipse, this node monitoring algorithms meeting or can be according to the oval notice that sends of table 1 pair.Can use similar data structure and top and bottom border rectangle value to handle vertical dimensions in a similar manner.Those skilled in the art it should further be appreciated that a plurality of borders rectangle can be used for approaching the node of non-rectangle when the more accurate notice of needs.In addition, motion by other dimension be followed the tracks of and be handled to the present invention imagination can by the node monitoring algorithms, for example, the 3rd how much (degree of depth or ratio) dimensions, and non-how much dimensions, for example time, Content Ratings (adult, PG-13 etc.) and content type (drama, comedy etc.).According to the number of the dimension of using, this algorithm is the intersection of detection boundaries section, rectangle and n dimension hypercube more accurately.

Except above-described node monitoring algorithms, exemplary of the present invention provides the resolution that can use in the scalable GUI according to illustrative embodiments of the invention consistent semantic convergent-divergent algorithm.Semantic convergent-divergent is meant the details that according to the amplification stage of assembly the assembly among the scalable GUI is increased, removes or changes assembly.For example, in the film browser interface that is described below, when the user with camera lens during enough near the image of film, this browser changes to show film metadata and playback controls.The calculating of this amplification stage is based on number of pixels that assembly uses in display apparatus.This scalable GUI can store the thresholding amplification stage, the indication of this thresholding amplification stage will when switch with for example never the view of film metadata and playback controls switch to view with film metadata and playback controls.

TV and graphoscope have the very big monitor resolution of variation range.Some monitors have sufficiently high resolution, thereby to such an extent as to make and can be presented on the picture in the lower resolution displays and text is too little can not read fully.This also for use semantic convergent-divergent, especially (for example, the application in HDTV) has problems at high resolution display.In this case, the semantic convergent-divergent sign indicating number of drawing based on the number of pixels that shows will change image before can reading more detailed view.Revise semantic convergent-divergent procedurally and only can play effect a resolution at the thresholding that its place changes assembly.

Ideal results is that all monitor resolution are all carried out consistent semantic convergent-divergent.A kind of solution is to use the display mode of low resolution in high-resolution monitor, so that the resolution in all displays is consistent.Yet if semantic convergent-divergent still carries out as expectation, high-resolution monitor user is preferred coming rendered images with the resolution of they the bests.Therefore, illustrative embodiments of the invention provide a kind of and support described different resolution to show and do not exist aforementioned semanteme to watch the semantic zoom technology of problem.This can be for example by realizing at the built-in upright virtual monitor of scene.Thereby in Figure 14, show by using embedded virtual camera node 1200 and increase logic to compensate monitor resolution.This virtual camera node 1200 has defined the vision area that size is mapped to user's line-of-sight distance and monitor dimension.For example, big virtual camera vision area is represented the user or is before very near-earth is sitting in monitor, or having enough big monitor offers an explanation many details.Alternatively, little vision area is represented the user away from monitor, and needs big font and image.This scalable GUI coding can be based on to be seen in the above-mentioned virtual camera, and uses the semantic scale conversion in the amplification stage of assembly of user best viewed status.

The principal phase machine node 1202 that is attached to display apparatus 1204 has by configuration so that it shows the vision area of the various things that virtual camera 1200 is showing.Because icon and text were not mapped to pixel before this principal phase machine 1202,, quality do not descend so can not taking place from virtual camera.This result is that the monitor display of pinpoint accuracy shows high-quality image, and can not trigger the semantic convergent-divergent change that makes demonstration be difficult to read.

According to exemplary of the present invention, above-mentioned processing is achieved as follows.Each camera in scene graph has relevant transition matrix (T with node _iTo T _n).These matrixes are converted to local coordinate system at the next node of display with the local coordinate system of node.In the drawings, T ₁From its vision area with coordinate conversion to the display coordinate.Equally, T ₂Its local coordinate system is transformed into the vision area of camera.If leaf node 1206 need be drawn some contents in display, the transition matrix below then it calculates:

A＝T ₁T ₂…T _n

This calculating can be performed by scene graph the time.Support semantic convergent-divergent to be based on virtual camera 1200 owing to change assembly, so the calculating below carrying out:

B＝T ₄T ₅…T _n

Usually can determine T in advance with checking scene graph by the resolution of inquiry monitor ₁To T ₃Therefore, from A determine B by following formula to these matrix inversions and the realization of multiplying each other:

B＝(T ₁T ₂T ₃) ^-1A

For calculated in advance T ₁To T ₃The time in-problem this situation, for example,, logic can be increased to virtual camera if figure API has hidden additional conversion, thus interception (intercept) it will be used for being plotted to the transition matrix of display.These transition matrixes that are blocked are then inverted and multiply each other (as mentioned above), with computing semantic convergent-divergent thresholding.

According to the favorable characteristics at the scalable interface of exemplary of the present invention be it have can be when the navigation interface ability of maintaining context.All interface assemblies appear to be present in the scalable world, and the user only needs to move and convergent-divergent reaches arbitrarily them.Above-mentioned semantic zoom technology changes the outward appearance of said modules according to convergent-divergent or amplification stage.Figure 15 (a) and 15 (b) provide the embodiment of the semantic convergent-divergent that is used for assembly, and wherein the version that dwindles of this assembly (Figure 15 (a)) is a picture, and amplified version (Figure 15 (b)) comprises identical picture and some control and details.These more more detailed embodiment are provided below.A challenge relevant with semantic convergent-divergent is can change suddenly between view, and can visually not provide satisfied result when for example the α switch technology of mixing is changed between two above-mentioned views.

Therefore, exemplary of the present invention provides the image and the text of some public (common) in all component view, to provide focus to transitions when carrying out semantic convergent-divergent.For example, in Figure 15 (a) and 15 (b), common elements is a picture.Transitions between amplified version and down-scaled version can for example use above-mentioned node monitoring algorithms to trigger in the following manner.At first,, monitor with above-mentioned node and to carry out registration when the down-scaled version of assembly is transformed into amplified version in the vision area of principal phase machine with the reception incident.Then, when incident takes place, can show that displaying is from their down-scaled version position convergent-divergents and the animation that is transformed into the common elements of their amplified version positions.Simultaneously, the camera vision area continues to be advanced to said modules.

After browsing other exemplary of describing below with reference to Figure 16-20, will become more apparent according to these abilities of graphic user interface of the present invention.In Figure 16-20, open beginning GUI screen 1400 and show a plurality of organization objects of operating as media groups.The media groups that is exemplary home theater, film, TV, physical culture, radio, music and music fully can comprise different, more or less media groups certainly.When of starting by the user in these icons, will show a plurality of images according to the GUI of this exemplary of the present invention, each in these a plurality of images is grouped into special classification and kind.For example, if the user starts " film " icon among Figure 16, then can show the GUI screen of Figure 17.In Figure 17, shown a large amount of (for example 120 or more) alternatives.These alternatives can be classified as special group, for example, and action, allusion, comedy, drama, domestic play and new issue film.Those of ordinary skill in the art should be realized that the kind that can provide more or less.In this exemplary embodiment, the image of media item can be to select related feature front cover with each film.Can not allow to show the detailed description of many groups relatively of this options image although the block in Figure 17 is too little, but in application, the amplification stage of these images can make the identity by image identifying these films related with film, even if to such an extent as to some or all in these texts can be too little can not easily the readings.

Can place on one group of cine-oriented image with vernier (not shown in Figure 17) and drive input equipment and select indication for of this group provides.In this exemplary embodiment, the user selects the drama group, and then graphical interface of user shows the zoom version in the drama image sets, as shown in figure 18.The same with previous embodiments, when the GUI screen of GUI from Figure 17 switches to GUI screen among Figure 18, also can show transitions, for example, before the convergent-divergent or in the middle of the convergent-divergent process, this GUI can move to the center of the GUI screen of view from Figure 17 the center of drama group image.Should be noted that although the zoom version of the drama group of Figure 18 has only shown the subclass of all images in the drama group, this zoom version can comprise all images in selected group alternatively.Show in the amplified version of arbitrarily given GUI screen or do not show that all images in selected group can be based on the number of (for example) media item in groups, and the minimum expectation amplification stage that is used for the media item of special zoom level carries out.Back feature according to GUI of the present invention can pre-determine by system planner/service provider, or customizes via the software setting among the GUI by the user.For example, can be by one among service provider and the terminal user and two number and minimum and/or maximum amplification stage that disposes media item in groups.Above-mentioned feature makes those users that for example have bad eyesight can increase the amplification stage of the media item that is shown.On the contrary, the extraordinary user of eyesight can reduce amplification stage, and is increased in any given incident and is presented at media item number in the GUI screen, and reduces the browsing time.

An exemplary transitions that can use in graphic user interface according to the present invention is called as " footwear are to details (shoe-to-detail) " view effect.When being activated, this transitions obtains image that dwindles and the image that simultaneously this is dwindled contraction and is transformed into littler view, that is, and and the convergent-divergent that next is more high-grade.The amplification stage that uses from the GUI screen among Figure 17 is transformed into the bigger amplification stage that uses in the GUI screen in Figure 18, and this causes GUI is that the image that is shown in the amplified version of Figure 18 discloses out additional details.Whether this GUI is shown in current selecteed zoom level well according to these details, comes the optionally details of the various zoom level of show or hide.No matter they are different for the visual capabilities camera convergent-divergent how of naked eyes with attempting to offer an explanation details, exemplary of the present invention provides configurable zoom level parameter, and these parameters have been stipulated to show all images when and when shown transfer point between the image version with the details of being arrested.This transfer point can be based on the inside resolution that is independent of iamge description, rather than based on the resolution of TV/ monitor 212.Utilize this mode, GUI according to the present invention is consistent, no matter and how little the resolution of the display device that uses in media system be.

In this exemplary embodiment, can be by being provided for the amplification of the additional amount of particular image on the vernier process particular image.This feature shows that in Figure 19 wherein, vernier is at film Apollo Personnel investigation Program 13 " image on roll.Although it is not shown in Figure 19, compare with the respective image that is in low amplification stage in the GUI screen among Figure 18, above-mentioned additional amplification for example can make the quoted passage " Houston, we have run into trouble " in the feature front cover of realizing with the related media item become clear more readable.The user is for the selection of this image, and for example by pressing the button on input equipment, further convergent-divergent is to show the details that shows among Figure 20.This also provides the embodiment of another foregoing semantic convergent-divergent, and this is to be displayed in the GUI screen of Figure 20 because of sightless various information in the GUI of Figure 19 screen and control element.For example, about film Apollo Personnel investigation Program 13 " the information that mainly comprises film projection time, price and performer be shown.The information that those skilled in the art will appreciate that other type can here provide.This GUI additionally comprises the GUI controlling object, and GUI control is used for buying this film to for example comprising, watch movie trailer or turn back to the button control object of last GUI screen (it can also be pressed large buttons at input equipment and realize).Can also use hyperlink to allow the user for example to jump to and the related GUI screen of associated film (at place, the lower right corner of the GUI of Figure 20 screen sign), or with this film in the relevant information of performer.In this embodiment, some or all movie name under title " film catalogue (Filmography) " can be embodied as hyperlink, when the user starts by input equipment, these links will make that GUI is that the film that is instructed to shows and the corresponding GUI screen of GUI screen among Figure 20.

When starting hyperlink, the user can also use transitions.Because hyperlink can produce by high amplification stage, therefore, jump to the media item that is linked simply and can make the user lose he or she selects " mapping graph " to media item tracking.Correspondingly, exemplary of the present invention provides transitions with when starting hyperlink, helps to keep user's the rational topognosia in ground.An exemplary transitions that uses is the conversion of jumping for this reason.In the starting stage of this transitions, GUI dwindles and the direction of the item that points to hyperlink moves.Dwindle and moved before purpose image and original image are all seen by the user and continue to carry out.Reuse the embodiment of Figure 20, if the user selects hyperlink to be used for " Saving Private Ryan ", then the phase one of the jump transitions of this hyperlink will comprise and dwindle and move towards image " Saving Private Ryan ", up to image that is used for " Saving Private Ryan " and Apollo Personnel investigation Program 13 " be visual position for the user.At this, this transitions brings the eye impressions that move up towards the purpose image with arc to the user.In case target image appears in the view, the subordinate phase of transitions is brought to the user and is amplified and for example move to the eye impressions of purpose image with second half of above-mentioned arc.Hopping time promptly shows the time quantum of phase one and subordinate phase, can be to fix between any two hyperlink image term.Alternatively, hopping time can change, for example based on the distance of advancing on GUI.For example, hopping time can be turned to by parameter: hopping time (HopTime)=A log (magnification ratio grade/jump summit ratio grade)+B (distance between the hyperlink media item)+C, wherein, A, B and C are chosen as steady state value suitably.

The node monitoring algorithms of describing with reference to Fig. 9-13 (b) can also be with changing between the zoom level of helping describe in the exemplary GUI screen of the exemplary GUI screen of Figure 19 and Figure 20 above.Compare with the image of low convergent-divergent grade, comprise the stronger calculating and/or the storage of drafting needs of the GUI screen of text and/or control element (they are invisible in other zoom level of selecteed image).Correspondingly, this node monitoring algorithms can be used in exemplary of the present invention, with the navigation nodes by monitoring GUI discerning scaled media item more apace, thereby help to be written in advance GUI screen, the GUI screen that for example in Figure 20, shows.

Be comprised in the exemplary realization of the present invention is screen position and based on the Navigation Control of semanteme.At vernier during near customer location, perhaps vernier be in the zone related with these controls in the screen (these controls therein are correct, as shown in figure 21) in the time, these control areas appearance.For example, when playing film, F.F., these so-called skill (trick) functions such as to retreat, suspend, stop be suitable semantically.In this exemplary embodiment, be assigned with screen area with these functions, and near vernier is positioned at these zones or when being positioned at wherein, be used for the icon sets demonstration of these skill functions in the lower right corner.These icons finish or disappear when vernier is arranged in other position of screen again in the definite functions that is used.Can also use same technology to cover other navigation characteristic of selecting as text search and indoor screen.In this exemplary realization, these are controlled on all screens is semantic relevant, and joins with their zone and be positioned at the upper right corner.Vernier be positioned at these zones or these regional near the time, be used for the icon sets demonstration of these Navigation Control.These icons are activated or vernier disappears when being positioned at other position of screen again in function.Should be noted that, be in the purpose of user training, related control icon can show temporarily alternatively initially that (for example 5 seconds) in some or all of screen associated, notice existing of they will cause unskilled user.

Behind some embodiment that provide according to scalable graphic user interface of the present invention, exemplary architecture and the structure (infrastructure) of using above-mentioned interface described now.Figure 22 provides Organization Chart, wherein, is played the part of by primitive (primitive) 1902 (being called as " atom " in the drawings) with the scalable interface that various senior application 1900 (for example, film, TV, radio and physical culture) are related.In this exemplary embodiment, primitive 1902 comprises location, click, convergent-divergent, hover and rolling, and those skilled in the art should be realized that other primitive also can be included in this group certainly, for example, moves and pulls.As mentioned above, the position of vernier is determined in location and the operation of clicking primitive, and when for example user starts the amplification in the handheld input device or dwindles button trigger event.These primitives have been simplified navigation and have been removed the needs that are used for repetition upper and lower, left and right button.As mentioned above, the convergent-divergent primitive is used for scanning possible selection, and gives user's context when the user dwindles his or her selection.This design makes this interface select and show that arbitrarily size stipulates according to a large amount of medium.The rolling primitive is handled the input from the scroll wheel input equipment in the exemplary handheld input device, and can be used for for example quickening linear menu navigation.Hover button primitive dynamically amplifies the selection (and/or changing the content of selecting) below the pointer, so that the user can browse potential selection and need not actual execution.Can in GUI according to the present invention, start the operation of aforementioned each primitive in a large amount of different modes.For example, location, click, hover, hover and convergent-divergent can be related with the different gestures that the user can carry out.These postures can be passed to system via input equipment (no matter whether it is free space pointer, tracking ball, touch pads etc.), and are converted into the startup of suitable primitive.Equally, each primitive can be related with voice command separately.

Have various software and hardware structures 1904 between rudimentary primitive 1902 and senior application 1900, they are used for producing and the scalable related image of GUI according to the present invention.As seeing among Figure 22, this structure 1904 can comprise handheld input device/pointer, application programming interfaces (API), scalable GUI screen, developing instrument etc.

The exemplary of front is fully descriptive in itself.Being provided to user's zoom level number and special information and control with each grade changes.Those skilled in the art should be realized that the invention provides revolutionary technology is used to use scalable interface to present the media item of big and group, thereby make that the user can easily search for, browses, tissue and playback media item, for example film and music.Graphic user interface according to the present invention organizing media item in virtual surface is selected, thereby makes similar selection be combined in together.Begin most, the view that dwindles on above-mentioned surface is transmitted at this interface, and under most of situation, is to cannot see actual selection in this grade, and only sees their group name.Along with the inside gradually convergent-divergent of user, show more details about groups of media items or selection.Can obtain different control in each zoom level, thereby make the user can play the group or the single selection of selection or arrive virtual other parts to browse other relevant media item.Can comprise the image kind that is nested into any degree of depth and the kind of kind according to the convergent-divergent graphic user interface of illustrative embodiments of the invention.Media item can comprise the broadcasting of content, the broadcasting provider of local storage, via from the direct connection of content supplier or the information that on reciprocity basis, receives.Media item can provide with the form of schedule, and wherein, date information provides at the GUI of some grades.In addition, framework and the GUI according to illustrative embodiments of the invention can also be applied to the television commerce that the item that will be used for selecting is therein sold the user.

It is to be used for describing rather than restriction the present invention that above-mentioned exemplary embodiment is tending towards in all respects.Therefore, the present invention can have many mutation in concrete realization, and those skilled in the art can obtain these mutation by the description that is included in herein.These all mutation and modification are counted as and fall in the present invention's scope and spirit that claim defines below.Only offering some clarification on, is key or necessary otherwise the element that uses in the application's instructions, action or instruction can not be interpreted as the present invention.And as using in this article, the intention of article " (a) " has comprised one or more.

Claims (9)

1. control architecture that is used to organize, select and start a plurality of media item comprises:

Be used to organize the device of described a plurality of media item, described a plurality of media item are represented by a plurality of different images;

Be used to navigate to the device of one of described a plurality of media item, one of described a plurality of media item are represented by one of described a plurality of different images;

Be used to select one of described a plurality of media item to be used for different semantic grade device shown; And

Be used at the device that is converted to described different semantic grade by the current semantic grade that will show one of described a plurality of media item when carrying out following operation, the described size that changes one of described a plurality of different images that is operating as, and its primary importance from described current semantic grade is transferred to the second place of described different semantic grade.

2. control architecture as claimed in claim 1, wherein, the described device that is used to navigate to one of described a plurality of media item comprises free space pointing devices.

3. control architecture as claimed in claim 1, wherein, the described device that is used to navigate to one of described a plurality of media item comprises tracking ball.

4. control architecture as claimed in claim 1, wherein, the described device that is used to navigate to one of described a plurality of media item comprises voice recognition unit.

5. control architecture as claimed in claim 1, wherein, the described device that is used to navigate to one of described a plurality of media item comprises the gesture recognition unit.

6. control architecture as claimed in claim 1 wherein, describedly is used to select the device of one of a plurality of different semantic grades also to comprise:

Display unit is used to show the object that comprises the alternative related with described a plurality of media item; And

Scalable graphic user interface is used for generating described alternative with described a plurality of different semantic grades.

7. control architecture as claimed in claim 6, wherein, first group of predetermined information object is visual on the first semantic grade, second group of predetermined information object is visual on the second semantic grade.

8. control architecture as claimed in claim 7, wherein, it is disabled selecting option to select speech for the user on the described first semantic grade, the above selects option for user's selection and Yan Shiyong in the described second semantic grade.

9. control architecture as claimed in claim 1, wherein, the described device that is used to organize also comprises:

Set of diagrams mark, its each all related with specific medium kind, wherein, select in the described icon each to cause a plurality of options in each medium kind to be revealed.

Images