CA2161423A1 - Interactive multimedia delivery engine - Google Patents
Interactive multimedia delivery engineInfo
- Publication number
- CA2161423A1 CA2161423A1 CA002161423A CA2161423A CA2161423A1 CA 2161423 A1 CA2161423 A1 CA 2161423A1 CA 002161423 A CA002161423 A CA 002161423A CA 2161423 A CA2161423 A CA 2161423A CA 2161423 A1 CA2161423 A1 CA 2161423A1
- Authority
- CA
- Canada
- Prior art keywords
- panel
- objects
- topic
- flow control
- displayed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 17
- 238000000034 method Methods 0.000 claims description 28
- 238000011156 evaluation Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 2
- 239000000945 filler Substances 0.000 description 41
- 230000009471 action Effects 0.000 description 12
- 230000000007 visual effect Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 9
- 238000013515 script Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 150000002500 ions Chemical class 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 241000256844 Apis mellifera Species 0.000 description 1
- 241000726103 Atta Species 0.000 description 1
- 101100202428 Neopyropia yezoensis atps gene Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- YWXYYJSYQOXTPL-SLPGGIOYSA-N isosorbide mononitrate Chemical compound [O-][N+](=O)O[C@@H]1CO[C@@H]2[C@@H](O)CO[C@@H]21 YWXYYJSYQOXTPL-SLPGGIOYSA-N 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000004513 sizing Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/74—Browsing; Visualisation therefor
- G06F16/748—Hypervideo
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/93—Document management systems
- G06F16/94—Hypermedia
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S345/00—Computer graphics processing and selective visual display systems
- Y10S345/949—Animation processing method
- Y10S345/96—Iterative display of preconfigured images
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99944—Object-oriented database structure
- Y10S707/99945—Object-oriented database structure processing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99948—Application of database or data structure, e.g. distributed, multimedia, or image
Abstract
An engine for the interactive delivery and presentation of multimedia data. The delivery mechanism is premised on a topic and panel metaphor. A topic is comprised of a series of panels. Each panel further defines the multimedia data which will be used when the panel is displayed. Each panel may also contain control and qualifier elements which are used to determine the next panel to be displayed. Control elements are used to solicit information from the user. The information is then used by the qualifier elements in determining the next panel to be displayed. Each element is stored in a database as a discrete object. This allows sharing of objects amongst different panels or topics.
Prior to delivery and presentation of the data, the topic must be set up. Setup occurs by retrieving all the necessary panel, control, qualifier and data descriptor objects. The actual data associated with multimedia data objects is only retrieved when the data is to be displayed. The various qualifier objects are evaluated while panels are being viewed to determine the next panel to be displayed.
Prior to delivery and presentation of the data, the topic must be set up. Setup occurs by retrieving all the necessary panel, control, qualifier and data descriptor objects. The actual data associated with multimedia data objects is only retrieved when the data is to be displayed. The various qualifier objects are evaluated while panels are being viewed to determine the next panel to be displayed.
Description
~WO 94127237 216 1~ 2 3 PCT/US94/04991 INTERACTIVE MULTIMEDIA DELIVERY ENGlNE
A portion of the disclosure of this patent docllment contains material which is subject to copyright protection. The copyright owner has no objection to the f~ imile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Tr~ mArk patent file or records, but otherwise reserves all copyright rights whatsoever.
BACKGROllND OF THE INVENTION
1. Field of the Invention The present invention relates to the field of storage and retrieval of data, inparticular, to organizing and storing multimediA data for use in interactive data delivery and display systems such as those used for instrllctionAl or educ~honAl purposes.
A portion of the disclosure of this patent docllment contains material which is subject to copyright protection. The copyright owner has no objection to the f~ imile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Tr~ mArk patent file or records, but otherwise reserves all copyright rights whatsoever.
BACKGROllND OF THE INVENTION
1. Field of the Invention The present invention relates to the field of storage and retrieval of data, inparticular, to organizing and storing multimediA data for use in interactive data delivery and display systems such as those used for instrllctionAl or educ~honAl purposes.
2. Description of the Related Art As use of computer systems become more widespread, the desire to utilize n llltime~liA has become more imme~iAte. Multimedia refers to the integrated use of text, graphics, video and audio information media's. In order to effectively use multimedia, mer~Anicmc for delivering the data to the user must be developed. For example, in order to create an interactive instructional application, various screens or panels are created which contain the instrllctionAl information. The panels must be logically linked and displayed in an order corresponding to the state of the interactive instruction.
Two known systems provide interactive mllltimerliA delivery capabilities; MacroMe~ Director and HyperCard~) MacroMedia Director, a product of MacroMedia Inc., use a score, stage, and cast metaphor to deliver interactive mllltimerliA A score (timeline) is used to place cast members (mllltime-liA objects) on the stage (display). Objects can be layered visually by their placement along the vertical axis of the score and temporally by their plAcement along the hori7ontAl axis. What the user sees at any point in time is repr~s~nterl by a column in the score. The flow of delivery (i.e.
instruction) is controlled by the score. Scripts can be AttA~e~l to cast members and the score to alter the flow of delivery. Scripts require SUBSTITUTE SHEET (RULE 26) wo 94/27237 PcT/Uss4lo49sl ~
2 ~ 2 ~ 2 pror~ssing external to the flow of delivery. Further, subsequent alterations to the flow are cumbersome.
HyperCard, a product of Apple~ Computer, Inc. uses a notecard and object metaphor to deliver interactive multimP~ instruction. Visual objects are placed on cards. The cards are then ordered. The flow of delivery is controlled by the card order. Scripts can be ~ rh~l to visual objects and cards to alter the flow of instruction. A HyperCard card is equivalent to a column in the MacromP-liA Director score. A HyperCard object is equivalent to a cast member in MacroM~ Director. Both e1P~n~nt~ in both products can have scripts ~tt~r~te~l The prior art systems have various ~lPfirir-nri~s. First, the flow of data delivery may only be altered via scripting. It would be desirable to be able to alter the flow of instruction inherently within the design of the instruction. Further, there is no backtracking once a flow of instruction has commenced.
Accordingly, it would be desirable to have an interactive ml~ltime~
delivery system which o~ comes the foregoing ~efirir-ncies of the prior art.
SUBSTITUTE S~EET (RU~ E 26) ~10 94127237 2 ~ 6 14 2 3 PCT/US94/04991 SUMMARY
A delivery engine for interactive presentation and display of mllltimP~iA data is disclosed. The delivery me~ h~ni~m is prPmi~e~1 on a topic and panel metaphor. A topic is comprised of panels. Each panel ~l~fin~c the mllltimecli~ data which will be used when the panel is displayed. Each panel may also contain control and qllAlifiPr PlPmPnt~
which are used to determine the next panel to be displayed. Control PlPmPnt~ are used to solicit inform~tion from the user which is then used by the qualifier PlPmpnts in determining the next panel to be displayed.
Each PlPmPnt is stored in a database as a discrete object. This allows re-use of objects ~mon~st different panels or topics. Prior to delivery of the data, the topic must be set-up. Set-up occurs by retrieving all the necessary panel, control qll~lifier and data descriptor objects. The actual data ~csori~te~l with data media objects is only retrieved when the data is to be displayed. The various qllAlifiPr objects are evaluated while panels are being viewed.
The ~l~t~h~ce is generated via an authoring step. In the authoring step, the topic and the various elPmPntc are combined to create a flow for the ~,æs~ hon of the mllltime~ data. Once the authoring is completed, the ~ TnPnt~ are processed in a manner similar to compiling a software program, to create the rl~t~h~ce SUBSTi~TE ~H~ET ~RU~ 2~) wo 94/27237 PcT/uss4/0499l ~
~1~142~ 4 BRIEF DESCRIPIION OF THE FIGURE
Figure 1 is a block diagram illustrating the components of a computer system as may be llhli7e~ in the currently yref~.~ed embo~ Pnt of the present invention.
Figure 2 is a block diagram illustrating the components of the delivery engine of the currently yrere~led embo~lim~nt of the present invention.
Figure 3 is a diagram which illustrates the logical relAticI ~hip Amongct the database objects of the currently yrefelled embo~liment of the present invention.
Figure 4 illustrates a PresentAti-~n Window as may be lltili7e~1 in the currently y~ef~lled embo~limPnt of the present invention "~)1993 Apple Computer, Inc." (17 U.S.C. 401).
Figure 5a is a block diagram of a database as generated in the currently yrer~:lled emboflimPnt of the present invention.
Figure 5b illustrates the linkage between a topic and cc lresyo~ ing panels of the rlAtAbAse of Figure 5a.
Figure 5c illustrates the linkage between a panel and cclle:jyon.1ing visual objects and qualifiers of the ~lAtAhAce of Figure 5a.
Figure 6 is a flowchart which illustrates the steps retrieving objects for setting up a topic for delivery in the currently yref~:lled embo~isn.ont of the present invention.
Figure 7 is a flowchart which illustrates the steps retrieving objects for delivery of panels in the currently preferred embodiment of the present invention.
SlJBSTITUTE SH~FT (RULE 26) ~O 94/27237 Z I 61~ 2 3 PCT/US94/04991 Figure 8 is a flowchart which illustrates the steps taken during the evaluation of a Show_If or Skip_If q~ fier in the currently preferred embodiment of the present invention.
Figure 9 is a flowchart which illustrates the steps undertaken for evaluating a Make-Sure qualifier in the currently ~ref~lled embo-lintent of the present invention.
Figure 10 is a flowchart which illustrates the steps taken by the delivery engine responsive to a user invoking a Go-Back option in the currently ~re~lled embodiment of the present invention.
Figures lla-llc represent an example of a topic comprising three panels that may be l~tili7e~ in the currently ~re~lled embo~impnt of the ~resel~t invention "1993 Apple Computer, Inc." (17 U.S.C. 401).
Figure 12 is a listing of object definitions for implem~Pnti- g the topic illustrated in the example of Figures lla-llc.
Figure 13 illustrates the linkage between the objects of the example of Figures ll-llc.
SUBS~ITUTE SHEET (R(~LE 26) 2~g~'~2~ 6 DETAILED DESCRIPIION OF THE PREFERRED EMBODIMENT
This spe~-ifirAtic-n is related to the following spe~ific~tions which are ~signPd to the same Assignee, Apple Computer, Inc.:
Serial No. , filed , entitled "Method and Apparatus For Providing A Help Based Window System Using Multiple Access Methods"; Serial No. , filed , Pntitle~l "Method and Apparatus For Displaying And Scrolling Data In A Window-Based Graphic User Interface"; Serial No. , filed entitled "Method and Apparatus For Presenting Information In A Display Using Floating Windows"; and Serial No. filed .entitled "Method and Apparatus For Providing Visual Cues In A Graphical User Interface".
An interactive multime-liA instrl~ctio~Al delivery engine is fiP~rriked. In the following description, numerous specific details are set forth such as coding examples, in order to provide a thorough under~Pn~ing of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be prActice~l without these specific ~iet~ils In other instances, well-known circuits, for displaying text, graphics and video data have not been shown in detail in order not to mnecessArily obscure the present invention.
Overview of the Computer Svstem of the P~efel.ed Embodiment The computer system of the ~refel,ed embo-limPnt is described with refelel~ce to Figure 1. The present invention is ~rer~,ably implPm~nte~ on a general purpose microcomputer such as one of the mPmkPrs of the Apple~ ~Arinto~ ) family of computers. In any event, a computer system as may be l~ e-l by the ~ref~lled embodiment generally comprises a bus or other communication means 101 for comnlllnicAting information, a ~roc~ssillg means 102 coupled with said bus 101 for proc~ssin~
inform~tion, a r~n~lorn access memory (RAM) or other storage device 103 (commonly referred to as a main m~mory) coupled with said bus 101 for storing informA~ion and instructions for said processor 102, a read only memory (ROM) or other static storage device 104 coupled with said bus 101 for storing static information and instructions for said processor 102, a data SUBSTITUTE SHEET (R~LE 26) ~WO 94/27237 21~14 2 ~ PCT/US94/04991 storage device 105, such as a magnetic disk and disk drive, coupled with said bus 101 for storing information and instructions, an alphanumeric input device 106 including alphanumeric and other keys coupled to said bus 101 for communicating information and command selections to said processor 102, a cursor control device 107, such as a mouse, trA~ khAll, cursor control keys, etc., coupled to said bus 101 for commllni~ Ahng informAtion and command sPlections to said processor 102 and for controlling cursor movement, and a display device 108 coupled to bus 101 for displaying textual, graphical or video output. Additionally, it is useful if the ~y~Lelll in~ es a hardcopy device 109, such as a printer, for providing permAnent copies of information. The hardcopy device 109 is coupled with the processor 102 through bus 101. The computer system of Figure 1 may also include means for processing and re-playing audio data (not illustrated).
The computer system illustrated in Figure 1 is one that generally provides a window based graphical user interface which is well known (for example the Apple Macintosh family of computers, available from Apple Computers, Inc.), and hence, a ~etAile-l description is not neCpssAry herein.
In operation, the window based user interface of the currently plefelled emborliTnPnt is generated and displayed using software stored in the RAM
103 or ROM 104 and executed by the processor 102, as is known.
As will become apparent in the description below, various computer PlPTnPntc provide the filnCtic)TlAlity described. For example, a PrpsentAtion Window would be displayed on the display device 108. The processor 102 would provide procPssing resources for the object generation means, evaluation of ql-Alifiers, and other processin~ means comprising the delivery engine. The generated 1At~hAce would be stored in the data storage device 105 and loA~e~l into the RAM 103 during the delivery process.
In the currently ~lef~lled embo~ nent the multinlelliA delivery engine is imple~nente~l as part of a Help facility for providing assistance in the operation of the computer system. It will be apparent from the description below, that the present invention's features and attributes have applicAtion in a wide variety of areas, e.g. for interactive presentAtioP~, instruction, game development or referelLce guides. Accordingly, the SUBSTIT~ITE S~E~T (RULE 2~) 2 ~ 2 3 8 present invention is not limited by the following description of the currently ~lefel,ed embofiiment, and it will be understood by those skilled in the art, that the present invention may be applied to a computer systems offered by a variety of manufacturers, and to systems having both instr-lctionAl and non-instructional uses.
Overview of the Delivery Engine The delivery engine provides for the presentation of information using multiple media's. The currently yrerelred embo~limPnt is in a Help system which provides user instruction. The instruction is organi7e-1 as a series of topics. Each topic contains general topic information and r~re~ ces to panels. Each panel contains references to visual media such as text, graphics, video, as well as control and qualifier objects. The topics, panels, visual media, and qllalifiers are all encapsula~ed as individual entiti~oc in a ~ t~b~ce~ That is they are all objects in a rl~t~b~ce Figure 2 illustrates the basic components of the delivery engine and a pres~nt~tiorl window of the currently ~le~,red embodiment. The presPnt~tir~n window is a pre-~lefinell window having various display controls and which are used for displaying the panel's visual media data. Referring to Figure 2, a pr~cent~tion window 201 is coupled to a database retrieval engine 202. The prPsPnt~tion window of the currently ~lef~lled emborlimpnt is described in ~;reat~l detail below. The ~i~tAh~ce retrieval engine 202 operates responsive to current and prior operator input to retrieve from database 203, the next data to be displayed.
The database 203 contains a plurality of objects comprised of topic objects 204, panel objects 205 and panel objects 206. Generally spe~king, the various objects are fl~fine-l as follows:
Topic A series of Panels.
Panel A coll~ction of Panel Objects, Qualifiers and a display format which comprise what is seen by the user.
Panel Objects Text, graphics and video data, controls, qllalifiers and formats.
SL~BST~TU~E SHEET (RU~ 26) ~o 94/27237 216 1 4 2 3 PCTIUS94/04991 Controls Standard Macintosh control items such as push-buttons, radio buttons and check boxes. Radio buttons and check boxes are used to get user input for conditional tests for qllAlifiers.
Q11A1;fi~rS Condition which are used to determine which panels will be displayed to the user.
Formats Optional des~ Lor~ of how panel objects are displayed.
Movies Animated graphical images with or without synchronized sound.
In the currently ~rer~,ed emboliim~nt, objects may be used by more than one other object. This notion of re-use is one of the aspects of what is rommonly referred to as object oriented progrAmming. For example, a single item of text data may be used by multiple panels. The objects listed above are not meant to be exhaustive or limiting. Other objects could be impl~mente~ without causing departure from the spirit and scope of the present invention In the currently ~.e~e.led embodiment, the C++ progr~mmin~
language is used. C++ is used because of it's inherent capability to support object oriented progr~n~ming. However, it would be apparent to one skilled in the art to use alternative programming languages such as ADA
or Modula-2, would not cause departure from the spirit and scope of the present invention.
Figure 3 illustrates the logical rPlAtion~hips between the objects in a database. Referring to Figure 3 a topic 301 points to a series of panels, panel 1 302, panel 2, 303 and panel n 304. In Figure 3, the panel 1 302 is linked to qllAlih~r 305, graphics data 306, control 307, video data 308 and text data 309.
SUBSTITUTE SHEET (RULE 26) 2i~2~ 1 o Depending on the ~ont~nt~ a particular panel may have none, one or multiples of the foregoing panel objects. Referring back to Figure 3, the panel 2 303 is also linked to text data 309. Thus the text data 309 is displayedwhen either panel 1 302 or panel 2 303, is displayed.
Presentation Window A pr~cPntAtion window as may be llhli7e~ in the currently yrerell~d embodiment is illustrated in Figure 4. Referring to Figure 4, a presentAtion window 401 includes a topic title area 402 and a panel visual data area 403.
The topic title area 402 will rontAin the name of the topic being viewed.
The panel visual data area 403 will contain the visual data that is AcsoriAted with the panel being viewed. It is the panel visual data area 403 to which the data delivery engine will provide data for display. In the currently yrerel~ed embodiment, the panel visual data area 403 is 340 pixels wide.
The length is determined by the objects on the panel and the size of the screen. A panel with lots of text will be longer than a panel with little text.
The panel consists of any number of graphic, text, prompt, control and video objects. The only limit to the number of objects on a panel is the RAM available.
The objects are drawn in the panel in the order specified in the panel object. As a result, objects later in the sequence can be made to overlay previous objects in the sequence. Text can be positioned in fours different ways on a panel; tag, body, full panel and author positione~ Tag, body and full panel text are positioneli by the delivery engine. Tag text is typically right jl~stifiefi in a bold font type. Body text is typically left justified with a normal font type. Full panel text is used when only text will appear on the panel. ~iiti~ ~Al formats may be created, stored in the ~AtAh~e~ and used.
Text, graphic, video (movie) and control data positioI~c are deterrnined by parameters in the object definition stored in the database.
The prPsPrltAtion window 401 further contains a presentation window control area 404. The control area 404 provides means for ~a~ g through the instruction path and includes a previous panel button 405 and a next panel button 407. Activation of these buttons, typically by a point and click operation, causes backward (button 405) and SU8STITUTE S~EET (RU~E 26) ~WO 94127237 2i 6 I 4 23 PCT/US94/04991 forward (button 407) traversal through the series of panels comprising the topic. A current panel number indicator 406 allows the user to know what panel that they are currently viewing. Up to three (3) additional controls may be added to the control area 404, e.g. the control button 408. The controls may be for altering the flow of the pr~sPntAtion of panels (e.g.
switching topics.) These optional controls are stored in the IlAtAhA~e. The topic defines whether or not they should appear. The delivery engine retrieves and places the optional controls in the control area 404.
Finally, the presentation window 401 includes window control areas that are fAn~iliAr to those who use window based graphical user interfaces.
Window drag bar 410 allows the presentation window to be re-located via a cursor drag operation. Window re-sizing box 411 allows the size of the window to be shrunk/expanded as desired. Window close box 409 provides a quick means for closing the window and thus terminating the instruction (help) sP~sion.
Database Overview The present invention operates responsive to initial user queries for data. Once the initial query is made, the present invention operates to retrieve data for display in a presenPtion window. The r.olAtior~chip of the various objects in the IlAtAh~e is further illustrated in Figures 5a-5c.
Referring to Figure 5a, a rlAtAhAce 500 is illustrated with Topics 501, 508 and 515, Graphic object, 502, Ql]Alifi~r Object 503, Button 504, Panels 507 and 513, Control object 509 and 512, Format objects 505 and 510 and Text Objects 506 and 514. Figure 5b illustrates a linkage of Topic 501 with panels 507 and 513 and button 504. Finally, Figure 5c illustrates the linkage of Panel 507 and graphic object 502, QllAlifier 503, Control Object 509 and Text Object 514. The manner in which these linkages are made will become apparent in the description below.
Generating the Database As described above, the database is comprised of a plurality of dataobjects. Generation of the ~lAtAhAce is essentially a two step process;
authoring and object generation. This is analogous to writing and SUBSTITI~TE SHEET (R~LE 26) WO 94/27237 ~ PCT/US94/04991 21~2?~
compiling a software program. Authoring is similar to writing a program or a script. In this case, authoring is used to generate the topic conhonts The topic contents are then provided to an object generation means for creation of the objects for entry into the database. The object generation means performs a step that is similar to compil~ion The rl~tAh~e itself is analogous to an object file created during compilation.
In the currently preferred embodiment, various authoring tools may be used. In one instance the topic rontPnt~ may be generated as a set of instrl~ctioIl~ The topic contents may also be generated via a What You See Is What You Get (WYSIWIG) tool. In this case, the panels are generated as the would be displayed. Of course different object generation means would be used for each instance. Other techniques for creating the topic cont~nt~
used to generate the ~i~tAh~se would be apparent to one skilled in the art and would not depart from the spirit and scope of the present invention.
The fi~t~h~se is conceptually comprised of resources and data. The resources are the data objects. The data is the actual text, graphical or video data that is displayed. Data objects are retrieved by an object type and an i~lentific~tion number ("id"). In the currently ~ler~led embo~im~nt, the object type is a 4-character string and the object id is an integer number.
Table A lists a set of "bllilding-block" data objects that are available for thepresPnt~tion window in the currently ~rer~lled embodiment of the present invention. The list is not meant to be exhaustive, but merely representative of commonly used objects.
S~iBSTIT~T~ ~HEET (RIJLE 26) ~WO 94/27237 2 1 6 1 ~ 2 3 PCT/US94/04991 TABLE A
DATA OJBECI' LIST
Category Type Label Description Topic acPT kResTopic Topic script Panel ppUA kResPanel Panel Content pclX; kResSTxt Text Content .pcTF; kResSTxtF Full panel text Content pcTB; kResSTxtB Body text Content pcTT; kResSTxtT Tag text Content pcGA; kResGraphic Graphic Content pcGF; kResGraphicF Full panel graphic Content pcPR; kResPrompt Prompt Content pcCI; kResCtrlItem Control item Content pcMO; kResMovie Movie Content pcMC; kResMovieC Movie with controller Qualifier pqCN kShowIf Show panel if ~onditions are true.
Qualifier pqCS kSkipIf Skip panel if ,,onditions are true.
Qualifier pqMS kResMakeSure Make sure the conditions are true ~efore showing this panel. If not, show the Oops topic.
Qualifier pqCN kResCondition Condition Context pxCA kResContext ' he specification for a context check.
In the currently preferred embo~in ent, each object is defined as a ~A~intosh resource. A MA~intosh resource is code or data that may be used by various appli~ ~tion~. Macintosh resources as described in the S~JBS~ITUTE SHEET (RULE 26) WO 94127237 . PCT/US94/04991 publication entitled "Inside MArintosh Volume 1" published by Addison-Wesley Publishing Company, Inc. The actual resource definition for many of the objects described in Table A is provided in Appendix A. These resource definitions are exemplary of how the resource may be ~l~fin~1 and is not meant to be exhaustive.
Referring back to Table A, the first column Category in~ AtPS the general category of the data object, namely, Topic, Panel, Content, Action, Ql-~lifier or Context. The second column Type lists the object type. The third column is the object label. The object label is an alternative way to le~~ ce the object and is used by the authoring tools and by the delivery engine implementation code (C++). Finally, the fourth column is a brief description of the use for the particular object.
Operation of the Delivery Engitte In the currently ~re~led of the present invention, elements are used only when they are nee~ip~l Before starting the instruction (displaying panels), the delivery engine must set-up the topic to be viewed.
The steps for setting-up a topic are described with respect to the flowchart of Figure 6. Referring to Figure 6, the topic a is retrieved from ~e ~l~t~h~ce and is inst~ntiAte~l as an object, step 601. By inct~nti~tirlg as an object it is meant to create an instance of its class. The topic object contains general topic inform~tion and a list of panels associated with the topic. Each panel that is linked to the topic (i.e. in the topic's list of panels) is then retrieved from the database and also inst~nti~te~l as an object, step 602. The retrieved panels each cont~in a list of panel objects. Each panel object is then retrieved from the database and inst~nti~t~d as an object, step 603. Only the panel object description itself is retrieved and instantiated, not the content of the panel object. For example, a text panel object conPin~ a r~ ence to the text co~t~nt and the location or format of the text on the panel. The text content is not retrieved until the panel is displayed.
Next, each qll~lifipr linked to the panel is retrieved and instantiated as an object, step 604. The qualifier is attached to the panel object as a collaborator. A collaborator is an independent object that works to support, aid or assist another object. Finally, if any panel objects follow the panel in SlJBSTITUT~ SI~EET (RlJL~ 26) ~WO 94/27237 2 16 ~ ~ 2 3 PCT/US94/04991 the topic list (as distinguished from the panel list), they are retrieved and instantiated as objects, step 605. These objects are appended to the panel that was previously instantiated. This allows a panel descriptor to be modified within the topic. For example, prompts can be appended to a panel from the topic. At this point the topic of instruction is ready to be delivered.
The delivery of panels in the during the course of instruction is described with respect to the steps of the flowchart in Figure. Referring to Figure 7, once all the panels and panel objects have been retrieved and inst~nti~te.1, the delivery sequence begins with the first panel whose q~ ifiers determine that it can be shown or which has no ql]AlifiPrs step 701. The m~nnPr in which the qualifiers are Px~minerl is described below with refe~ ce to Figures 8 and 9. Here it is assumed that a first panel has been iclPntifie~l The panel object contents are then retrieved, step 702. As noted above, only the panel object descriptors were initially retrieved. The panel object ro~t~ntc are the text, graphics or video data that is to be displayed on the panel. Once the panel contents are retrieved, they are displayed in the panel display area of the presPnt~tion window, step 703. In this display step, the display of the objects may be formatted by the author or by the delivery engine by the author's choice of object type for the data.
At this point, a panel is displayed and the author may provide the user with various options. One option that an author may provide to the user is a go-back option. In the go-back option the user may jump back to a previously displayed panel. The go-back option is described below in more detail with respect to Figure 10. The other options are to traverse backwards or forwards through the current list of panels which is described as follows. These options are provided as part of the prPcPnPtion window functionality (see Figure 4). Referring back to Figure 7, a determin~hon is made by the delivery engine whether or not there are any more forward (next) panels to be displayed, steps 704, 705. This detern~in~tion is made transparent to the user while they are ex~mining the current panel. Lf no more panels are to be displayed in the forward direction, the "next" option is ~lic~klell~ step 706. In this instance, the user options are limite~l to showing a previous panel or to end the instruction, step 707. If the user SUBSTITUTE SI~EET (RUL~ 26~
2~ ~423 1 6 chooses to end instruction, the instruction is ter~in~te~l Otherwise, the previous panel is displayed, step 710.
Responding in the affirmative to the question of more panels to be displayed of step 705 causes the "next" and "previous" options to be enabled. The user must respond by sPlectin~ one of the options, step 708. If the "next" option is sPlPcte~, the next eligible panel is displayed, step 709.
The next eligible panel is i~lentifie~l by the state of the qualifiers associated with the next sllcces~ive panels (or by the lack of qualifiers for a panel).
The delivery engine then repeats the process by determining if any other panels remain to be displayed (step 704). If the previous option is sPlPcte~, the previous panel is displayed, step 710 and the process repeats with the delivery engine determining if any other panels remain to be displayed (step 704).
From the foregoing description it is apparent that the delivery engine m~int~ins a display list of panels that have been displayed. This display list facilitates the traversing of the panels using the go-back option and the make sure qualifier (as will be described below).
Evn~trrfion of Qualifiers The qll~lifiprs that may be associated with a panel f~rilit~te the interactive nature in which the panels of a topic may be displayed. The qll~lifiPrs and the con~litions referred to therein provide for control of its flow of panel display. Qll~lifiprs are evaluated by the delivery PngirlPs before each panel is shown and in the background while the user is not interacting with the delivery er~gine. Qualifiers can be based on user controls within any panel, not nece5sArily the one being currently shown, or on a test of the user's context. For example, if the instruction is to display a short cut terhnique for yelro~lning a function or a long way terhnique for performing a filnction, a user may be given the option of ~lerifling which way they want to learn to y~lrolm the function in a panel X. Assume that the panels for the instruction are linked seqllenti~lly as panels X through panels X+m. Further assume that the panels X+1 to X+n show the short cut terhni(lue, while the panels X+n + 1 to X + m show the long way technique. If the user at panel X chooses the to proceed by SUBSTI~TE SI~E~T (RUL~ 26) ~wo 94/27237 2 1 6 1 4 2 3 PCT/US94/04991 learning the long way technique, the panels X+1 to X+n (colles~onding to the short cut technique) would have qllAlifiers which would cause them to be skipped.
The evaluation for the show-if and skip-if qllAlifiPrs is described with rererence to Figure 8. Referring to Figure 8, if the panel has a Show-If or Skip-If qll~lifiPr, retrieve the condition for the qll~lifier from the ~lAtAkAce and instantiate it as an object, step 801. The condition is then evaluated, steps 802, 803. The condition may consist of a user control, a context check, or another coIl-lition These PleTnPnts are paired with and/or logic (author's choice) and can be nested in~lefinitely. Hence, the Show-Lf or Skip-Lf qll~lifi~r is a binary decision tree. The tree is evaluated.
If the eventual result is false, the panel will be skipped (Show-If) or shown (Skip-If), step 804. Lf the eventual result is true, the panel will be shown (Show-If) or skipped (Skip-Ifl, step 805.
A third qllAlifier is the Make-Sure qllAlifi~r. The Make-Sure qllalifiPr provides for the switching of topics (e.g. for r.ome~i~l instruction or a notice that the user must go back to a prior step) and as the criteria checked for in a go-back operation. The former instance of the Make-Sure ql~lifi~r is described with rer~ ce to Figure 9. First, the cc~n~liho~c for the qll~lifi~r are retrieved from the ~ AkAce and in.ctanhated it as an object, step 901. The retrieved panel conditions are then evAl-~Ate~, step 902 and 903. The con~lition is evaluated in the same way as the Show-If or Skip-If, but the result is used differently. If the result is false, another topic is A--tomatically shown, step 904. The i-lentifi~ ~tion of which topic is to be shown is ccntAine~l in the Make-Sure descriptor (See Appendix A for description). The first Make-Sure qualifier that yields a false result has its topic shown. If the result is true, then a check is made for Afl~litionAl Make-Sure qllAlifi~r5, step 905. Lf more Make-Sure qualifiers exist for the panel, they are evaluated per step 901. If there are no further Make-Surer qllAlifi~rs and all the Make-Sure qllAlifiers associated with the panel have evaluated to true, the panel may be shown, step 906.
The Make-Sure qualifier as used with the go-back option is described with rer~ ce to Figure 10. Referring to Figure 10, the user is presented SIJBS~U~ E~ ~R~E ~
wo 94/27237 PcT/uss4/04991 21~1423 1 8 with a "go-back" option by the author and the user elects to take it, step 1001. In this instance, the panels previously shown are searched in reverse order until a panel is found with no qualifiers or with qll~lifiPrs whose conditions (Skip-If, Show-If, Make-Sure) allow it to be shown, step 1002.
The panel is then displayed, step 1003. The Make-Sure ql-~lifi~r allows an author to specify a set of conditions (a Make-Sure) which, if changed, automatically provide a warning. The engine can then search back to find where in the sequence the conditions were acceptable, allowing the user an opportunity to redo the conditions correctly.
Context rherking iS an optional element of a conc1i~ior~l operation.
Note that a condition can specify one or more combinations of the fol!owing:
A panel control item such as a radio button or checkbox A context-check sperifi.-Afion Other con-litionc A context-check sperifi~tion is a resource (object) which, when evaluated in light of the user's context, results in a true or false. For example, ~ecking for an open "window" is a context-check specification.
Other context checks which can be performed in the currently ~ier~l~ed embo~im~nt include:
Window <title> is open and front;
Window <title> is open but invisible;
Application <name> is open <running>;
Application <name> is open and in front.
Switchittg of Topics SVBSTITUTE SHEET (R~IL~ 261 ~ 2161~23 As noted above, topics may be switched via the Make-Sure option of a panel. Moreover, in the currently ~refelled embodiment, multiple topics can be shown simultaneously. The author may elect to present a new topic while still displaying the old topic, close the old topic and replace it with a new one, or hide the old topic and show a new one (called an "Oops"
topic). A hi~lllen topic is automatically shown again when the "Oops" topic is closed. This is made possible by a displayed panel list maintained by the delivery engine.
Of course, whenever a topic switch the topic set-up described with refer~llce to Figure 6, must be performed.
Example An example of the present invention is now provided. Figures 11a-11c are the panels for a topic to be authored. In this example, instru*ion for performing certain operations on a computer system is being provided.
Here, the instruction may alternatively be provided via textll~l instrllctior or via a video showing the instruction being carried out. So if the instruction was for rlosin~ a window, the user has the option of reading how to m~nel1ver the cursor to point to a close box and then rlirking a button on the cursor control device, or simply watching the action being ~el~orl~led.
Referring to Figure 11a, a panel 1101 has two objects; a text object 1120 and a control object 1130. The text object 1120 will co~t~in the text which describes the topic of instruction. The control object 1130 is a checkbox by which a user may specify that they desire video instruction.
Figure 11b illustrates a panel 1102 which has one object namely a full panel text object 1121 which describes how to perform the desired operation.
Figure 11c illustrates a panel 1103 which has one object, namely a video object which is a move which shows the desired operation being carried out.
-Assuming that topic set-up has occurred, the panel 1101 will be displayed. The user is pr~sPnted with the option of having the instruction by text or video. Video instruction is specified by r~erking the VIDEO
SUB~TI~IJT~ SHEET (~U~E 26) wo 94/27237 PcT/uss4/04991 ,~
INSTRUCTION checkbox. Upon going to the next panel via the panel traversal operators of the pres~n~tion window, the panel 1102 will be displayed if the checkbox is not checked and the panel 1103 will be displayed if the checkbox is checked.
Figure 12 contains pseudo-code exemplary of how the foregoing example may be encoded using the objects described in Table A above.
Referring to Figure 12, line 1 contains the topic object definition. Lines 2-4 con~in the deffnition of the panel object cont~nt~, i.e. the text and the video data. Line 5 starts the definition of panel 1101. The objects associated with panel 1101 are on lines 6 and 7 (inll~nte~ for clarity), a text object and checkbox object, respectively. Line 8 contains the definition for the checkbox control object. Note that whereas the checkbox object is re~l.ced by panel 1101, it is rlPfinp~ outside of the object. This allows the checkbox object to be llfili7etl my multiple panels. Further note that the panel object contents may also be defined within the panel ~i~finition in which the object is used.
Line 9 begins the definition of panel 1102. The panel 1102 includes a text object and Skip-If qualifier object, lines 10 and 11, respectively. At line12, the qll~lifi~r object linked by panel 1102 is ~l~fine~ Note that the Ski~If qll~lifiPr object links to the checkbox object as a rcn~lition (line 13).
Line 14 begins the definition of panel 1103. The panel 1103 includes a video object and a Show-If Qll~lifipr object, lines 15 and 16, respectively.
Finally, the Show-If qualifier object is ~l~fine~ at lines 17-18. As in the caseof the Skip-If qllAlifier, the Show-If qll~lifiPr links to the checkbox object as a co~ ;on (line 18).
Figure 13 illustrates the linkages between the objects in the example.
Referring to Figure 13, the topic 1100 would be linked to the panels 1101, 1102 and 1103. The panel 1101 would have links to the text 1120 and control 1130. As is apparent, multiple objects will link to the control 1130.
The panel 1102 links to text object 1121 and Skip-IF qualifier 1160. The qualiffer 1160 in turn links to the control 1130. Finally, the panel 1103 links SUBST~TUTE S~EE i (~IJL~ 26) ~WO 94/27237 21 614 2 3 PCT/US94/04991 to video object 1140 and to Show-If qualifier 1170. The Show-If Qll~lifier in turn links to control 1130.
During set-up of the topic, the objects are loaded as described above with refe~ ce to Figure 6. Delivery of the panels would then occur in the manner as described with rerer~l,ce to Figures 7-10.
Thus, a method and apparatus for interactive delivery of ml1ltinlefli~ data for display on a computer system, is described.
SfJBSTITlJTE SHEET ~RULE 26) WOg4/27~7 . PCT~S94/04991 ~1~1423 22 ~ PENDIX A
// Copyright ~ 1993 Apple Computer, Inc. (17 U.S.C. 401) All rights reserved.
// Date: 05-May-93 // Delivery Engine Data Types // Coach Mark resouce type kResCoachMark {
longint none=kTargetNone, // target application signature self=kTargetSel~, front=kTargetFront, topic=kTargetTopic;
longint balloon='ndex', location='mLoc', menu='mMrk', object='draw';
// event id integer = $$Count of(keys);
array keys {
switch {
case Number:
key longint='iNum';
pstring; // balloon or dialog or menu item number case Origin:
key longint='oCon'; // origin reference for rectangle pstring topLeft="0", bottomLeft="1", topRight-"2", bottomRight="3";
case Rect:
key longint='iRct'; // marking rectangle pstring; // "rct=" top,left,bottom,right case Macro:
key longint='lMac'; // location macro pstring GrowBox="GrowBox", ZoomBox="ZoomBox", CloseBox="CloseBox", TitleBar="TitleBar", BootDisk="BootDisk";
case WindowName:
SUB~TI~UTE SHEET (RU~E 2~) ~ W094/27~7 21~ 1'12 3 PCT~S94/04991 key longint='wNam';
pstring; // window name case FrontWindow:
key longint='wNam';
pstring=" _ FRONTWINDOW _ "; // window name case DesktopWindow:
key longint='wNam';
pstring=" _ DESKWINDOW _ "; // window name case BalloonNumber:
key longint='iNum';
pstring; // balloon item number case DialogItemNumber:
key longint='iNum';
pstring; // dialog item number case MenuTitleName:
key longint='mNam';
pstring; // menu title name case MenuTitleNumber:
key longint='mNum';
pstring; // menu title number case MenuItemName:
key longint='iNam';
pstring; // menu item name case MenuItemNumber:
key longint='iNum';
pstring; // menu item number case ObjectName:
key longint='iNam';
pstring; // menu item name case MarkStyle:
key longint='sNam';
pstring red_circle="defaultl", - yellow x="default2", red_underline="default3", red_arrow="default4"; // mark style name case MenuItemColor:
S! IBSTITUTE SHEET ~RULE 26) wog4n7~7 PCT~S94/04991 ~1~142~ 24 key longint='mCol';
pstring myblack="l", yellow="2", // myblack instead of black // because black is defined in ViewTypes.r magenta="3", red="4", cyan="S", green="6", blue=~'7", whitez~8~; // menu item color number case MenuItemStyle:
key longint='mFac';
pstring cc_bold="l", cc_italic="2", cc_underline="4", cc_outline="8", cc_shadow="l6", cc_condense="32", cc_extend="64"; // font style number case StyleOption:
key longint='sOpt';
pstring; // "Lng=0,0,S,F" where S is start // and F is finish // l=NW, 2=N., 3=NE, 4=E, ... 8=W
}; // switch }; // array align word;
} ;
// Topic type kResTopic {
longint none=0; // target application signature integer none-kNone, noButton=kNone, defaultButton=kDefault, huhButton=kHuhBtnId, goStartButton=kGoStartBtnId; // optional button #l id integer none=kNone, noButton=kNone, defaultButton=kDefault, huhButton=kHuhBtnId, goStartButton=kGoStartBtnId; // optional button #2 id integer none=kNone, noButton=kNone, SlJBSTl~UTE SHEET (RU~E 26!
W094/27~7 2 1 6 14 2 3 PCT~S94/04991 defaultButton=kDefault, huhButton=kHuhBtnId, goStartButton=kGoStartBtnId; // optional button #3 id integer none=kNone, noMinimum=kNone, defaultMinimum=kNone; // minimum panel height integer none=kNoMaxHeight, noMaximum=kNoMaxHeight, default~x;mllm=kNoMaxHeight; // maximum panel height integer; // topic title TEXT ID number integer; // reserved field, always enter 0.
integer; // reserved field, always enter 0.
integer; // reserved field, always enter 0 integer = $$Countof(entries);
array entries {
longint; // entry type lonqint; // resource id or other };
};
// Panel type kResPanelUnivA {
integer = $$CountOf(items);
array items {
longint; // scope integer; // resId (DBRef) for scope }; // array }; // type // Text format- for full, tag, body, and custom formats.
type kResFormat {
integer teFlushDefault=0, teCenter=l, teFlushRight=-l, teFlushLeft=-2; // justification SIJ~STITUTE SHEET (RULE 26) W094l27237 PCT~S94/04991 integer normal=0; // face integer; // size integer; // top integer; // left integer; // bottom integer; // right pstring; // font name align word; // Espy Serif, Espy Sans, Espy Serif Bold....
} ;
// Text item type kResSTxt {
integer; // resId for TEXT/styl integer; // top integer; // left integer; // bottom integer; // right } ;
// Graphic item type kResGraphic I
integer; // resId for PICT
integer; // top integer; // left } ;
// Movie item type kResMovie {
integer; // top integer; // left pstring; // movie file name align word;
} ;
SU~STITUTE SHE~T (R~LE 26) W094/27~7 21 614 2 ~ PCT~S94/04991 .
// Ctrl item // Standard Macintosh controls type kResCtrlItem {
switch {
case PushButton:
key int = pushButProcReno;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case RadioButton:
key int = radioButProcReno;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case CheckBox:
key int = checkBoxProcReno;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title SLlBSTITllTE S~EET (~L~ 26) W094/27~7 PCT~S94/04991 2161~3 28 case PushButtonSysFont:
key int = pushButProcRenoSysFont;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case RadioButtonSysFont:
key int = radioButProcRenoSysFont;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case CheckBoxSysFont:
key int = checkBoxProcRenoSysFont;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title }; // switch align word;
}; // type // Apple Event SUBSTITIJTE SI~ET (RIJL~ 2~?
W094/27~7 2 1 ~ 1~ 2 ~ PCT~S94/04991 type kResEvent {
longint none=kTargetNone, // target application signature self=kTargetSelf, front=kTargetFront, topic=kTargetTopic, reno=kTargetReno;
longint; // class longint none=0; // event id integer none=0; // optional short integer with key 'IOPT' longint none=0; // optional key pstring none=""; // data for optional key align word;
} ;
// Condition // Referenced by Show-If, Skip-If, and Make-Sure.
type kResCondition {
integer none=0, not=kNot; // optional not or kNot integer update=kUpdate, dontUpdate=kDontUpdate;
longint; // type (EG kResCtrlItem, kResContext, kResCondition) integer; // resId integer none=kNone, or=kOr, and=kAnd; // logic integer none=0, not=kNot; // optional not or kNot integer update=kUpdate, dontUpdate=kDontUpdate;
longint none=0; // type integer none=0; // resId } ;
// Make-Sure type kResMakeSure I
integer; // resource Id of kResCondition resource.
integer; // resource Id of kResTopic resource for Oops.
} ;
// Prompt SUBST~UTE S~E~T (RIJ~ 26) W094/27~7 PCT~S94/04991 2 ~ 3 0 type kResPrompt {
integer; // Id of first (beginning) TEXT/styl resource integer; // Id of second (middle) TEXT/styl resource integer; // Id of third (end) TEXT/styl resource integer; // Id of fourth (choice) TEXT/styl resource } ;
// Hot panel area type kResHot ~
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEventId number } ;
// Custom button type kResRenoBtn {
integer=1; // (always momentary) integer; // off-up PICT id integer; // on-down PICT id integer=0; // (dimmed not available) longint none=0; // action, kResEvent id number integer=l; // (not used, kResEvent is always immediate) integer; // top of top-left corner location integer; // left of top-left corner location integer=0; // (always fixed) integer none=0; // string index for Help Balloon text.
} ;
// PreWin button SU~STITUTE SHE~T (RULE 26) W094l27~7 21614 2 ~ PCT~S94/04991 // Placed in the control area along the bottom // of the presentation window.
type kResPreWinBtn {
integer=l; // ~always momentary) integer; // off-up PICT id integer; // on-down PICT id integer; // dimmed longint none=0, // action, kResEvent id number integer=l; // (not used, kResEvent is always immediate) integer=0; // top of top-left corner location is preset integer=0; // left of top-left corner location is preset integer align=l, movable=l, fixed=0; align (l) or fixed (0) integer none=0; // string index for Help Balloon text.
} ;
// PerWin Button.
// Placed in the control area alo~g the bottom // of the presentation window.
type kResPreWinBtn {
integer=l; // (always momentary) integer; // off-up PICT id integer; // on down PICT id integer; // dimmed longint none=0; //action, KResEvent id number integer=l; //not used, kResEvent is always immediate) integer=0; // top of top-left corner location is preset integer=0; // left of top-left corner location is preset integer align=l, moveable=l, fixed=0; // align (l) or fixed (0) SUBSTl~U~ S'~ET (RULE 26) W094/27~7 PCT~S94/04991 21~1423 32 integer non=O; // string index for Help Ballon text.
} ;
type kResContext {
longint none=kTargetNone, // target Process signature self=kTargetSelf, front=kTargetFront, topic=kTargetTopic, reno=kTargetReno;
switch {
case Chooser:
key longint = kCntxChooser;
switch {
case isPrinterDirect:
key longint = 0;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isPrintersOnZone:
key longint = 1;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isPrinterSerial:
key longint = 2;
longint ModemPort=0, PrinterPort=l, EitherPort=2;
fill byte[12]; // filler for 2 longs & 2 StringSpecs SUBST~TU~E ~HE~ ~ (R~LE 26) W094/27~7 PCT~S~ 1991 33 2i~1~23 case isPrinterSCSI:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isPrinterType:
key longint = 4;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches-10, contains=11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Printer name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isAppleTalk:
key longint = 5;
longint off=0, on=1;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isNumberOfZones:
key longint = 6;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isPortSelected:
key longint = 7;
longint ModemPort=0, SUBSTITUT~ SHEET (R~ILE 26) W094/27~7 PCT~S94/04991 2 ~ 2 3 3 4 PrinterPort=l, EitherPort=2;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isPortInUse:
key longint = 8;
longint ModemPort=0, PrinterPort=l, EitherPort=2;
fill byte[12]; // filler for 2 longs ~ 2 StringSpecs }; // Chooser switch case Dialog:
key longint = kCntxDialog;
switch {
case isFront:
key longint = 0;
longint; // Resource ID
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isOpen:
key longint = l;
longint; // Resource ID
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isInvisible:
key longint = 2;
longint; // Resource ID
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isFindDialog:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isFindMoreDialog:
key longint = 4;
SUBSTITIJT~ S!~E~T (RULE 2~) W094/27~7 21 6 1~ ~ 3 PCT~S94/04991 fill byte[16]; // filler for 3 longs & 2 StringSpecs case isAnyDialogActive:
key longint = 5;
fill byte[16]; // filler for 3 longs & 2 StringSpecs }; // Dialog switch case File:
key longint = kCntxFile;
switch {
case isFrontStartup:
key longint = 0;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isOpenStartup:
key longint = 1;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isCDStartup:
key longint = 2;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isPrintMonitor:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isAppleShare:
key longint = 4;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isEtherTalk:
key longint = 5;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isTokenTalk:
key longint = 6;
fill byte[16]; // filler for 3 longs & 2 StringSpecs Sl~BST)~lJT~ SHEET (Rl~E 26) W094/27~7 PCT~S94/04991 2~-1423 36 case isControlPanel:
key longint = 7;
fill byte[8]; // filler for 2 longs pstring; // Nameof Control Panel align word;
fill byte [6]; // filler for 1 long & 1 StringSpec case isFileExists:
key longint = 9;
longint SystemFolder='macs', DesktopFolder='desk', TrashFolder='trsh', EmptyTrashFolder='empt', PrintMonitorFolder='prnt', StartupFolder='strt', AppleMenuFolder='amnu', ControlPanelFolder='ctrl', ExtensionsFolder='extn', PreferencesFolder='pref', TemporaryFolder='temp'; // Sy$tem folder selector fill byte[4]; // filler for 1 long pstring; // Name of desired file align word;
fill byte[6]; // filler for 1 long & 1 StringSpec }; // File switch case Finder:
key longint = kCntxFinder;
fill long; // Finder selector fill byte[16]; // filler for 3 longs & 2 StringSpecs case Process:
key longint = kCntxProcess;
switch {
case isFront:
St~B~ITUTE SH~T (t~ 26) ~W094/27~7 2 ~ ~ 14 2 3 PCT~S94/04991 key longint = 0;
longint; // Process signature fill byte[l2]; // filler for 2 longs & 2 StringSpecs case isOpen:
key longint = l;
longint; // Process signature fill byte[12]; // filler for 2 longs & 2 StringSpecs }; // Process switch case System:
key longint z kCntxSystem;
switch {
case gestaltCheck:
key longint 5 0;
longint; // Gestalt Selector longint equals=2, notEqualTo=3, greaterThan=4, lessThan-5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector fill byte[2]; // not used longint; // Gestalt compare value fill byte[2]; // filler for 1 StringSpec case menuItemExists:
key longint z 1;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
SUBSTITUT~ SHE~T (RI~LE 26) W094/27~7 PCT~S94/04991 longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case menuItemMarked:
key longint = 2;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith-13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
longint containsAnything=8, empty=9, matches=10, contains=11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case menuItemF.n~hled:
key longint = 3;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, SUB~TITlJTE S~E~T (RlJLE 26) W094/27~7 2 1 6 14 2 ~ PCT~S94/04991 startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case menuItemDisabled:
key longint = 4;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=lO, contains=11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case isFileSharing:
key longint = 6;
longint off=0, ~3~jB~ HEET (~ULE 26) W094/27~7 PCT~S94/04991 on=1;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isVideoHWBitDepth:
key longint = 7;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isMonitorBitDepth:
key longint ~ 8;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isComputerNamed:
key longint = 9;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains-11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Printer name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec SllB~T~UTE ~ T (RUL~ 2~
W094/27~7 2 1 6 1~ Z ~ PCT~S94/04991 case isGuestAccess:
key longint = 10;
longint off=0, on=l;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isNumberMonitors:
key longint = 11;
longint equals=2, notEqualTo=3, greaterThan-4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo-7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs }; // System switch case Window:
key longint = kCntxWindow;
switch {
case isFront:
key longint = 0;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Window name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isOpen:
key longint = l;
SUBSTITUTE SHEET (RULE 26) W094/27~7 PCT~S94/04991 2 ~ 2 3 4 2 fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=11, startsWith=12, endsWithc13; // StringCompareMethod pstring; // Window name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isInvisible:
key longint = 2;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches~10, contains-11, startsWith312, endsWith-13; // StringCompareMethod pstring; // Window name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isShareWindowOpen:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isShareWIndowActive:
key longint 2 4;
fill byte[16]; // filler for 3 longs & 2 StringSpecs }; // Window switch }; // switch }; // type SU3Sr1TUTE Sl IEET (RULE 26)
Two known systems provide interactive mllltimerliA delivery capabilities; MacroMe~ Director and HyperCard~) MacroMedia Director, a product of MacroMedia Inc., use a score, stage, and cast metaphor to deliver interactive mllltimerliA A score (timeline) is used to place cast members (mllltime-liA objects) on the stage (display). Objects can be layered visually by their placement along the vertical axis of the score and temporally by their plAcement along the hori7ontAl axis. What the user sees at any point in time is repr~s~nterl by a column in the score. The flow of delivery (i.e.
instruction) is controlled by the score. Scripts can be AttA~e~l to cast members and the score to alter the flow of delivery. Scripts require SUBSTITUTE SHEET (RULE 26) wo 94/27237 PcT/Uss4lo49sl ~
2 ~ 2 ~ 2 pror~ssing external to the flow of delivery. Further, subsequent alterations to the flow are cumbersome.
HyperCard, a product of Apple~ Computer, Inc. uses a notecard and object metaphor to deliver interactive multimP~ instruction. Visual objects are placed on cards. The cards are then ordered. The flow of delivery is controlled by the card order. Scripts can be ~ rh~l to visual objects and cards to alter the flow of instruction. A HyperCard card is equivalent to a column in the MacromP-liA Director score. A HyperCard object is equivalent to a cast member in MacroM~ Director. Both e1P~n~nt~ in both products can have scripts ~tt~r~te~l The prior art systems have various ~lPfirir-nri~s. First, the flow of data delivery may only be altered via scripting. It would be desirable to be able to alter the flow of instruction inherently within the design of the instruction. Further, there is no backtracking once a flow of instruction has commenced.
Accordingly, it would be desirable to have an interactive ml~ltime~
delivery system which o~ comes the foregoing ~efirir-ncies of the prior art.
SUBSTITUTE S~EET (RU~ E 26) ~10 94127237 2 ~ 6 14 2 3 PCT/US94/04991 SUMMARY
A delivery engine for interactive presentation and display of mllltimP~iA data is disclosed. The delivery me~ h~ni~m is prPmi~e~1 on a topic and panel metaphor. A topic is comprised of panels. Each panel ~l~fin~c the mllltimecli~ data which will be used when the panel is displayed. Each panel may also contain control and qllAlifiPr PlPmPnt~
which are used to determine the next panel to be displayed. Control PlPmPnt~ are used to solicit inform~tion from the user which is then used by the qualifier PlPmpnts in determining the next panel to be displayed.
Each PlPmPnt is stored in a database as a discrete object. This allows re-use of objects ~mon~st different panels or topics. Prior to delivery of the data, the topic must be set-up. Set-up occurs by retrieving all the necessary panel, control qll~lifier and data descriptor objects. The actual data ~csori~te~l with data media objects is only retrieved when the data is to be displayed. The various qllAlifiPr objects are evaluated while panels are being viewed.
The ~l~t~h~ce is generated via an authoring step. In the authoring step, the topic and the various elPmPntc are combined to create a flow for the ~,æs~ hon of the mllltime~ data. Once the authoring is completed, the ~ TnPnt~ are processed in a manner similar to compiling a software program, to create the rl~t~h~ce SUBSTi~TE ~H~ET ~RU~ 2~) wo 94/27237 PcT/uss4/0499l ~
~1~142~ 4 BRIEF DESCRIPIION OF THE FIGURE
Figure 1 is a block diagram illustrating the components of a computer system as may be llhli7e~ in the currently yref~.~ed embo~ Pnt of the present invention.
Figure 2 is a block diagram illustrating the components of the delivery engine of the currently yrere~led embo~lim~nt of the present invention.
Figure 3 is a diagram which illustrates the logical relAticI ~hip Amongct the database objects of the currently yrefelled embo~liment of the present invention.
Figure 4 illustrates a PresentAti-~n Window as may be lltili7e~1 in the currently y~ef~lled embo~limPnt of the present invention "~)1993 Apple Computer, Inc." (17 U.S.C. 401).
Figure 5a is a block diagram of a database as generated in the currently yrer~:lled emboflimPnt of the present invention.
Figure 5b illustrates the linkage between a topic and cc lresyo~ ing panels of the rlAtAbAse of Figure 5a.
Figure 5c illustrates the linkage between a panel and cclle:jyon.1ing visual objects and qualifiers of the ~lAtAhAce of Figure 5a.
Figure 6 is a flowchart which illustrates the steps retrieving objects for setting up a topic for delivery in the currently yref~:lled embo~isn.ont of the present invention.
Figure 7 is a flowchart which illustrates the steps retrieving objects for delivery of panels in the currently preferred embodiment of the present invention.
SlJBSTITUTE SH~FT (RULE 26) ~O 94/27237 Z I 61~ 2 3 PCT/US94/04991 Figure 8 is a flowchart which illustrates the steps taken during the evaluation of a Show_If or Skip_If q~ fier in the currently preferred embodiment of the present invention.
Figure 9 is a flowchart which illustrates the steps undertaken for evaluating a Make-Sure qualifier in the currently ~ref~lled embo-lintent of the present invention.
Figure 10 is a flowchart which illustrates the steps taken by the delivery engine responsive to a user invoking a Go-Back option in the currently ~re~lled embodiment of the present invention.
Figures lla-llc represent an example of a topic comprising three panels that may be l~tili7e~ in the currently ~re~lled embo~impnt of the ~resel~t invention "1993 Apple Computer, Inc." (17 U.S.C. 401).
Figure 12 is a listing of object definitions for implem~Pnti- g the topic illustrated in the example of Figures lla-llc.
Figure 13 illustrates the linkage between the objects of the example of Figures ll-llc.
SUBS~ITUTE SHEET (R(~LE 26) 2~g~'~2~ 6 DETAILED DESCRIPIION OF THE PREFERRED EMBODIMENT
This spe~-ifirAtic-n is related to the following spe~ific~tions which are ~signPd to the same Assignee, Apple Computer, Inc.:
Serial No. , filed , entitled "Method and Apparatus For Providing A Help Based Window System Using Multiple Access Methods"; Serial No. , filed , Pntitle~l "Method and Apparatus For Displaying And Scrolling Data In A Window-Based Graphic User Interface"; Serial No. , filed entitled "Method and Apparatus For Presenting Information In A Display Using Floating Windows"; and Serial No. filed .entitled "Method and Apparatus For Providing Visual Cues In A Graphical User Interface".
An interactive multime-liA instrl~ctio~Al delivery engine is fiP~rriked. In the following description, numerous specific details are set forth such as coding examples, in order to provide a thorough under~Pn~ing of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be prActice~l without these specific ~iet~ils In other instances, well-known circuits, for displaying text, graphics and video data have not been shown in detail in order not to mnecessArily obscure the present invention.
Overview of the Computer Svstem of the P~efel.ed Embodiment The computer system of the ~refel,ed embo-limPnt is described with refelel~ce to Figure 1. The present invention is ~rer~,ably implPm~nte~ on a general purpose microcomputer such as one of the mPmkPrs of the Apple~ ~Arinto~ ) family of computers. In any event, a computer system as may be l~ e-l by the ~ref~lled embodiment generally comprises a bus or other communication means 101 for comnlllnicAting information, a ~roc~ssillg means 102 coupled with said bus 101 for proc~ssin~
inform~tion, a r~n~lorn access memory (RAM) or other storage device 103 (commonly referred to as a main m~mory) coupled with said bus 101 for storing informA~ion and instructions for said processor 102, a read only memory (ROM) or other static storage device 104 coupled with said bus 101 for storing static information and instructions for said processor 102, a data SUBSTITUTE SHEET (R~LE 26) ~WO 94/27237 21~14 2 ~ PCT/US94/04991 storage device 105, such as a magnetic disk and disk drive, coupled with said bus 101 for storing information and instructions, an alphanumeric input device 106 including alphanumeric and other keys coupled to said bus 101 for communicating information and command selections to said processor 102, a cursor control device 107, such as a mouse, trA~ khAll, cursor control keys, etc., coupled to said bus 101 for commllni~ Ahng informAtion and command sPlections to said processor 102 and for controlling cursor movement, and a display device 108 coupled to bus 101 for displaying textual, graphical or video output. Additionally, it is useful if the ~y~Lelll in~ es a hardcopy device 109, such as a printer, for providing permAnent copies of information. The hardcopy device 109 is coupled with the processor 102 through bus 101. The computer system of Figure 1 may also include means for processing and re-playing audio data (not illustrated).
The computer system illustrated in Figure 1 is one that generally provides a window based graphical user interface which is well known (for example the Apple Macintosh family of computers, available from Apple Computers, Inc.), and hence, a ~etAile-l description is not neCpssAry herein.
In operation, the window based user interface of the currently plefelled emborliTnPnt is generated and displayed using software stored in the RAM
103 or ROM 104 and executed by the processor 102, as is known.
As will become apparent in the description below, various computer PlPTnPntc provide the filnCtic)TlAlity described. For example, a PrpsentAtion Window would be displayed on the display device 108. The processor 102 would provide procPssing resources for the object generation means, evaluation of ql-Alifiers, and other processin~ means comprising the delivery engine. The generated 1At~hAce would be stored in the data storage device 105 and loA~e~l into the RAM 103 during the delivery process.
In the currently ~lef~lled embo~ nent the multinlelliA delivery engine is imple~nente~l as part of a Help facility for providing assistance in the operation of the computer system. It will be apparent from the description below, that the present invention's features and attributes have applicAtion in a wide variety of areas, e.g. for interactive presentAtioP~, instruction, game development or referelLce guides. Accordingly, the SUBSTIT~ITE S~E~T (RULE 2~) 2 ~ 2 3 8 present invention is not limited by the following description of the currently ~lefel,ed embofiiment, and it will be understood by those skilled in the art, that the present invention may be applied to a computer systems offered by a variety of manufacturers, and to systems having both instr-lctionAl and non-instructional uses.
Overview of the Delivery Engine The delivery engine provides for the presentation of information using multiple media's. The currently yrerelred embo~limPnt is in a Help system which provides user instruction. The instruction is organi7e-1 as a series of topics. Each topic contains general topic information and r~re~ ces to panels. Each panel contains references to visual media such as text, graphics, video, as well as control and qualifier objects. The topics, panels, visual media, and qllalifiers are all encapsula~ed as individual entiti~oc in a ~ t~b~ce~ That is they are all objects in a rl~t~b~ce Figure 2 illustrates the basic components of the delivery engine and a pres~nt~tiorl window of the currently ~le~,red embodiment. The presPnt~tir~n window is a pre-~lefinell window having various display controls and which are used for displaying the panel's visual media data. Referring to Figure 2, a pr~cent~tion window 201 is coupled to a database retrieval engine 202. The prPsPnt~tion window of the currently ~lef~lled emborlimpnt is described in ~;reat~l detail below. The ~i~tAh~ce retrieval engine 202 operates responsive to current and prior operator input to retrieve from database 203, the next data to be displayed.
The database 203 contains a plurality of objects comprised of topic objects 204, panel objects 205 and panel objects 206. Generally spe~king, the various objects are fl~fine-l as follows:
Topic A series of Panels.
Panel A coll~ction of Panel Objects, Qualifiers and a display format which comprise what is seen by the user.
Panel Objects Text, graphics and video data, controls, qllalifiers and formats.
SL~BST~TU~E SHEET (RU~ 26) ~o 94/27237 216 1 4 2 3 PCTIUS94/04991 Controls Standard Macintosh control items such as push-buttons, radio buttons and check boxes. Radio buttons and check boxes are used to get user input for conditional tests for qllAlifiers.
Q11A1;fi~rS Condition which are used to determine which panels will be displayed to the user.
Formats Optional des~ Lor~ of how panel objects are displayed.
Movies Animated graphical images with or without synchronized sound.
In the currently ~rer~,ed emboliim~nt, objects may be used by more than one other object. This notion of re-use is one of the aspects of what is rommonly referred to as object oriented progrAmming. For example, a single item of text data may be used by multiple panels. The objects listed above are not meant to be exhaustive or limiting. Other objects could be impl~mente~ without causing departure from the spirit and scope of the present invention In the currently ~.e~e.led embodiment, the C++ progr~mmin~
language is used. C++ is used because of it's inherent capability to support object oriented progr~n~ming. However, it would be apparent to one skilled in the art to use alternative programming languages such as ADA
or Modula-2, would not cause departure from the spirit and scope of the present invention.
Figure 3 illustrates the logical rPlAtion~hips between the objects in a database. Referring to Figure 3 a topic 301 points to a series of panels, panel 1 302, panel 2, 303 and panel n 304. In Figure 3, the panel 1 302 is linked to qllAlih~r 305, graphics data 306, control 307, video data 308 and text data 309.
SUBSTITUTE SHEET (RULE 26) 2i~2~ 1 o Depending on the ~ont~nt~ a particular panel may have none, one or multiples of the foregoing panel objects. Referring back to Figure 3, the panel 2 303 is also linked to text data 309. Thus the text data 309 is displayedwhen either panel 1 302 or panel 2 303, is displayed.
Presentation Window A pr~cPntAtion window as may be llhli7e~ in the currently yrerell~d embodiment is illustrated in Figure 4. Referring to Figure 4, a presentAtion window 401 includes a topic title area 402 and a panel visual data area 403.
The topic title area 402 will rontAin the name of the topic being viewed.
The panel visual data area 403 will contain the visual data that is AcsoriAted with the panel being viewed. It is the panel visual data area 403 to which the data delivery engine will provide data for display. In the currently yrerel~ed embodiment, the panel visual data area 403 is 340 pixels wide.
The length is determined by the objects on the panel and the size of the screen. A panel with lots of text will be longer than a panel with little text.
The panel consists of any number of graphic, text, prompt, control and video objects. The only limit to the number of objects on a panel is the RAM available.
The objects are drawn in the panel in the order specified in the panel object. As a result, objects later in the sequence can be made to overlay previous objects in the sequence. Text can be positioned in fours different ways on a panel; tag, body, full panel and author positione~ Tag, body and full panel text are positioneli by the delivery engine. Tag text is typically right jl~stifiefi in a bold font type. Body text is typically left justified with a normal font type. Full panel text is used when only text will appear on the panel. ~iiti~ ~Al formats may be created, stored in the ~AtAh~e~ and used.
Text, graphic, video (movie) and control data positioI~c are deterrnined by parameters in the object definition stored in the database.
The prPsPrltAtion window 401 further contains a presentation window control area 404. The control area 404 provides means for ~a~ g through the instruction path and includes a previous panel button 405 and a next panel button 407. Activation of these buttons, typically by a point and click operation, causes backward (button 405) and SU8STITUTE S~EET (RU~E 26) ~WO 94127237 2i 6 I 4 23 PCT/US94/04991 forward (button 407) traversal through the series of panels comprising the topic. A current panel number indicator 406 allows the user to know what panel that they are currently viewing. Up to three (3) additional controls may be added to the control area 404, e.g. the control button 408. The controls may be for altering the flow of the pr~sPntAtion of panels (e.g.
switching topics.) These optional controls are stored in the IlAtAhA~e. The topic defines whether or not they should appear. The delivery engine retrieves and places the optional controls in the control area 404.
Finally, the presentation window 401 includes window control areas that are fAn~iliAr to those who use window based graphical user interfaces.
Window drag bar 410 allows the presentation window to be re-located via a cursor drag operation. Window re-sizing box 411 allows the size of the window to be shrunk/expanded as desired. Window close box 409 provides a quick means for closing the window and thus terminating the instruction (help) sP~sion.
Database Overview The present invention operates responsive to initial user queries for data. Once the initial query is made, the present invention operates to retrieve data for display in a presenPtion window. The r.olAtior~chip of the various objects in the IlAtAh~e is further illustrated in Figures 5a-5c.
Referring to Figure 5a, a rlAtAhAce 500 is illustrated with Topics 501, 508 and 515, Graphic object, 502, Ql]Alifi~r Object 503, Button 504, Panels 507 and 513, Control object 509 and 512, Format objects 505 and 510 and Text Objects 506 and 514. Figure 5b illustrates a linkage of Topic 501 with panels 507 and 513 and button 504. Finally, Figure 5c illustrates the linkage of Panel 507 and graphic object 502, QllAlifier 503, Control Object 509 and Text Object 514. The manner in which these linkages are made will become apparent in the description below.
Generating the Database As described above, the database is comprised of a plurality of dataobjects. Generation of the ~lAtAhAce is essentially a two step process;
authoring and object generation. This is analogous to writing and SUBSTITI~TE SHEET (R~LE 26) WO 94/27237 ~ PCT/US94/04991 21~2?~
compiling a software program. Authoring is similar to writing a program or a script. In this case, authoring is used to generate the topic conhonts The topic contents are then provided to an object generation means for creation of the objects for entry into the database. The object generation means performs a step that is similar to compil~ion The rl~tAh~e itself is analogous to an object file created during compilation.
In the currently preferred embodiment, various authoring tools may be used. In one instance the topic rontPnt~ may be generated as a set of instrl~ctioIl~ The topic contents may also be generated via a What You See Is What You Get (WYSIWIG) tool. In this case, the panels are generated as the would be displayed. Of course different object generation means would be used for each instance. Other techniques for creating the topic cont~nt~
used to generate the ~i~tAh~se would be apparent to one skilled in the art and would not depart from the spirit and scope of the present invention.
The fi~t~h~se is conceptually comprised of resources and data. The resources are the data objects. The data is the actual text, graphical or video data that is displayed. Data objects are retrieved by an object type and an i~lentific~tion number ("id"). In the currently ~ler~led embo~im~nt, the object type is a 4-character string and the object id is an integer number.
Table A lists a set of "bllilding-block" data objects that are available for thepresPnt~tion window in the currently ~rer~lled embodiment of the present invention. The list is not meant to be exhaustive, but merely representative of commonly used objects.
S~iBSTIT~T~ ~HEET (RIJLE 26) ~WO 94/27237 2 1 6 1 ~ 2 3 PCT/US94/04991 TABLE A
DATA OJBECI' LIST
Category Type Label Description Topic acPT kResTopic Topic script Panel ppUA kResPanel Panel Content pclX; kResSTxt Text Content .pcTF; kResSTxtF Full panel text Content pcTB; kResSTxtB Body text Content pcTT; kResSTxtT Tag text Content pcGA; kResGraphic Graphic Content pcGF; kResGraphicF Full panel graphic Content pcPR; kResPrompt Prompt Content pcCI; kResCtrlItem Control item Content pcMO; kResMovie Movie Content pcMC; kResMovieC Movie with controller Qualifier pqCN kShowIf Show panel if ~onditions are true.
Qualifier pqCS kSkipIf Skip panel if ,,onditions are true.
Qualifier pqMS kResMakeSure Make sure the conditions are true ~efore showing this panel. If not, show the Oops topic.
Qualifier pqCN kResCondition Condition Context pxCA kResContext ' he specification for a context check.
In the currently preferred embo~in ent, each object is defined as a ~A~intosh resource. A MA~intosh resource is code or data that may be used by various appli~ ~tion~. Macintosh resources as described in the S~JBS~ITUTE SHEET (RULE 26) WO 94127237 . PCT/US94/04991 publication entitled "Inside MArintosh Volume 1" published by Addison-Wesley Publishing Company, Inc. The actual resource definition for many of the objects described in Table A is provided in Appendix A. These resource definitions are exemplary of how the resource may be ~l~fin~1 and is not meant to be exhaustive.
Referring back to Table A, the first column Category in~ AtPS the general category of the data object, namely, Topic, Panel, Content, Action, Ql-~lifier or Context. The second column Type lists the object type. The third column is the object label. The object label is an alternative way to le~~ ce the object and is used by the authoring tools and by the delivery engine implementation code (C++). Finally, the fourth column is a brief description of the use for the particular object.
Operation of the Delivery Engitte In the currently ~re~led of the present invention, elements are used only when they are nee~ip~l Before starting the instruction (displaying panels), the delivery engine must set-up the topic to be viewed.
The steps for setting-up a topic are described with respect to the flowchart of Figure 6. Referring to Figure 6, the topic a is retrieved from ~e ~l~t~h~ce and is inst~ntiAte~l as an object, step 601. By inct~nti~tirlg as an object it is meant to create an instance of its class. The topic object contains general topic inform~tion and a list of panels associated with the topic. Each panel that is linked to the topic (i.e. in the topic's list of panels) is then retrieved from the database and also inst~nti~te~l as an object, step 602. The retrieved panels each cont~in a list of panel objects. Each panel object is then retrieved from the database and inst~nti~t~d as an object, step 603. Only the panel object description itself is retrieved and instantiated, not the content of the panel object. For example, a text panel object conPin~ a r~ ence to the text co~t~nt and the location or format of the text on the panel. The text content is not retrieved until the panel is displayed.
Next, each qll~lifipr linked to the panel is retrieved and instantiated as an object, step 604. The qualifier is attached to the panel object as a collaborator. A collaborator is an independent object that works to support, aid or assist another object. Finally, if any panel objects follow the panel in SlJBSTITUT~ SI~EET (RlJL~ 26) ~WO 94/27237 2 16 ~ ~ 2 3 PCT/US94/04991 the topic list (as distinguished from the panel list), they are retrieved and instantiated as objects, step 605. These objects are appended to the panel that was previously instantiated. This allows a panel descriptor to be modified within the topic. For example, prompts can be appended to a panel from the topic. At this point the topic of instruction is ready to be delivered.
The delivery of panels in the during the course of instruction is described with respect to the steps of the flowchart in Figure. Referring to Figure 7, once all the panels and panel objects have been retrieved and inst~nti~te.1, the delivery sequence begins with the first panel whose q~ ifiers determine that it can be shown or which has no ql]AlifiPrs step 701. The m~nnPr in which the qualifiers are Px~minerl is described below with refe~ ce to Figures 8 and 9. Here it is assumed that a first panel has been iclPntifie~l The panel object contents are then retrieved, step 702. As noted above, only the panel object descriptors were initially retrieved. The panel object ro~t~ntc are the text, graphics or video data that is to be displayed on the panel. Once the panel contents are retrieved, they are displayed in the panel display area of the presPnt~tion window, step 703. In this display step, the display of the objects may be formatted by the author or by the delivery engine by the author's choice of object type for the data.
At this point, a panel is displayed and the author may provide the user with various options. One option that an author may provide to the user is a go-back option. In the go-back option the user may jump back to a previously displayed panel. The go-back option is described below in more detail with respect to Figure 10. The other options are to traverse backwards or forwards through the current list of panels which is described as follows. These options are provided as part of the prPcPnPtion window functionality (see Figure 4). Referring back to Figure 7, a determin~hon is made by the delivery engine whether or not there are any more forward (next) panels to be displayed, steps 704, 705. This detern~in~tion is made transparent to the user while they are ex~mining the current panel. Lf no more panels are to be displayed in the forward direction, the "next" option is ~lic~klell~ step 706. In this instance, the user options are limite~l to showing a previous panel or to end the instruction, step 707. If the user SUBSTITUTE SI~EET (RUL~ 26~
2~ ~423 1 6 chooses to end instruction, the instruction is ter~in~te~l Otherwise, the previous panel is displayed, step 710.
Responding in the affirmative to the question of more panels to be displayed of step 705 causes the "next" and "previous" options to be enabled. The user must respond by sPlectin~ one of the options, step 708. If the "next" option is sPlPcte~, the next eligible panel is displayed, step 709.
The next eligible panel is i~lentifie~l by the state of the qualifiers associated with the next sllcces~ive panels (or by the lack of qualifiers for a panel).
The delivery engine then repeats the process by determining if any other panels remain to be displayed (step 704). If the previous option is sPlPcte~, the previous panel is displayed, step 710 and the process repeats with the delivery engine determining if any other panels remain to be displayed (step 704).
From the foregoing description it is apparent that the delivery engine m~int~ins a display list of panels that have been displayed. This display list facilitates the traversing of the panels using the go-back option and the make sure qualifier (as will be described below).
Evn~trrfion of Qualifiers The qll~lifiprs that may be associated with a panel f~rilit~te the interactive nature in which the panels of a topic may be displayed. The qll~lifiPrs and the con~litions referred to therein provide for control of its flow of panel display. Qll~lifiprs are evaluated by the delivery PngirlPs before each panel is shown and in the background while the user is not interacting with the delivery er~gine. Qualifiers can be based on user controls within any panel, not nece5sArily the one being currently shown, or on a test of the user's context. For example, if the instruction is to display a short cut terhnique for yelro~lning a function or a long way terhnique for performing a filnction, a user may be given the option of ~lerifling which way they want to learn to y~lrolm the function in a panel X. Assume that the panels for the instruction are linked seqllenti~lly as panels X through panels X+m. Further assume that the panels X+1 to X+n show the short cut terhni(lue, while the panels X+n + 1 to X + m show the long way technique. If the user at panel X chooses the to proceed by SUBSTI~TE SI~E~T (RUL~ 26) ~wo 94/27237 2 1 6 1 4 2 3 PCT/US94/04991 learning the long way technique, the panels X+1 to X+n (colles~onding to the short cut technique) would have qllAlifiers which would cause them to be skipped.
The evaluation for the show-if and skip-if qllAlifiPrs is described with rererence to Figure 8. Referring to Figure 8, if the panel has a Show-If or Skip-If qll~lifiPr, retrieve the condition for the qll~lifier from the ~lAtAkAce and instantiate it as an object, step 801. The condition is then evaluated, steps 802, 803. The condition may consist of a user control, a context check, or another coIl-lition These PleTnPnts are paired with and/or logic (author's choice) and can be nested in~lefinitely. Hence, the Show-Lf or Skip-Lf qll~lifi~r is a binary decision tree. The tree is evaluated.
If the eventual result is false, the panel will be skipped (Show-If) or shown (Skip-If), step 804. Lf the eventual result is true, the panel will be shown (Show-If) or skipped (Skip-Ifl, step 805.
A third qllAlifier is the Make-Sure qllAlifi~r. The Make-Sure qllalifiPr provides for the switching of topics (e.g. for r.ome~i~l instruction or a notice that the user must go back to a prior step) and as the criteria checked for in a go-back operation. The former instance of the Make-Sure ql~lifi~r is described with rer~ ce to Figure 9. First, the cc~n~liho~c for the qll~lifi~r are retrieved from the ~ AkAce and in.ctanhated it as an object, step 901. The retrieved panel conditions are then evAl-~Ate~, step 902 and 903. The con~lition is evaluated in the same way as the Show-If or Skip-If, but the result is used differently. If the result is false, another topic is A--tomatically shown, step 904. The i-lentifi~ ~tion of which topic is to be shown is ccntAine~l in the Make-Sure descriptor (See Appendix A for description). The first Make-Sure qualifier that yields a false result has its topic shown. If the result is true, then a check is made for Afl~litionAl Make-Sure qllAlifi~r5, step 905. Lf more Make-Sure qualifiers exist for the panel, they are evaluated per step 901. If there are no further Make-Surer qllAlifi~rs and all the Make-Sure qllAlifiers associated with the panel have evaluated to true, the panel may be shown, step 906.
The Make-Sure qualifier as used with the go-back option is described with rer~ ce to Figure 10. Referring to Figure 10, the user is presented SIJBS~U~ E~ ~R~E ~
wo 94/27237 PcT/uss4/04991 21~1423 1 8 with a "go-back" option by the author and the user elects to take it, step 1001. In this instance, the panels previously shown are searched in reverse order until a panel is found with no qualifiers or with qll~lifiPrs whose conditions (Skip-If, Show-If, Make-Sure) allow it to be shown, step 1002.
The panel is then displayed, step 1003. The Make-Sure ql-~lifi~r allows an author to specify a set of conditions (a Make-Sure) which, if changed, automatically provide a warning. The engine can then search back to find where in the sequence the conditions were acceptable, allowing the user an opportunity to redo the conditions correctly.
Context rherking iS an optional element of a conc1i~ior~l operation.
Note that a condition can specify one or more combinations of the fol!owing:
A panel control item such as a radio button or checkbox A context-check sperifi.-Afion Other con-litionc A context-check sperifi~tion is a resource (object) which, when evaluated in light of the user's context, results in a true or false. For example, ~ecking for an open "window" is a context-check specification.
Other context checks which can be performed in the currently ~ier~l~ed embo~im~nt include:
Window <title> is open and front;
Window <title> is open but invisible;
Application <name> is open <running>;
Application <name> is open and in front.
Switchittg of Topics SVBSTITUTE SHEET (R~IL~ 261 ~ 2161~23 As noted above, topics may be switched via the Make-Sure option of a panel. Moreover, in the currently ~refelled embodiment, multiple topics can be shown simultaneously. The author may elect to present a new topic while still displaying the old topic, close the old topic and replace it with a new one, or hide the old topic and show a new one (called an "Oops"
topic). A hi~lllen topic is automatically shown again when the "Oops" topic is closed. This is made possible by a displayed panel list maintained by the delivery engine.
Of course, whenever a topic switch the topic set-up described with refer~llce to Figure 6, must be performed.
Example An example of the present invention is now provided. Figures 11a-11c are the panels for a topic to be authored. In this example, instru*ion for performing certain operations on a computer system is being provided.
Here, the instruction may alternatively be provided via textll~l instrllctior or via a video showing the instruction being carried out. So if the instruction was for rlosin~ a window, the user has the option of reading how to m~nel1ver the cursor to point to a close box and then rlirking a button on the cursor control device, or simply watching the action being ~el~orl~led.
Referring to Figure 11a, a panel 1101 has two objects; a text object 1120 and a control object 1130. The text object 1120 will co~t~in the text which describes the topic of instruction. The control object 1130 is a checkbox by which a user may specify that they desire video instruction.
Figure 11b illustrates a panel 1102 which has one object namely a full panel text object 1121 which describes how to perform the desired operation.
Figure 11c illustrates a panel 1103 which has one object, namely a video object which is a move which shows the desired operation being carried out.
-Assuming that topic set-up has occurred, the panel 1101 will be displayed. The user is pr~sPnted with the option of having the instruction by text or video. Video instruction is specified by r~erking the VIDEO
SUB~TI~IJT~ SHEET (~U~E 26) wo 94/27237 PcT/uss4/04991 ,~
INSTRUCTION checkbox. Upon going to the next panel via the panel traversal operators of the pres~n~tion window, the panel 1102 will be displayed if the checkbox is not checked and the panel 1103 will be displayed if the checkbox is checked.
Figure 12 contains pseudo-code exemplary of how the foregoing example may be encoded using the objects described in Table A above.
Referring to Figure 12, line 1 contains the topic object definition. Lines 2-4 con~in the deffnition of the panel object cont~nt~, i.e. the text and the video data. Line 5 starts the definition of panel 1101. The objects associated with panel 1101 are on lines 6 and 7 (inll~nte~ for clarity), a text object and checkbox object, respectively. Line 8 contains the definition for the checkbox control object. Note that whereas the checkbox object is re~l.ced by panel 1101, it is rlPfinp~ outside of the object. This allows the checkbox object to be llfili7etl my multiple panels. Further note that the panel object contents may also be defined within the panel ~i~finition in which the object is used.
Line 9 begins the definition of panel 1102. The panel 1102 includes a text object and Skip-If qualifier object, lines 10 and 11, respectively. At line12, the qll~lifi~r object linked by panel 1102 is ~l~fine~ Note that the Ski~If qll~lifiPr object links to the checkbox object as a rcn~lition (line 13).
Line 14 begins the definition of panel 1103. The panel 1103 includes a video object and a Show-If Qll~lifipr object, lines 15 and 16, respectively.
Finally, the Show-If qualifier object is ~l~fine~ at lines 17-18. As in the caseof the Skip-If qllAlifier, the Show-If qll~lifiPr links to the checkbox object as a co~ ;on (line 18).
Figure 13 illustrates the linkages between the objects in the example.
Referring to Figure 13, the topic 1100 would be linked to the panels 1101, 1102 and 1103. The panel 1101 would have links to the text 1120 and control 1130. As is apparent, multiple objects will link to the control 1130.
The panel 1102 links to text object 1121 and Skip-IF qualifier 1160. The qualiffer 1160 in turn links to the control 1130. Finally, the panel 1103 links SUBST~TUTE S~EE i (~IJL~ 26) ~WO 94/27237 21 614 2 3 PCT/US94/04991 to video object 1140 and to Show-If qualifier 1170. The Show-If Qll~lifier in turn links to control 1130.
During set-up of the topic, the objects are loaded as described above with refe~ ce to Figure 6. Delivery of the panels would then occur in the manner as described with rerer~l,ce to Figures 7-10.
Thus, a method and apparatus for interactive delivery of ml1ltinlefli~ data for display on a computer system, is described.
SfJBSTITlJTE SHEET ~RULE 26) WOg4/27~7 . PCT~S94/04991 ~1~1423 22 ~ PENDIX A
// Copyright ~ 1993 Apple Computer, Inc. (17 U.S.C. 401) All rights reserved.
// Date: 05-May-93 // Delivery Engine Data Types // Coach Mark resouce type kResCoachMark {
longint none=kTargetNone, // target application signature self=kTargetSel~, front=kTargetFront, topic=kTargetTopic;
longint balloon='ndex', location='mLoc', menu='mMrk', object='draw';
// event id integer = $$Count of(keys);
array keys {
switch {
case Number:
key longint='iNum';
pstring; // balloon or dialog or menu item number case Origin:
key longint='oCon'; // origin reference for rectangle pstring topLeft="0", bottomLeft="1", topRight-"2", bottomRight="3";
case Rect:
key longint='iRct'; // marking rectangle pstring; // "rct=" top,left,bottom,right case Macro:
key longint='lMac'; // location macro pstring GrowBox="GrowBox", ZoomBox="ZoomBox", CloseBox="CloseBox", TitleBar="TitleBar", BootDisk="BootDisk";
case WindowName:
SUB~TI~UTE SHEET (RU~E 2~) ~ W094/27~7 21~ 1'12 3 PCT~S94/04991 key longint='wNam';
pstring; // window name case FrontWindow:
key longint='wNam';
pstring=" _ FRONTWINDOW _ "; // window name case DesktopWindow:
key longint='wNam';
pstring=" _ DESKWINDOW _ "; // window name case BalloonNumber:
key longint='iNum';
pstring; // balloon item number case DialogItemNumber:
key longint='iNum';
pstring; // dialog item number case MenuTitleName:
key longint='mNam';
pstring; // menu title name case MenuTitleNumber:
key longint='mNum';
pstring; // menu title number case MenuItemName:
key longint='iNam';
pstring; // menu item name case MenuItemNumber:
key longint='iNum';
pstring; // menu item number case ObjectName:
key longint='iNam';
pstring; // menu item name case MarkStyle:
key longint='sNam';
pstring red_circle="defaultl", - yellow x="default2", red_underline="default3", red_arrow="default4"; // mark style name case MenuItemColor:
S! IBSTITUTE SHEET ~RULE 26) wog4n7~7 PCT~S94/04991 ~1~142~ 24 key longint='mCol';
pstring myblack="l", yellow="2", // myblack instead of black // because black is defined in ViewTypes.r magenta="3", red="4", cyan="S", green="6", blue=~'7", whitez~8~; // menu item color number case MenuItemStyle:
key longint='mFac';
pstring cc_bold="l", cc_italic="2", cc_underline="4", cc_outline="8", cc_shadow="l6", cc_condense="32", cc_extend="64"; // font style number case StyleOption:
key longint='sOpt';
pstring; // "Lng=0,0,S,F" where S is start // and F is finish // l=NW, 2=N., 3=NE, 4=E, ... 8=W
}; // switch }; // array align word;
} ;
// Topic type kResTopic {
longint none=0; // target application signature integer none-kNone, noButton=kNone, defaultButton=kDefault, huhButton=kHuhBtnId, goStartButton=kGoStartBtnId; // optional button #l id integer none=kNone, noButton=kNone, defaultButton=kDefault, huhButton=kHuhBtnId, goStartButton=kGoStartBtnId; // optional button #2 id integer none=kNone, noButton=kNone, SlJBSTl~UTE SHEET (RU~E 26!
W094/27~7 2 1 6 14 2 3 PCT~S94/04991 defaultButton=kDefault, huhButton=kHuhBtnId, goStartButton=kGoStartBtnId; // optional button #3 id integer none=kNone, noMinimum=kNone, defaultMinimum=kNone; // minimum panel height integer none=kNoMaxHeight, noMaximum=kNoMaxHeight, default~x;mllm=kNoMaxHeight; // maximum panel height integer; // topic title TEXT ID number integer; // reserved field, always enter 0.
integer; // reserved field, always enter 0.
integer; // reserved field, always enter 0 integer = $$Countof(entries);
array entries {
longint; // entry type lonqint; // resource id or other };
};
// Panel type kResPanelUnivA {
integer = $$CountOf(items);
array items {
longint; // scope integer; // resId (DBRef) for scope }; // array }; // type // Text format- for full, tag, body, and custom formats.
type kResFormat {
integer teFlushDefault=0, teCenter=l, teFlushRight=-l, teFlushLeft=-2; // justification SIJ~STITUTE SHEET (RULE 26) W094l27237 PCT~S94/04991 integer normal=0; // face integer; // size integer; // top integer; // left integer; // bottom integer; // right pstring; // font name align word; // Espy Serif, Espy Sans, Espy Serif Bold....
} ;
// Text item type kResSTxt {
integer; // resId for TEXT/styl integer; // top integer; // left integer; // bottom integer; // right } ;
// Graphic item type kResGraphic I
integer; // resId for PICT
integer; // top integer; // left } ;
// Movie item type kResMovie {
integer; // top integer; // left pstring; // movie file name align word;
} ;
SU~STITUTE SHE~T (R~LE 26) W094/27~7 21 614 2 ~ PCT~S94/04991 .
// Ctrl item // Standard Macintosh controls type kResCtrlItem {
switch {
case PushButton:
key int = pushButProcReno;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case RadioButton:
key int = radioButProcReno;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case CheckBox:
key int = checkBoxProcReno;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title SLlBSTITllTE S~EET (~L~ 26) W094/27~7 PCT~S94/04991 2161~3 28 case PushButtonSysFont:
key int = pushButProcRenoSysFont;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case RadioButtonSysFont:
key int = radioButProcRenoSysFont;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title case CheckBoxSysFont:
key int = checkBoxProcRenoSysFont;
integer active=0, inactive=255;
integer unchecked=0, checked=l;
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEvent Id number pstring; // title }; // switch align word;
}; // type // Apple Event SUBSTITIJTE SI~ET (RIJL~ 2~?
W094/27~7 2 1 ~ 1~ 2 ~ PCT~S94/04991 type kResEvent {
longint none=kTargetNone, // target application signature self=kTargetSelf, front=kTargetFront, topic=kTargetTopic, reno=kTargetReno;
longint; // class longint none=0; // event id integer none=0; // optional short integer with key 'IOPT' longint none=0; // optional key pstring none=""; // data for optional key align word;
} ;
// Condition // Referenced by Show-If, Skip-If, and Make-Sure.
type kResCondition {
integer none=0, not=kNot; // optional not or kNot integer update=kUpdate, dontUpdate=kDontUpdate;
longint; // type (EG kResCtrlItem, kResContext, kResCondition) integer; // resId integer none=kNone, or=kOr, and=kAnd; // logic integer none=0, not=kNot; // optional not or kNot integer update=kUpdate, dontUpdate=kDontUpdate;
longint none=0; // type integer none=0; // resId } ;
// Make-Sure type kResMakeSure I
integer; // resource Id of kResCondition resource.
integer; // resource Id of kResTopic resource for Oops.
} ;
// Prompt SUBST~UTE S~E~T (RIJ~ 26) W094/27~7 PCT~S94/04991 2 ~ 3 0 type kResPrompt {
integer; // Id of first (beginning) TEXT/styl resource integer; // Id of second (middle) TEXT/styl resource integer; // Id of third (end) TEXT/styl resource integer; // Id of fourth (choice) TEXT/styl resource } ;
// Hot panel area type kResHot ~
integer; // top integer; // left integer; // bottom integer; // right integer none=0; // action, kResEventId number } ;
// Custom button type kResRenoBtn {
integer=1; // (always momentary) integer; // off-up PICT id integer; // on-down PICT id integer=0; // (dimmed not available) longint none=0; // action, kResEvent id number integer=l; // (not used, kResEvent is always immediate) integer; // top of top-left corner location integer; // left of top-left corner location integer=0; // (always fixed) integer none=0; // string index for Help Balloon text.
} ;
// PreWin button SU~STITUTE SHE~T (RULE 26) W094l27~7 21614 2 ~ PCT~S94/04991 // Placed in the control area along the bottom // of the presentation window.
type kResPreWinBtn {
integer=l; // ~always momentary) integer; // off-up PICT id integer; // on-down PICT id integer; // dimmed longint none=0, // action, kResEvent id number integer=l; // (not used, kResEvent is always immediate) integer=0; // top of top-left corner location is preset integer=0; // left of top-left corner location is preset integer align=l, movable=l, fixed=0; align (l) or fixed (0) integer none=0; // string index for Help Balloon text.
} ;
// PerWin Button.
// Placed in the control area alo~g the bottom // of the presentation window.
type kResPreWinBtn {
integer=l; // (always momentary) integer; // off-up PICT id integer; // on down PICT id integer; // dimmed longint none=0; //action, KResEvent id number integer=l; //not used, kResEvent is always immediate) integer=0; // top of top-left corner location is preset integer=0; // left of top-left corner location is preset integer align=l, moveable=l, fixed=0; // align (l) or fixed (0) SUBSTl~U~ S'~ET (RULE 26) W094/27~7 PCT~S94/04991 21~1423 32 integer non=O; // string index for Help Ballon text.
} ;
type kResContext {
longint none=kTargetNone, // target Process signature self=kTargetSelf, front=kTargetFront, topic=kTargetTopic, reno=kTargetReno;
switch {
case Chooser:
key longint = kCntxChooser;
switch {
case isPrinterDirect:
key longint = 0;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isPrintersOnZone:
key longint = 1;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isPrinterSerial:
key longint = 2;
longint ModemPort=0, PrinterPort=l, EitherPort=2;
fill byte[12]; // filler for 2 longs & 2 StringSpecs SUBST~TU~E ~HE~ ~ (R~LE 26) W094/27~7 PCT~S~ 1991 33 2i~1~23 case isPrinterSCSI:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isPrinterType:
key longint = 4;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches-10, contains=11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Printer name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isAppleTalk:
key longint = 5;
longint off=0, on=1;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isNumberOfZones:
key longint = 6;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isPortSelected:
key longint = 7;
longint ModemPort=0, SUBSTITUT~ SHEET (R~ILE 26) W094/27~7 PCT~S94/04991 2 ~ 2 3 3 4 PrinterPort=l, EitherPort=2;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isPortInUse:
key longint = 8;
longint ModemPort=0, PrinterPort=l, EitherPort=2;
fill byte[12]; // filler for 2 longs ~ 2 StringSpecs }; // Chooser switch case Dialog:
key longint = kCntxDialog;
switch {
case isFront:
key longint = 0;
longint; // Resource ID
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isOpen:
key longint = l;
longint; // Resource ID
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isInvisible:
key longint = 2;
longint; // Resource ID
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isFindDialog:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isFindMoreDialog:
key longint = 4;
SUBSTITIJT~ S!~E~T (RULE 2~) W094/27~7 21 6 1~ ~ 3 PCT~S94/04991 fill byte[16]; // filler for 3 longs & 2 StringSpecs case isAnyDialogActive:
key longint = 5;
fill byte[16]; // filler for 3 longs & 2 StringSpecs }; // Dialog switch case File:
key longint = kCntxFile;
switch {
case isFrontStartup:
key longint = 0;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isOpenStartup:
key longint = 1;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isCDStartup:
key longint = 2;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isPrintMonitor:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isAppleShare:
key longint = 4;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isEtherTalk:
key longint = 5;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isTokenTalk:
key longint = 6;
fill byte[16]; // filler for 3 longs & 2 StringSpecs Sl~BST)~lJT~ SHEET (Rl~E 26) W094/27~7 PCT~S94/04991 2~-1423 36 case isControlPanel:
key longint = 7;
fill byte[8]; // filler for 2 longs pstring; // Nameof Control Panel align word;
fill byte [6]; // filler for 1 long & 1 StringSpec case isFileExists:
key longint = 9;
longint SystemFolder='macs', DesktopFolder='desk', TrashFolder='trsh', EmptyTrashFolder='empt', PrintMonitorFolder='prnt', StartupFolder='strt', AppleMenuFolder='amnu', ControlPanelFolder='ctrl', ExtensionsFolder='extn', PreferencesFolder='pref', TemporaryFolder='temp'; // Sy$tem folder selector fill byte[4]; // filler for 1 long pstring; // Name of desired file align word;
fill byte[6]; // filler for 1 long & 1 StringSpec }; // File switch case Finder:
key longint = kCntxFinder;
fill long; // Finder selector fill byte[16]; // filler for 3 longs & 2 StringSpecs case Process:
key longint = kCntxProcess;
switch {
case isFront:
St~B~ITUTE SH~T (t~ 26) ~W094/27~7 2 ~ ~ 14 2 3 PCT~S94/04991 key longint = 0;
longint; // Process signature fill byte[l2]; // filler for 2 longs & 2 StringSpecs case isOpen:
key longint = l;
longint; // Process signature fill byte[12]; // filler for 2 longs & 2 StringSpecs }; // Process switch case System:
key longint z kCntxSystem;
switch {
case gestaltCheck:
key longint 5 0;
longint; // Gestalt Selector longint equals=2, notEqualTo=3, greaterThan=4, lessThan-5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector fill byte[2]; // not used longint; // Gestalt compare value fill byte[2]; // filler for 1 StringSpec case menuItemExists:
key longint z 1;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
SUBSTITUT~ SHE~T (RI~LE 26) W094/27~7 PCT~S94/04991 longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case menuItemMarked:
key longint = 2;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith-13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
longint containsAnything=8, empty=9, matches=10, contains=11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case menuItemF.n~hled:
key longint = 3;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, SUB~TITlJTE S~E~T (RlJLE 26) W094/27~7 2 1 6 14 2 ~ PCT~S94/04991 startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case menuItemDisabled:
key longint = 4;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=lO, contains=11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu name (variable-length) align word;
longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Menu Item name (variable-length) align word;
case isFileSharing:
key longint = 6;
longint off=0, ~3~jB~ HEET (~ULE 26) W094/27~7 PCT~S94/04991 on=1;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isVideoHWBitDepth:
key longint = 7;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isMonitorBitDepth:
key longint ~ 8;
longint equals=2, notEqualTo=3, greaterThan=4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo=7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs case isComputerNamed:
key longint = 9;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains-11, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Printer name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec SllB~T~UTE ~ T (RUL~ 2~
W094/27~7 2 1 6 1~ Z ~ PCT~S94/04991 case isGuestAccess:
key longint = 10;
longint off=0, on=l;
fill byte[12]; // filler for 2 longs & 2 StringSpecs case isNumberMonitors:
key longint = 11;
longint equals=2, notEqualTo=3, greaterThan-4, lessThan=5, greaterThanOrEqualTo=6, lessThanOrEqualTo-7; // Number compare selector longint; // Compare value fill byte[8]; // filler for 1 long & 2 StringSpecs }; // System switch case Window:
key longint = kCntxWindow;
switch {
case isFront:
key longint = 0;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=ll, startsWith=12, endsWith=13; // StringCompareMethod pstring; // Window name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isOpen:
key longint = l;
SUBSTITUTE SHEET (RULE 26) W094/27~7 PCT~S94/04991 2 ~ 2 3 4 2 fill byte[4]; // not used longint containsAnything=8, empty=9, matches=10, contains=11, startsWith=12, endsWithc13; // StringCompareMethod pstring; // Window name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isInvisible:
key longint = 2;
fill byte[4]; // not used longint containsAnything=8, empty=9, matches~10, contains-11, startsWith312, endsWith-13; // StringCompareMethod pstring; // Window name (variable-length) align word;
fill byte[6]; // filler for 1 long & 1 StringSpec case isShareWindowOpen:
key longint = 3;
fill byte[16]; // filler for 3 longs & 2 StringSpecs case isShareWIndowActive:
key longint 2 4;
fill byte[16]; // filler for 3 longs & 2 StringSpecs }; // Window switch }; // switch }; // type SU3Sr1TUTE Sl IEET (RULE 26)
Claims (15)
1. A method for interactive delivery of multimedia data in a computer controlled display system, said method comprising the steps of:
a) authoring a topic, said topic comprised of a series of panels, media elements, and flow control elements;
b) generating a database from said topic, said database comprised of a plurality of topic, panel, media and flow control objects;
c) setting-up said topic for display by retrieving said topic, panel, media and flow control objects from said database;
d) identifying a first panel to be displayed;
e) retrieving the media content for said first panel to be displayed; f) displaying said first panel;
g) identifying a second panel to be displayed by evaluating said flow control objects;
h) retrieving the media content for said second panel to be displayed; and i) displaying said second panel.
a) authoring a topic, said topic comprised of a series of panels, media elements, and flow control elements;
b) generating a database from said topic, said database comprised of a plurality of topic, panel, media and flow control objects;
c) setting-up said topic for display by retrieving said topic, panel, media and flow control objects from said database;
d) identifying a first panel to be displayed;
e) retrieving the media content for said first panel to be displayed; f) displaying said first panel;
g) identifying a second panel to be displayed by evaluating said flow control objects;
h) retrieving the media content for said second panel to be displayed; and i) displaying said second panel.
2. The method as recited in Claim 1 wherein said step of identifying a first panel to be displayed is further comprised of the steps of:
a) examining a first panel in said series of panels to determine if has at least one flow control element;
b) if said first panel in said series of panels does not have a flow control object, identifying said first panel as the first panel to be displayed;c) if said first panel in said series of panels does have a flow control object, evaluating said flow control objects;
d) if said flow control objects yield a first state, identifying said first panel as the first panel to be displayed;
e) if said flow control objects yield a second state, skipping said panel; and f) continuing steps a)-e) for said series of panels until a panel is found which either has no flow control elements or said flow control objects yield said first state.
a) examining a first panel in said series of panels to determine if has at least one flow control element;
b) if said first panel in said series of panels does not have a flow control object, identifying said first panel as the first panel to be displayed;c) if said first panel in said series of panels does have a flow control object, evaluating said flow control objects;
d) if said flow control objects yield a first state, identifying said first panel as the first panel to be displayed;
e) if said flow control objects yield a second state, skipping said panel; and f) continuing steps a)-e) for said series of panels until a panel is found which either has no flow control elements or said flow control objects yield said first state.
3. The method as recited in Claim 2 wherein said step of evaluating said flow control objects is further comprised of the steps of:
a) identifying conditional elements associated with said flow control elements;
b) determining the logical values of said conditional elements;
c) performing any necessary logical operations on said logical values to derive a state for said flow control objects.
a) identifying conditional elements associated with said flow control elements;
b) determining the logical values of said conditional elements;
c) performing any necessary logical operations on said logical values to derive a state for said flow control objects.
4. The method as recited in Claim 1 wherein said step of identifying a second panel to be displayed by evaluating said flow control objects is further comprised of the steps of:
a) identifying flow control objects associated with a next panel in said series of panels;
b) identifying conditional elements associated with said flow control elements;
c) determining the logical values of said conditional elements;
d) performing any necessary logical operations on said logical values to derive a state for said flow control objects;
e) repeating steps a)-d) until a panel having flow control objects in said first state is identified.
a) identifying flow control objects associated with a next panel in said series of panels;
b) identifying conditional elements associated with said flow control elements;
c) determining the logical values of said conditional elements;
d) performing any necessary logical operations on said logical values to derive a state for said flow control objects;
e) repeating steps a)-d) until a panel having flow control objects in said first state is identified.
5. The method as recited in Claim 1 wherein said flow control objects further causes a topic switch when said flow control object is evaluated to be in said second state.
6. A computer display system having a central processing unit (CPU) coupled to a display, comprising: topic authoring means for creating a topic, said topic comprised of a series of panels, each of said panels further comprised of panel elements; database generation means for generating a database from a topic, said database comprised of a plurality of topic objects, panel objects and panel objects; topic delivery means for delivering panels of a topic for display in an order based on user input and further based on context; and media content retrieval means for retrieving media data corresponding to panel objects for display on said display.
7. The computer display system as recited in Claim 6 wherein said panel objects is further comprised of video objects, text objects, graphics objects, qualifier objects and condition objects.
8. The computer display system as recited in Claim 7 wherein said topic delivery means is further comprised of:
object retrieval means for retrieving objects from said database for setting up delivery of said topic;
qualifier evaluation means for determining a logical state for said qualifier objects; and next panel determination means for determining the next panel to be displayed based on the logical state of said qualifier objects for sicceedingpanels.
object retrieval means for retrieving objects from said database for setting up delivery of said topic;
qualifier evaluation means for determining a logical state for said qualifier objects; and next panel determination means for determining the next panel to be displayed based on the logical state of said qualifier objects for sicceedingpanels.
9. The computer display system as recited in Claim 7 wherein said topic delivery means is further comprised of:
topic switching means for switching topics responsive to a first qualifier object for a panel being displayed being in a first predetermined logical state.
topic switching means for switching topics responsive to a first qualifier object for a panel being displayed being in a first predetermined logical state.
10. The computer system as recited in Claim 9 is further comprised of panel listing means for maintaining an ordered list of panels previously displayed.
11. The computer system as recited in Claim 6 is further comprised of panel n display formatting means for specifying the arrangement of display of media data on a panel.
12. The computer system as recited in Claim 6 wherein said topic authoring means is a What You See Is What You Get application.
13. In a computer controlled display system, a method for interactive display of multimedia data comprising the steps of:
a) organizing multimedia data to be displayed into a topic comprised of a sequence of panels;
b) for each panel, specifying multimedia elements and flow control elements linked to said panel;
c) generating a database from said topic to create a set of topic objects, panel objects, multimedia data objects and flow control objects;
d) retrieving objects corresponding to a topic to be viewed; and for each panel in said sequence of panels performing the steps of:
e) determining that a panel can be displayed based on the state of flow control objects linked to said panel; and f) displaying the multimedia elements for said panel.
a) organizing multimedia data to be displayed into a topic comprised of a sequence of panels;
b) for each panel, specifying multimedia elements and flow control elements linked to said panel;
c) generating a database from said topic to create a set of topic objects, panel objects, multimedia data objects and flow control objects;
d) retrieving objects corresponding to a topic to be viewed; and for each panel in said sequence of panels performing the steps of:
e) determining that a panel can be displayed based on the state of flow control objects linked to said panel; and f) displaying the multimedia elements for said panel.
14. The method as recited in Claim 13 wherein said flow control objects are comprised of panel display qualifiers and condition indicators said step of determining that a panel can be displayed based on the state of flow control objects linked to said panel is further comprised of the steps of:
a) identifying a panel display qualifier associated with a panel;
b) determining the state of condition indicators associated with said panel display qualifiers;
c) evaluating the state of said panel display qualifier based on the associated condition indicators;
d) if said panel display qualifier is in a first state, identifying said panel as capable of display; and e) if said panel display qualifier in not in said first state, identifying said panel as not capable of display.
a) identifying a panel display qualifier associated with a panel;
b) determining the state of condition indicators associated with said panel display qualifiers;
c) evaluating the state of said panel display qualifier based on the associated condition indicators;
d) if said panel display qualifier is in a first state, identifying said panel as capable of display; and e) if said panel display qualifier in not in said first state, identifying said panel as not capable of display.
15. The method as recited in Claim 14 wherein said step of determining that a panel can be displayed based on the state of flow control objects linked to said panel is performed while a prior panel is being displayed.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/059,542 US5428731A (en) | 1993-05-10 | 1993-05-10 | Interactive multimedia delivery engine |
US08/059,542 | 1993-05-10 |
Publications (1)
Publication Number | Publication Date |
---|---|
CA2161423A1 true CA2161423A1 (en) | 1994-11-24 |
Family
ID=22023647
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA002161423A Abandoned CA2161423A1 (en) | 1993-05-10 | 1994-05-06 | Interactive multimedia delivery engine |
Country Status (7)
Country | Link |
---|---|
US (1) | US5428731A (en) |
EP (1) | EP0698244B1 (en) |
JP (1) | JP4021474B2 (en) |
AU (1) | AU6670794A (en) |
CA (1) | CA2161423A1 (en) |
DE (1) | DE69429860T2 (en) |
WO (1) | WO1994027237A1 (en) |
Families Citing this family (218)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5682490A (en) * | 1992-07-15 | 1997-10-28 | New Media Development Association | User adaptive parametric macro creation system and method |
US20080158261A1 (en) * | 1992-12-14 | 2008-07-03 | Eric Justin Gould | Computer user interface for audio and/or video auto-summarization |
US8381126B2 (en) * | 1992-12-14 | 2013-02-19 | Monkeymedia, Inc. | Computer user interface with non-salience deemphasis |
US5623588A (en) * | 1992-12-14 | 1997-04-22 | New York University | Computer user interface with non-salience deemphasis |
JP3175399B2 (en) * | 1993-05-18 | 2001-06-11 | セイコーエプソン株式会社 | Card data management device |
US6574350B1 (en) | 1995-05-08 | 2003-06-03 | Digimarc Corporation | Digital watermarking employing both frail and robust watermarks |
US6681029B1 (en) | 1993-11-18 | 2004-01-20 | Digimarc Corporation | Decoding steganographic messages embedded in media signals |
US6449377B1 (en) * | 1995-05-08 | 2002-09-10 | Digimarc Corporation | Methods and systems for watermark processing of line art images |
US6681028B2 (en) | 1995-07-27 | 2004-01-20 | Digimarc Corporation | Paper-based control of computer systems |
US6122403A (en) | 1995-07-27 | 2000-09-19 | Digimarc Corporation | Computer system linked by using information in data objects |
US5841978A (en) | 1993-11-18 | 1998-11-24 | Digimarc Corporation | Network linking method using steganographically embedded data objects |
US5982365A (en) * | 1993-11-19 | 1999-11-09 | Apple Computer, Inc. | System and methods for interactively generating and testing help systems |
EP0663639A1 (en) * | 1994-01-14 | 1995-07-19 | International Business Machines Corporation | Method for creating a multimedia application |
US6522770B1 (en) | 1999-05-19 | 2003-02-18 | Digimarc Corporation | Management of documents and other objects using optical devices |
US5734862A (en) * | 1994-05-31 | 1998-03-31 | Kulas; Charles J. | System for selectively buffering and displaying relevant frames from interleaving frames associated with respective animation sequences stored in a medium in response to user selection |
US6134547A (en) * | 1994-07-11 | 2000-10-17 | Muze, Inc. | Computerized method and system for user-interactive, multimedia cataloguing, navigation and previewing of film and films on video |
US5675752A (en) * | 1994-09-15 | 1997-10-07 | Sony Corporation | Interactive applications generator for an interactive presentation environment |
US8094949B1 (en) | 1994-10-21 | 2012-01-10 | Digimarc Corporation | Music methods and systems |
US6721440B2 (en) | 1995-05-08 | 2004-04-13 | Digimarc Corporation | Low visibility watermarks using an out-of-phase color |
US6760463B2 (en) * | 1995-05-08 | 2004-07-06 | Digimarc Corporation | Watermarking methods and media |
US6590996B1 (en) | 2000-02-14 | 2003-07-08 | Digimarc Corporation | Color adaptive watermarking |
US6744906B2 (en) | 1995-05-08 | 2004-06-01 | Digimarc Corporation | Methods and systems using multiple watermarks |
JP3338585B2 (en) * | 1995-05-16 | 2002-10-28 | 富士通株式会社 | Apparatus and method for converting presentation data |
JP2798119B2 (en) * | 1995-05-23 | 1998-09-17 | 日本電気株式会社 | Electronic book display device |
US5740436A (en) * | 1995-06-06 | 1998-04-14 | Apple Computer, Inc. | System architecture for configuring input and output devices of a computer |
IL114432A (en) * | 1995-07-03 | 1999-08-17 | Wiernik Ady | Multi-media method |
US6466241B1 (en) | 1995-07-17 | 2002-10-15 | Gateway, Inc. | Method and apparatus for managing program activation through use of a favorite program button menu |
US6359636B1 (en) | 1995-07-17 | 2002-03-19 | Gateway, Inc. | Graphical user interface for control of a home entertainment system |
US6829368B2 (en) | 2000-01-26 | 2004-12-07 | Digimarc Corporation | Establishing and interacting with on-line media collections using identifiers in media signals |
US6411725B1 (en) | 1995-07-27 | 2002-06-25 | Digimarc Corporation | Watermark enabled video objects |
US7051086B2 (en) | 1995-07-27 | 2006-05-23 | Digimarc Corporation | Method of linking on-line data to printed documents |
US7171018B2 (en) | 1995-07-27 | 2007-01-30 | Digimarc Corporation | Portable devices and methods employing digital watermarking |
US6577746B1 (en) | 1999-12-28 | 2003-06-10 | Digimarc Corporation | Watermark-based object linking and embedding |
US6408331B1 (en) * | 1995-07-27 | 2002-06-18 | Digimarc Corporation | Computer linking methods using encoded graphics |
US6788800B1 (en) | 2000-07-25 | 2004-09-07 | Digimarc Corporation | Authenticating objects using embedded data |
US5737552A (en) * | 1995-07-28 | 1998-04-07 | Starwave Corporation | Machine, method and medium for linear programming with interactive conversational interface |
US5659742A (en) * | 1995-09-15 | 1997-08-19 | Infonautics Corporation | Method for storing multi-media information in an information retrieval system |
US5640553A (en) * | 1995-09-15 | 1997-06-17 | Infonautics Corporation | Relevance normalization for documents retrieved from an information retrieval system in response to a query |
US5717914A (en) * | 1995-09-15 | 1998-02-10 | Infonautics Corporation | Method for categorizing documents into subjects using relevance normalization for documents retrieved from an information retrieval system in response to a query |
US5675788A (en) * | 1995-09-15 | 1997-10-07 | Infonautics Corp. | Method and apparatus for generating a composite document on a selected topic from a plurality of information sources |
US5742816A (en) * | 1995-09-15 | 1998-04-21 | Infonautics Corporation | Method and apparatus for identifying textual documents and multi-mediafiles corresponding to a search topic |
US5822731A (en) * | 1995-09-15 | 1998-10-13 | Infonautics Corporation | Adjusting a hidden Markov model tagger for sentence fragments |
US5721902A (en) * | 1995-09-15 | 1998-02-24 | Infonautics Corporation | Restricted expansion of query terms using part of speech tagging |
US5873076A (en) * | 1995-09-15 | 1999-02-16 | Infonautics Corporation | Architecture for processing search queries, retrieving documents identified thereby, and method for using same |
US7047241B1 (en) | 1995-10-13 | 2006-05-16 | Digimarc Corporation | System and methods for managing digital creative works |
US6807534B1 (en) * | 1995-10-13 | 2004-10-19 | Trustees Of Dartmouth College | System and method for managing copyrighted electronic media |
AU7330396A (en) | 1995-10-23 | 1997-05-29 | Hypermed Ltd. | Structured focused hypertext data structure |
US5925103A (en) * | 1996-01-26 | 1999-07-20 | Magallanes; Edward Patrick | Internet access device |
JP2000512039A (en) | 1996-03-15 | 2000-09-12 | ザパ デジタル アーツ リミテッド | Programmable computer graphic objects |
US5892506A (en) * | 1996-03-18 | 1999-04-06 | Discreet Logic, Inc. | Multitrack architecture for computer-based editing of multimedia sequences |
US5768581A (en) * | 1996-05-07 | 1998-06-16 | Cochran; Nancy Pauline | Apparatus and method for selecting records from a computer database by repeatedly displaying search terms from multiple list identifiers before either a list identifier or a search term is selected |
JPH1021261A (en) * | 1996-07-05 | 1998-01-23 | Hitachi Ltd | Method and system for multimedia data base retrieval |
JPH10145722A (en) | 1996-11-07 | 1998-05-29 | Sony Corp | Reproducing control data generation device and method therefor |
EP0901282B1 (en) | 1997-09-03 | 2006-06-28 | Hitachi, Ltd. | Method for recording and reproducing electronic watermark information |
US7054463B2 (en) | 1998-01-20 | 2006-05-30 | Digimarc Corporation | Data encoding using frail watermarks |
US6154752A (en) * | 1998-02-13 | 2000-11-28 | Lockheed Martin Corporation | Chronological identification of hyper text links during back tracking |
US7272298B1 (en) | 1998-05-06 | 2007-09-18 | Burst.Com, Inc. | System and method for time-shifted program viewing |
US6230162B1 (en) | 1998-06-20 | 2001-05-08 | International Business Machines Corporation | Progressive interleaved delivery of interactive descriptions and renderers for electronic publishing of merchandise |
US6356921B1 (en) | 1998-06-20 | 2002-03-12 | International Business Machines Corporation | Framework for progressive hierarchial and adaptive delivery rich media presentations and associated meta data |
US8380041B2 (en) | 1998-07-30 | 2013-02-19 | Tivo Inc. | Transportable digital video recorder system |
US6233389B1 (en) | 1998-07-30 | 2001-05-15 | Tivo, Inc. | Multimedia time warping system |
US7558472B2 (en) | 2000-08-22 | 2009-07-07 | Tivo Inc. | Multimedia signal processing system |
US8577205B2 (en) | 1998-07-30 | 2013-11-05 | Tivo Inc. | Digital video recording system |
US6448980B1 (en) * | 1998-10-09 | 2002-09-10 | International Business Machines Corporation | Personalizing rich media presentations based on user response to the presentation |
US6868497B1 (en) | 1999-03-10 | 2005-03-15 | Digimarc Corporation | Method and apparatus for automatic ID management |
US6393158B1 (en) | 1999-04-23 | 2002-05-21 | Monkeymedia, Inc. | Method and storage device for expanding and contracting continuous play media seamlessly |
US10051298B2 (en) | 1999-04-23 | 2018-08-14 | Monkeymedia, Inc. | Wireless seamless expansion and video advertising player |
US8103542B1 (en) | 1999-06-29 | 2012-01-24 | Digimarc Corporation | Digitally marked objects and promotional methods |
DE60045248D1 (en) | 1999-09-20 | 2010-12-30 | Tivo Inc | CAPTION labeling |
US6608919B1 (en) * | 1999-11-10 | 2003-08-19 | Digimarc Corporation | Method and apparatus for encoding paper with information |
US6546397B1 (en) * | 1999-12-02 | 2003-04-08 | Steven H. Rempell | Browser based web site generation tool and run time engine |
US6625297B1 (en) | 2000-02-10 | 2003-09-23 | Digimarc Corporation | Self-orienting watermarks |
US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
US7725812B1 (en) * | 2000-03-31 | 2010-05-25 | Avid Technology, Inc. | Authoring system for combining temporal and nontemporal digital media |
US7555557B2 (en) * | 2000-04-07 | 2009-06-30 | Avid Technology, Inc. | Review and approval system |
US6804377B2 (en) | 2000-04-19 | 2004-10-12 | Digimarc Corporation | Detecting information hidden out-of-phase in color channels |
US7111168B2 (en) * | 2000-05-01 | 2006-09-19 | Digimarc Corporation | Digital watermarking systems |
US8055899B2 (en) | 2000-12-18 | 2011-11-08 | Digimarc Corporation | Systems and methods using digital watermarking and identifier extraction to provide promotional opportunities |
US7266704B2 (en) * | 2000-12-18 | 2007-09-04 | Digimarc Corporation | User-friendly rights management systems and methods |
US7039643B2 (en) | 2001-04-10 | 2006-05-02 | Adobe Systems Incorporated | System, method and apparatus for converting and integrating media files |
US20020188628A1 (en) * | 2001-04-20 | 2002-12-12 | Brian Cooper | Editing interactive content with time-based media |
US7930624B2 (en) * | 2001-04-20 | 2011-04-19 | Avid Technology, Inc. | Editing time-based media with enhanced content |
US7007073B2 (en) * | 2001-06-06 | 2006-02-28 | Learning Sciences International | Method and apparatus providing electronic concurrent delivery of multimedia content to general purpose computers over a computer network |
US7063264B2 (en) | 2001-12-24 | 2006-06-20 | Digimarc Corporation | Covert variable information on identification documents and methods of making same |
US7024626B2 (en) * | 2001-11-30 | 2006-04-04 | Apple Computer, Inc. | System and method of producing user interface information messages |
WO2003055119A2 (en) * | 2001-12-06 | 2003-07-03 | New York University | Logic arrangement, data structure, system and method for multilinear representation of multimodal data ensembles for synthesis, recognition and compression |
ATE509326T1 (en) | 2001-12-18 | 2011-05-15 | L 1 Secure Credentialing Inc | MULTIPLE IMAGE SECURITY FEATURES FOR IDENTIFYING DOCUMENTS AND METHOD FOR PRODUCING THEM |
US7793846B2 (en) | 2001-12-24 | 2010-09-14 | L-1 Secure Credentialing, Inc. | Systems, compositions, and methods for full color laser engraving of ID documents |
US7694887B2 (en) | 2001-12-24 | 2010-04-13 | L-1 Secure Credentialing, Inc. | Optically variable personalized indicia for identification documents |
US7728048B2 (en) | 2002-12-20 | 2010-06-01 | L-1 Secure Credentialing, Inc. | Increasing thermal conductivity of host polymer used with laser engraving methods and compositions |
US7824029B2 (en) | 2002-05-10 | 2010-11-02 | L-1 Secure Credentialing, Inc. | Identification card printer-assembler for over the counter card issuing |
US7804982B2 (en) | 2002-11-26 | 2010-09-28 | L-1 Secure Credentialing, Inc. | Systems and methods for managing and detecting fraud in image databases used with identification documents |
US7712673B2 (en) | 2002-12-18 | 2010-05-11 | L-L Secure Credentialing, Inc. | Identification document with three dimensional image of bearer |
GB2400287A (en) * | 2003-04-02 | 2004-10-06 | Autodesk Canada Inc | Three-Dimensional Image Compositing |
GB2400290A (en) * | 2003-04-04 | 2004-10-06 | Autodesk Canada Inc | Multidimensional image data processing in a hierarchical dat structure |
EP1614064B1 (en) | 2003-04-16 | 2010-12-08 | L-1 Secure Credentialing, Inc. | Three dimensional data storage |
US7669134B1 (en) | 2003-05-02 | 2010-02-23 | Apple Inc. | Method and apparatus for displaying information during an instant messaging session |
US7379925B2 (en) * | 2003-07-25 | 2008-05-27 | New York University | Logic arrangement, data structure, system and method for multilinear representation of multimodal data ensembles for synthesis, rotation and compression |
US7424685B2 (en) * | 2003-10-31 | 2008-09-09 | Sap Aktiengesellschaft | Providing access to database objects |
JP2007518199A (en) * | 2004-01-13 | 2007-07-05 | ニューヨーク・ユニバーシティ | Method, system, storage medium, and data structure for image recognition using multiple linear independent element analysis |
US7744002B2 (en) | 2004-03-11 | 2010-06-29 | L-1 Secure Credentialing, Inc. | Tamper evident adhesive and identification document including same |
US7620895B2 (en) * | 2004-09-08 | 2009-11-17 | Transcensus, Llc | Systems and methods for teaching a person to interact with a computer program having a graphical user interface |
CA2588630C (en) | 2004-11-19 | 2013-08-20 | Tivo Inc. | Method and apparatus for secure transfer of previously broadcasted content |
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US9304675B2 (en) * | 2006-09-06 | 2016-04-05 | Apple Inc. | Portable electronic device for instant messaging |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US8224816B2 (en) * | 2006-12-15 | 2012-07-17 | O'malley Matthew | System and method for segmenting information |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US9954996B2 (en) | 2007-06-28 | 2018-04-24 | Apple Inc. | Portable electronic device with conversation management for incoming instant messages |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US8327272B2 (en) | 2008-01-06 | 2012-12-04 | Apple Inc. | Portable multifunction device, method, and graphical user interface for viewing and managing electronic calendars |
US8478769B2 (en) | 2008-02-22 | 2013-07-02 | Accenture Global Services Limited | Conversational question generation system adapted for an insurance claim processing system |
US8515786B2 (en) | 2008-02-22 | 2013-08-20 | Accenture Global Services Gmbh | Rule generation system adapted for an insurance claim processing system |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US20100030549A1 (en) | 2008-07-31 | 2010-02-04 | Lee Michael M | Mobile device having human language translation capability with positional feedback |
US9959870B2 (en) | 2008-12-11 | 2018-05-01 | Apple Inc. | Speech recognition involving a mobile device |
US20120311585A1 (en) | 2011-06-03 | 2012-12-06 | Apple Inc. | Organizing task items that represent tasks to perform |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US9431006B2 (en) | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
DE202011111062U1 (en) | 2010-01-25 | 2019-02-19 | Newvaluexchange Ltd. | Device and system for a digital conversation management platform |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US8559793B2 (en) | 2011-05-26 | 2013-10-15 | Avid Technology, Inc. | Synchronous data tracks in a media editing system |
US20120317492A1 (en) * | 2011-05-27 | 2012-12-13 | Telefon Projekt LLC | Providing Interactive and Personalized Multimedia Content from Remote Servers |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US8994660B2 (en) | 2011-08-29 | 2015-03-31 | Apple Inc. | Text correction processing |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
US10199051B2 (en) | 2013-02-07 | 2019-02-05 | Apple Inc. | Voice trigger for a digital assistant |
US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
WO2014144579A1 (en) | 2013-03-15 | 2014-09-18 | Apple Inc. | System and method for updating an adaptive speech recognition model |
CN105027197B (en) | 2013-03-15 | 2018-12-14 | 苹果公司 | Training at least partly voice command system |
WO2014197334A2 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
WO2014197336A1 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
WO2014197335A1 (en) | 2013-06-08 | 2014-12-11 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
CN110442699A (en) | 2013-06-09 | 2019-11-12 | 苹果公司 | Operate method, computer-readable medium, electronic equipment and the system of digital assistants |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
KR101809808B1 (en) | 2013-06-13 | 2017-12-15 | 애플 인크. | System and method for emergency calls initiated by voice command |
DE112014003653B4 (en) | 2013-08-06 | 2024-04-18 | Apple Inc. | Automatically activate intelligent responses based on activities from remote devices |
US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
EP3480811A1 (en) | 2014-05-30 | 2019-05-08 | Apple Inc. | Multi-command single utterance input method |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
DK179309B1 (en) | 2016-06-09 | 2018-04-23 | Apple Inc | Intelligent automated assistant in a home environment |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
DK179049B1 (en) | 2016-06-11 | 2017-09-18 | Apple Inc | Data driven natural language event detection and classification |
DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
DK179343B1 (en) | 2016-06-11 | 2018-05-14 | Apple Inc | Intelligent task discovery |
DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
DK201770431A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5111409A (en) * | 1989-07-21 | 1992-05-05 | Elon Gasper | Authoring and use systems for sound synchronized animation |
-
1993
- 1993-05-10 US US08/059,542 patent/US5428731A/en not_active Expired - Lifetime
-
1994
- 1994-05-06 WO PCT/US1994/004991 patent/WO1994027237A1/en active IP Right Grant
- 1994-05-06 CA CA002161423A patent/CA2161423A1/en not_active Abandoned
- 1994-05-06 EP EP94915453A patent/EP0698244B1/en not_active Expired - Lifetime
- 1994-05-06 AU AU66707/94A patent/AU6670794A/en not_active Abandoned
- 1994-05-06 DE DE69429860T patent/DE69429860T2/en not_active Expired - Lifetime
- 1994-05-06 JP JP52554894A patent/JP4021474B2/en not_active Expired - Lifetime
Also Published As
Publication number | Publication date |
---|---|
DE69429860D1 (en) | 2002-03-21 |
JP4021474B2 (en) | 2007-12-12 |
US5428731A (en) | 1995-06-27 |
DE69429860T2 (en) | 2002-11-07 |
EP0698244B1 (en) | 2002-02-13 |
JPH08510075A (en) | 1996-10-22 |
AU6670794A (en) | 1994-12-12 |
WO1994027237A1 (en) | 1994-11-24 |
EP0698244A1 (en) | 1996-02-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2161423A1 (en) | Interactive multimedia delivery engine | |
EP0278722B1 (en) | Document composition system using named formats and named fonts | |
US5982365A (en) | System and methods for interactively generating and testing help systems | |
EP0706124B1 (en) | System for generating interactive software applications | |
US6374271B1 (en) | Hypermedia document authoring using a goals outline and a presentation outline | |
US6320602B1 (en) | Region layout in a view on a graphical display screen | |
US5517621A (en) | Method and apparatus for document formatting with efficient figure element layout manipulation | |
CA1285076C (en) | Interactive video composition and presentation systems | |
US5752055A (en) | Systems and method for automatically linking parts within compound documents | |
US5619636A (en) | Multimedia publishing system | |
US5293473A (en) | System and method for editing a structured document to modify emphasis characteristics, including emphasis menu feature | |
US5033008A (en) | Dynamic selection of logical element data format as a document is created or modified | |
US5864338A (en) | System and method for designing multimedia applications | |
EP0413653A2 (en) | Dynamic selection of recursively nested logical element data formats | |
EP0842477B1 (en) | System and method for handling technical information | |
US20010005203A1 (en) | Method for generating multimedia presentation | |
US5856830A (en) | Animation display processor | |
JPH05266109A (en) | Method for defining geometric relation, and computer-aided design system | |
EP0642684B1 (en) | Software fault location method | |
JPS6136868A (en) | Information retriever | |
JPH0218669A (en) | Displaying system for selected information on electronic catalog | |
US6020899A (en) | System for viewing the structure of computer graphical elements | |
EP0410062B1 (en) | Dynamic selection of logical element data format | |
US20020029106A1 (en) | Vehicular information system developing apparatus, vehicular information system developing tool, vehicular information system developing method, and navigation system developing method | |
Bass et al. | Fault tree graphics |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FZDE | Discontinued |