US20120098761A1 - Display system and method of display for supporting multiple display modes - Google Patents

Display system and method of display for supporting multiple display modes Download PDF

Info

Publication number
US20120098761A1
US20120098761A1 US13/004,193 US201113004193A US2012098761A1 US 20120098761 A1 US20120098761 A1 US 20120098761A1 US 201113004193 A US201113004193 A US 201113004193A US 2012098761 A1 US2012098761 A1 US 2012098761A1
Authority
US
United States
Prior art keywords
display
screen
background
display screen
mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/004,193
Inventor
April Slayden Mitchell
Ian N. Robinson
Mark C. Solomon
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hewlett Packard Development Co LP
Original Assignee
Hewlett Packard Development Co LP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from PCT/US2010/053860 external-priority patent/WO2012054063A1/en
Application filed by Hewlett Packard Development Co LP filed Critical Hewlett Packard Development Co LP
Priority to US13/004,193 priority Critical patent/US20120098761A1/en
Assigned to HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P. reassignment HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MITCHELL, APRIL SLAYDEN, ROBINSON, IAN N, SOLOMON, MARK C
Publication of US20120098761A1 publication Critical patent/US20120098761A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012Head tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G06F1/1643Details related to the display arrangement, including those related to the mounting of the display in the housing the display being associated to a digitizer, e.g. laptops that can be used as penpads
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser

Definitions

  • a wide variety of displays for computer systems are available. Often display systems display content on an opaque background screen. However, systems are available which display content on a transparent background screen. There is software available which recognizes the rotation of display screen. This software can present content differently dependent on the viewing angles and user interface orientations dependent upon how the user is holding or has rotated a display. There are also displays for some computer systems that allow the display screen to be moved between a position into a second position which allows writing on the display screen surface.
  • FIG. 1 illustrates a block diagram of a front view of a display screen in an display system with an object positioned behind the display screen according to an embodiment of the invention
  • FIG. 2A shows a side perspective view of a desktop version of an display system with the display screen operating in an opaque operational mode where the display screen is positioned in a substantially vertical position according to an embodiment of the invention
  • FIG. 2B shows aside perspective view of a desktop version of an display system with the display screen operating in a transparent operational mode where the display screen is positioned in a second position according to an embodiment of the invention
  • FIG. 3 shows a back perspective view of the display system shown in FIG. 1 according to an embodiment of the invention
  • FIG. 4 shows a flow diagram for a method of display according to an embodiment of the invention
  • FIG. 5 shows a computer system for implementing the method shown in FIG. 4 and described in accordance with embodiments of the present invention.
  • the present invention describes a display system 100 , the display system 100 comprising: a display 110 , the display 110 including a display screen 112 for operating in at least a first background screen mode and a second background screen mode; at least one sensor 114 for sensing interaction with the display 110 ; and a background screen control component 118 for processing information from the at least one sensor 114 , wherein based on the background screen control sensor data 132 from the at least one background screen control sensor 114 a determination is made whether a background screen control condition 136 has been met, wherein responsive to the determination that the background screen control condition 136 has been met, the background of the display screen 112 is changed.
  • the display screen background switches between a first opaque background and a second transparent background
  • Switching of the background screens is controlled by predefined background screen control conditions 136 defined in the background screen control component
  • the background screen of the display can be changed based on system or user controlled instructions or automatically based on common usage assumptions.
  • the common usage instructions are defined in the background screen control component 118 as background screen control conditions 136 .
  • the background control sensor data 132 gathered by the at least one background screen control sensor is tested to see if the background control conditions 136 are met. Responsive to whether the background screen control conditions 136 are met, the display system determines the display screen background that is output.
  • the predefined common usage assumption is that when the user changes the position of the display screen, the user wants to change the operational mode and correspondingly change the display screen to the background screen associated with the operational mode.
  • the user may wish to change from a viewing mode where the display screen background is opaque to a “thru-screen” mode, a mode where the display screen background screen is transparent and objects can be viewed “thru” the transparent display screen. Details regarding a transparent thru-screen type display system and interacting with an object or device positioned behind the display screen is described in more detail in the pending patent applications entitled “An Augmented Reality Display System and Method of Display” filed on Oct.
  • the predefined common usage assumption is that when the user moves an object behind the display screen, the user wants to change the operational mode. For example, if the user is in the viewing mode (with an opaque display screen background) and moves an object behind the display screen, the common usage assumption is that the user wants to use the system in the thru-screen operational mode and that the display screen should be transparent.
  • FIG. 1 shows a block diagram of a front view of a display screen in a display system according to an embodiment of the invention.
  • the display system includes a display screen 112 for displaying content on a background screen.
  • the display screen 112 is capable of operating either in at least a first mode (with an opaque background screen) and a second mode (with a transparent background screen). Switching between the two modes can be controlled by (1) system or user instructions to the display system or (2) automatically based on common usage assumptions predefined in the background screen control component.
  • the background screen is preferably opaque.
  • An opaque background provides greater contrast so that person viewing the screen can more easily see the content.
  • an opaque background is preferable as viewing conditions for the user can be easily optimized—the opaque background screen and content displayed on the opaque background screen can be chosen to provide optimal contrast to make viewing the content easier by the user.
  • a transparent background screen allows the user to interact with an object or device positioned behind the screen. In the thru-screen operational mode, a transparent screen is necessary so that the user can see through the display screen so that the user can interact with an object positioned behind the display screen.
  • the transparent display screen is a transparent OLED (organic light-emitting diode) screen.
  • the display screen is comprised of transparent LCDs (liquid crystal display).
  • the transparent LCD screen implementation assumes a well lit working environment.
  • the display screen is comprised of a partially diffusing material and the content is projected onto the partially diffusing material.
  • the transparent display screen operates so that objects positioned behind the display screen can be easily seen or viewed by a user positioned in front of the display screen.
  • the display screen 112 is capable of switching between a first opaque display mode and a second transparent display mode.
  • first opaque background screen display mode the background screen upon which content is displayed would be opaque.
  • second transparent background screen display mode the background screen upon which content is displayed would be transparent.
  • a display screen capable of switching between two modes could be implemented in one embodiment as a transparent OLED screen with a liquid crystal shutter positioned behind the OLED screen. In a first mode the display screen would appear to be opaque (liquid crystal shutter behind the display screen closed) for use as a conventional display. In a second mode the transparent LED screen would be transparent (liquid crystal shutter open).
  • Background screen control conditions 136 are predefined based on common usage assumptions. Consider the common usage condition of a transparent screen in a first position, then in one embodiment, background control sensor or sensors 114 coupled to a display stands hinge(s) 128 would be used to sense when the display screen 112 moves to a first position. By using information or data 132 gathered from the at least one background control sensor, the display system 100 can determine whether a background control condition has been met.
  • the at least one background control sensor 114 senses an interaction with the display.
  • the display 110 includes a display stand 157 for supporting the display screen on a surface 155 .
  • the at least one sensor is a sensor coupled to the hinge 128 of the display stand.
  • the sensed information is used to determine whether a background screen control condition has been met. For example, if the background screen control condition is the hinge moving to a second position, when this condition is met—the background of the display screen is changed.
  • the display system 100 automatically controls the display screen's background to correlate it to the defined display screen operational mode.
  • the background screen control component 118 controls the display screen 112 to automate displaying the desired background screen, without requiring user input to specify the background screen or a change in the background screen. Having the display system 100 automatically react to these common usage event triggers, saves the user time and effort and creates a more natural user interface.
  • the display system 110 changes the screen background based on sensed information from the at least one background control.
  • the at least one background control sensor 114 detects the movement of the display screen hinge(s) 128 to a predefined position or alternatively through a certain range of motion.
  • the sensor 114 detect the a user held device or other object that the user is manipulating in the capture range behind the back surface of the display screen.
  • the type of sensor used in the display system depends on the type of interaction that the background control sensor 114 is required to sense or detect.
  • Examples of the types of interactions that the at least one background control sensor might sense include, but are not limited to, the examples of: sensing a change in the position of the display screen, display stand or hinges of the display stand, sensing an object placed behind the display screen, sensing touch on the back surface of the display screen, and sensing an electrical connection made to the display system.
  • more than one background control sensor 114 or more than one type of background control sensors is used.
  • more than one type of background control sensor might be used to provide additional verification that that a triggered or sensed event has occurred.
  • an object 120 positioned behind the screen might be identified using two types of background control sensors. For example, both a motion detection sensor (to detect an object positioned behind screen) and a depth camera or other image capture device (to identify an object positioned behind the screen) might be used to detect an object positioned behind the back surface of the display screen.
  • the at least one background control sensor senses interaction with the display.
  • the interaction is a user interaction that is instigated or initiated by the physical actions of a user.
  • the interaction sensed is the physical movement of the display screen from a first position to a second position.
  • the interaction sensed is the physical movement of an object in the capture region behind the display screen.
  • interaction with the display includes physical movements where the user is not physically touching the display itself, but where his physical movements are captured by the display—in this case captured by the sensors on the back surface of the display frame.
  • FIG. 1 shows at least one background screen control sensor, for example coupled to the hinge mechanism of the display stand.
  • Data 132 from the background control sensor is input to background screen control component 118 for use in determining the background that is used and displayed by the display generation component 126 .
  • the display generation component controls the content output to the display and the background screen displayed on the display screen.
  • FIG. 1 also shows data from a viewpoint assessment sensors 140 a , 140 b and object tracking sensors 148 a , 148 b that input information into the display controller component 130 for use by the display generation component 126 .
  • the background screen control sensor 114 is a separate sensor from the viewpoint assessment sensors and object tracking sensors used in the display system.
  • one sensor type for example, viewpoint assessment sensors or object tracking sensors
  • the background screen control sensor could be used to sense data for another sensor type (for example, the background screen control sensor).
  • a depth camera could be used both to perform the function required by the background control sensor and the object tracking sensor function.
  • depth camera data could be used to determine whether an object was positioned behind the screen (the background control sensor condition) and the location of the object (the object tracking sensor function) .
  • a single sensor might provide the data required—though the data gathered might be tested or processed differently dependent upon the requirements for the sensor.
  • the display system is capable of switching between a first opaque background screen mode and a second transparent background mode. If it is desirable for the background screen to change between opaque and transparent background screens based on the position of the display screen—the background screen control sensors would measure conditions regarding the position of the display screen. Based on whether the predefined conditions occurred (i.e. display screen at certain angle relative to stand base, etc.), the background screen would change or remain static.
  • switching between operational modes is controlled by moving the display screen from a first screen position to a second screen position.
  • FIG. 2A shows a side view of a desktop version of a display system with the display screen positioned in a first position—a position substantially vertical to the surface the display stand is resting on.
  • FIGS. 2A and 2B operate in the viewing mode when in the first position (opaque screen) and the thru-screen mode (transparent screen) in the second position.
  • FIG. 23 shows a side view of a desktop version of the same display system with the display screen in a second position.
  • the second position shown in FIG. 2B is more horizontal to the surface that the display stand is resting on.
  • the angle of the display screen in the first position shown in FIG. 2A is substantially vertical or at approximately 90 degree to the positive x axis, while the angle of the display screen in the second position shown in FIG. 23 is at an acute angle, say for purposes of example at an angle of approximately 75 degrees relative to the positive x axis.
  • the first and/or second positions are at a defined angle, however, more commonly the first and second positions are defined within a range of angles.
  • a user it is common for a user to move the display screen of a laptop or desktop monitor to a position that is at an acute angle to the referenced positive axis.
  • the vertical range of the display screen might be between 80 and 95 degrees relative to the positive x axis for the first position.
  • the second position relative to the first viewing position has a smaller acute angle range. This smaller acute angle range occurs in the thru-screen mode as it is desirable to have additional room behind the screen to position an object behind the screen to interact with the object. To achieve this additional space, the user pulls the display screen out and angles the screen at a smaller acute angle range compared to the display screen first position.
  • the background screen control sensors detect the movement of the hinges with respect to the display stand.
  • the background control sensor 114 used when the display screen or object that is effected by, coupled to or is part of the display (for example, the hinge) reaches a predefined position.
  • the background screen control sensors should sense the movement of the display hinges as it moves through a predefined range of motion.
  • background screen control sensors are coupled to or embedded in the hinges to determine whether so we know when the hinges are in a locked up or locked down position.
  • the sensors coupled to the hinges are accelerometers that detect the motion of the hinges.
  • the sensors are electrical components or devices, (i.e., a Hall effect device), which detect closing of a circuit.
  • a Hall effect device when the display screen moves from a first position to a second position a circuit is dosed resulting in a current flow.
  • the sensor could be a current sensing device which detects current flow when the hinge reaches the second position.
  • the background control sensor could be any sensor that detects rotation, such as a rotary encoder.
  • the rotation sensing device coupled to the hinge or bezel of the display senses when the display is at a first or second position or alternatively is moved from a first or second position.
  • the predefined common usage assumption is that when the user changes the position of the display screen, the user wants to change the operational mode.
  • the user indicates that he wishes to change the operational mode by placing an object behind the display screen.
  • a user operating the display screen in an operational mode that has an opaque background screen could indicate the intent to switch to a transparent background screen by placing an object (such as a keyboard, etc.) behind the display screen.
  • FIG. 3 shows a perspective back view of the display system shown in FIG. 1 according to an embodiment of the invention where the sensors 114 a , 114 b can be more clearly seen.
  • sensors 114 a , 114 b act as background control sensors.
  • the information sensed from sensors 114 a , 114 b from the back surface 158 of the display screen 112 provide information about whether the user has positioned a device or other object behind the display screen indicating the desire to change to the thru-screen mode (transparent background screen). In other words, if a display screen is in a mode which requires an opaque background and an object is positioned behind the display screen, the display screen would automatically change to a transparent screen.
  • the type of sensors used for the background control sensors 114 depends upon the type of interaction that the sensor is required to sense or detect.
  • the background control sensors should be able to detect an object positioned behind the display screen.
  • the background control sensors 114 a , 114 b are motion sensors which detect movement of an object behind the screen.
  • the background control sensors 114 a , 114 b could be an image capture devices.
  • image analysis software is used to detect changes from the previous image frames or in an alternatively is used to identify the type of object positioned behind the display screen.
  • the background control sensor can detect the addition or removal of an object from behind the display screen.
  • a camera with a capture region covering the region behind the display screen could be used for to detect the addition or removal of an object from behind the display screen.
  • an object being removed from behind the display screen could indicate that the user wished to return to the viewing mode or other mode where an opaque background display screen would be desirable.
  • the predefined common usage assumption upon which the predetermined background screen control conditions are based is that when the user touches the back surface of the display (including the display screen and/or the frame of the display) that the user wants to interact with the display screen in the thru-screen operational mode (transparent display screen background).
  • a user operating the display screen in an operational mode that has an opaque background screen could indicate the intent to switch to a transparent background screen by touching the back surface 158 of the display screen or by touching the back of the frame 154 of the display screen.
  • the type of sensors used for the background control sensors 114 depends upon the type of interaction that the sensor is required to sense or detect.
  • the background control sensors should be able to detect touch of the back surface of the display screen or display screen frame.
  • the background control sensors 114 are a plurality of sensors embedded in the bezel 154 around the entire periphery of the display screen that capable of detecting touch on the back surface 158 the display screen surface.
  • the background control sensor could be a plurality of image capture device which captures a plurality of images behind the display screen surface. Image analysis software is used to determine when an object (including the user's hand) makes physical contact with the display screen.
  • the user might be touching the back surface of the display screen frame merely to adjust the position of the display screen and not to change the operational mode.
  • an additional check may be made to verify that the intent was to change the mode and thus the background screen—and not just to adjust the position.
  • both the back and front touch sensitive sensors on the bezel indicate that a touch has occurred, then it is assumed that the user is adjusting the display stand position. If only the sensor on the back bezel of the display screen indicates touch, then it is assumed that the user wished to interact in the thru screen mode with a transparent screen background.
  • the background screen control component 118 can be set by system or user controlled instructions that set background screen conditions of the background screen control component.
  • the user could set his preferences or additional preferences for automatically switching between background modes. For example, in addition to turning on the transparent mode by touching the back of the screen, touch could be used to turn off the transparent mode. In effect, touch could be used as an off/switch for a given background screen. For instance, a first touch on the back display screen bezel 154 could indicate that the user wishes to change to the thru-screen mode (transparent screen). A second touch on the back bezel of the display screen could indicate that the user wishes to change back to the original background screen.
  • an opaque background i.e., displaying video
  • the user who might for example often interact with an opaque screen in the viewing mode might indicate that he wishes to interact with a transparent background screen by touching the front surface of the display screen bezel a predetermined number of times.
  • the user instead of a physical user interaction with the display by the user, the user interacts with a user interface that controls the background screen mode.
  • the user interface is an icon displayed on the display screen.
  • the icon could be a picture of a display with a dear or lighter colored display screen representing the transparent background mode and a darker colored display screen representing the opaque background mode.
  • the display screen icon could represent a transparent background mode when an outline of the display frame is shown and the opaque background mode when the outline of the display frame is shown and in addition the display screen filled in .
  • the user's selection clicking on the icon), could change the background screen to correspond to the background screen selected by the icon.
  • the user interface is a pulldown menu where the user selects a transparent screen or transparent screen based on the user's desired mode of operation.
  • the user interface is a button embedded in the frame of the display that is pressed in or released (on/off) to change background screen modes.
  • the background screen is chosen indirectly by interpreting the sensed user physical interactions.
  • common usage assumptions are associated with background screen control conditions 136 stored in the background screen control component 118 . If background screen control conditions are sensed (for example, hinge movement), then based on the control conditions sensed and the background screen mode associated with it, a background screen is selected.
  • the user selects the background screen mode directly using a background screen control user interface 141 . For example, by simply clicking on the transparent screen icon, the user can change the background screen mode.
  • FIG. 1 shows a background screen control user interface 141 ,
  • the background screen is selected. For example, as previously discussed the user could select an icon representing the desirable background screen or use a pulldown menu to select the desired background screen.
  • the user uses the background screen control user interface 141 to directly select whether the background screen is transparent or opaque.
  • FIG. 4 shows a flow diagram for a method of controlling the background screen display mode based on interaction with the display according to an embodiment of the invention.
  • FIG. 4 shows the steps of: determining whether a predefined interaction with a display has occurred, wherein the display is capable of displaying at least a first background screen and a second background screen (step 410 ); and determining whether the predefined interaction with the display meets the background control conditions, wherein responsive to meeting the background control conditions, the background screen corresponding to the predefined conditions is displayed (steps 412 , 414 ).
  • FIG. 5 shows a computer system for implementing the methods shown in FIG. 4 and described in accordance with embodiments of the present invention.
  • the method 400 represents generalized illustrations and that other steps may be added or existing steps may be removed, modified or rearranged without departing from the scopes of the method 400 .
  • the descriptions of the method 400 are made with reference to the system 100 illustrated in FIG. 1 and the system 500 illustrated in FIG. 5 and thus refers to the elements cited therein, It should, however, be understood that the method 400 is not limited to the elements set forth in the system 500 . Instead, it should be understood that the method 400 may be practiced by a system having a different configuration than that set forth in the system 500 .
  • the operations set forth in the method 400 may be contained as utilities, programs or subprograms, in any desired computer accessible medium.
  • the method 400 may be embodied by computer programs, which may exist in a variety of forms both active and inactive. For example, they may exist as software program(s) comprised of program instructions in source code, object code, executable code or other formats. Any of the above may be embodied on a computer readable medium, which include storage devices and signals, in compressed or uncompressed form.
  • FIG. 5 illustrates a block diagram of a computing apparatus 500 configured to implement or execute the methods 400 depicted in FIG. 4 , according to an example.
  • the computing apparatus 400 may be used as a platform for executing one or more of the functions described hereinabove with respect to the display controller component 130 .
  • the computing apparatus 500 includes one or more processor(s) 502 that may implement or execute some or all of the steps described in the methods 400 . Commands and data from the processor 502 are communicated over a communication bus 504 .
  • the computing apparatus 500 also includes a main memory 506 , such as a random access memory (RAM), where the program code for the processor 502 , may be executed during runtime, and a secondary memory 508 .
  • the secondary memory 508 includes, for example, one or more hard drives 510 and/or a removable storage drive 512 , representing a removable flash memory card, etc., where a copy of the program code for the method 500 may be stored.
  • the removable storage drive 512 reads from and/or writes to a removable storage unit 514 in a well-known manner.
  • Non-transitory computer readable storage devices that may be used to implement the present invention include but are not limited to conventional computer system RAM, ROM, EPROM, EEPROM, hard drives, flash memory and magnetic or optical disks or tapes. Concrete examples of the foregoing include distribution of the programs on a CD ROM or via Internet download.
  • any of the memory components described 506 , 508 , 514 may also store an operating system 530 , such as Mac OS, MS Windows, Unix, or Linux; network applications 532 ; and a display controller component 130 .
  • the operating system 530 may be multi-participant, multiprocessing, multitasking, multithreading, real-time and the like.
  • the operating system 530 may also perform basic tasks such as recognizing input from input devices, such as a keyboard or a keypad; sending output to the display 112 controlling peripheral devices, such as disk drives, printers, image capture device; and managing traffic on the one or more buses 504 .
  • the network applications 532 includes various components for establishing and maintaining network connections, such as software for implementing communication protocols including TCP/IP, HTTP, Ethernet, USB, and FireWire.
  • the computing apparatus 500 may also include an input devices 516 , such as a keyboard, a keypad, functional keys, etc., a pointing device, such as a tracking ball, cursors, etc., and a display(s) 110 , such as the display 110 shown for example in FIGS. 1-3 .
  • a display adaptor 522 may interface with the communication bus 504 and the display 112 and may receive display data from the processor 502 and convert the display data into display commands for the display 520 .
  • the processor(s) 502 may communicate over a network, for instance, a cellular network, the Internet, LAN, etc., through one or more network interfaces 524 such as a Local Area Network LAN, a wireless 802.11x LAN, a 3G mobile WAN or a WiMax WAN.
  • network interfaces 524 such as a Local Area Network LAN, a wireless 802.11x LAN, a 3G mobile WAN or a WiMax WAN.
  • an interface 526 may be used to receive an image or sequence of images from imaging components 528 such as the image capture device.

Abstract

The present invention is a display system that is comprised of a display, at least one background screen control sensor and a background screen control component. The display includes a display screen for operating in at least a first background screen mode and a second background screen mode. The at least one background control sensor for sensing interaction with the display. The background screen control component processes background control sensor data from the at least one sensor, wherein based on the background control sensor data a determination is made whether a background screen control condition has been met, wherein responsive to the determination that the background screen control condition has been met, the background of the display screen is changed.

Description

    CROSS-REFERENCE TO RELATED APPLICATION
  • This case is a continuation in part of the case entitled “An Augmented Reality Display System and Method of Display” filed on Oct. 22, 2010, having serial number PCT/US2010/053860, which is hereby incorporated by reference in its entirety. In addition this case is related to the case entitled “Display System and Method of Displaying Based on Device Interactions” filed on Oct. 29, 2010, having Ser. No. 12/915,311, which is hereby incorporated by reference in its entirety.
  • BACKGROUND
  • A wide variety of displays for computer systems are available. Often display systems display content on an opaque background screen. However, systems are available which display content on a transparent background screen. There is software available which recognizes the rotation of display screen. This software can present content differently dependent on the viewing angles and user interface orientations dependent upon how the user is holding or has rotated a display. There are also displays for some computer systems that allow the display screen to be moved between a position into a second position which allows writing on the display screen surface.
  • BRIEF DESCRIPTION OF DRAWINGS
  • The figures depict implementations/embodiments of the invention and not the invention itself. Some embodiments are described, by way of example, with respect to the following Figures.
  • FIG. 1 illustrates a block diagram of a front view of a display screen in an display system with an object positioned behind the display screen according to an embodiment of the invention;
  • FIG. 2A shows a side perspective view of a desktop version of an display system with the display screen operating in an opaque operational mode where the display screen is positioned in a substantially vertical position according to an embodiment of the invention;
  • FIG. 2B shows aside perspective view of a desktop version of an display system with the display screen operating in a transparent operational mode where the display screen is positioned in a second position according to an embodiment of the invention;
  • FIG. 3 shows a back perspective view of the display system shown in FIG. 1 according to an embodiment of the invention;
  • FIG. 4 shows a flow diagram for a method of display according to an embodiment of the invention;
  • FIG. 5 shows a computer system for implementing the method shown in FIG. 4 and described in accordance with embodiments of the present invention.
  • The drawings referred to in this Brief Description should not be understood as being drawn to scale unless specifically noted.
  • DETAILED DESCRIPTION OF THE EMBODIMENTS
  • For simplicity and illustrative purposes, the principles of the embodiments are described by referring mainly to examples thereof. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It will be apparent, however, to one of ordinary skill in the art, that the embodiments may be practiced without limitation to these specific details. Also, different embodiments may be used together. In some instances, well known methods and structures have not been described in detail so as not to unnecessarily obscure the description of the embodiments.
  • The present invention describes a display system 100, the display system 100 comprising: a display 110, the display 110 including a display screen 112 for operating in at least a first background screen mode and a second background screen mode; at least one sensor 114 for sensing interaction with the display 110; and a background screen control component 118 for processing information from the at least one sensor 114, wherein based on the background screen control sensor data 132 from the at least one background screen control sensor 114 a determination is made whether a background screen control condition 136 has been met, wherein responsive to the determination that the background screen control condition 136 has been met, the background of the display screen 112 is changed.
  • In one example, the display screen background switches between a first opaque background and a second transparent background, Switching of the background screens is controlled by predefined background screen control conditions 136 defined in the background screen control component, The background screen of the display can be changed based on system or user controlled instructions or automatically based on common usage assumptions. The common usage instructions are defined in the background screen control component 118 as background screen control conditions 136. The background control sensor data 132 gathered by the at least one background screen control sensor is tested to see if the background control conditions 136 are met. Responsive to whether the background screen control conditions 136 are met, the display system determines the display screen background that is output.
  • In one embodiment, the predefined common usage assumption is that when the user changes the position of the display screen, the user wants to change the operational mode and correspondingly change the display screen to the background screen associated with the operational mode. For example in one embodiment, the user may wish to change from a viewing mode where the display screen background is opaque to a “thru-screen” mode, a mode where the display screen background screen is transparent and objects can be viewed “thru” the transparent display screen. Details regarding a transparent thru-screen type display system and interacting with an object or device positioned behind the display screen is described in more detail in the pending patent applications entitled “An Augmented Reality Display System and Method of Display” filed on Oct. 22, 2010, having serial number PCT/US2010/053860 and the case entitled “Display System and Method of Displaying Based on Device Interactions” filed on Oct. 29, 2010, having Ser. No. 12/915,311, which are both hereby incorporated by reference in it's entirety.
  • In another alternative embodiment, the predefined common usage assumption is that when the user moves an object behind the display screen, the user wants to change the operational mode. For example, if the user is in the viewing mode (with an opaque display screen background) and moves an object behind the display screen, the common usage assumption is that the user wants to use the system in the thru-screen operational mode and that the display screen should be transparent.
  • Referring to FIG. 1 shows a block diagram of a front view of a display screen in a display system according to an embodiment of the invention. The display system includes a display screen 112 for displaying content on a background screen. In one embodiment, the display screen 112 is capable of operating either in at least a first mode (with an opaque background screen) and a second mode (with a transparent background screen). Switching between the two modes can be controlled by (1) system or user instructions to the display system or (2) automatically based on common usage assumptions predefined in the background screen control component.
  • In the viewing mode, the background screen is preferably opaque. An opaque background provides greater contrast so that person viewing the screen can more easily see the content. Thus an opaque background is preferable as viewing conditions for the user can be easily optimized—the opaque background screen and content displayed on the opaque background screen can be chosen to provide optimal contrast to make viewing the content easier by the user. However, a transparent background screen allows the user to interact with an object or device positioned behind the screen. In the thru-screen operational mode, a transparent screen is necessary so that the user can see through the display screen so that the user can interact with an object positioned behind the display screen.
  • Different screen materials may be used to provide the transparent display screen 112. In one embodiment, the transparent display screen is a transparent OLED (organic light-emitting diode) screen. In an alternative embodiment, the display screen is comprised of transparent LCDs (liquid crystal display). However, the transparent LCD screen implementation assumes a well lit working environment. In a third alternative embodiment, the display screen is comprised of a partially diffusing material and the content is projected onto the partially diffusing material. Although many alternative implementations are possible, the transparent display screen operates so that objects positioned behind the display screen can be easily seen or viewed by a user positioned in front of the display screen.
  • As previously described, in one embodiment the display screen 112 is capable of switching between a first opaque display mode and a second transparent display mode. For example, in the first opaque background screen display mode—the background screen upon which content is displayed would be opaque. Similarly, in the second transparent background screen display mode—the background screen upon which content is displayed would be transparent. A display screen capable of switching between two modes could be implemented in one embodiment as a transparent OLED screen with a liquid crystal shutter positioned behind the OLED screen. In a first mode the display screen would appear to be opaque (liquid crystal shutter behind the display screen closed) for use as a conventional display. In a second mode the transparent LED screen would be transparent (liquid crystal shutter open).
  • Background screen control conditions 136 are predefined based on common usage assumptions. Consider the common usage condition of a transparent screen in a first position, then in one embodiment, background control sensor or sensors 114 coupled to a display stands hinge(s) 128 would be used to sense when the display screen 112 moves to a first position. By using information or data 132 gathered from the at least one background control sensor, the display system 100 can determine whether a background control condition has been met.
  • The at least one background control sensor 114 senses an interaction with the display. The display 110 includes a display stand 157 for supporting the display screen on a surface 155. In one embodiment, for example, the at least one sensor is a sensor coupled to the hinge 128 of the display stand. When an interaction occurs (user physically moving the display screen), the sensed information is used to determine whether a background screen control condition has been met. For example, if the background screen control condition is the hinge moving to a second position, when this condition is met—the background of the display screen is changed.
  • If a common usage condition has occurred, the display system 100 automatically controls the display screen's background to correlate it to the defined display screen operational mode. In one embodiment, the background screen control component 118 controls the display screen 112 to automate displaying the desired background screen, without requiring user input to specify the background screen or a change in the background screen. Having the display system 100 automatically react to these common usage event triggers, saves the user time and effort and creates a more natural user interface.
  • The display system 110 changes the screen background based on sensed information from the at least one background control. In one embodiment, the at least one background control sensor 114 detects the movement of the display screen hinge(s) 128 to a predefined position or alternatively through a certain range of motion. In another embodiment, the sensor 114 detect the a user held device or other object that the user is manipulating in the capture range behind the back surface of the display screen.
  • The type of sensor used in the display system depends on the type of interaction that the background control sensor 114 is required to sense or detect. Examples of the types of interactions that the at least one background control sensor might sense include, but are not limited to, the examples of: sensing a change in the position of the display screen, display stand or hinges of the display stand, sensing an object placed behind the display screen, sensing touch on the back surface of the display screen, and sensing an electrical connection made to the display system.
  • In one embodiment, more than one background control sensor 114 or more than one type of background control sensors is used. For example, for the case where a single sensor does not cover the desired range behind the display screen, multiple sensors of the same type with overlapping capture regions might be positioned at different points along the bezel 154 of the display screen to expand the capture range of the background control sensors. In another embodiment, more than one type of background control sensor might be used to provide additional verification that that a triggered or sensed event has occurred. For example in one embodiment, when in the transparent mode, an object 120 positioned behind the screen might be identified using two types of background control sensors. For example, both a motion detection sensor (to detect an object positioned behind screen) and a depth camera or other image capture device (to identify an object positioned behind the screen) might be used to detect an object positioned behind the back surface of the display screen.
  • The at least one background control sensor senses interaction with the display. In one embodiment, the interaction is a user interaction that is instigated or initiated by the physical actions of a user. For example, in one embodiment the interaction sensed is the physical movement of the display screen from a first position to a second position. In another embodiment, the interaction sensed is the physical movement of an object in the capture region behind the display screen. Thus, interaction with the display includes physical movements where the user is not physically touching the display itself, but where his physical movements are captured by the display—in this case captured by the sensors on the back surface of the display frame.
  • Referring to FIG. 1 shows at least one background screen control sensor, for example coupled to the hinge mechanism of the display stand. Data 132 from the background control sensor is input to background screen control component 118 for use in determining the background that is used and displayed by the display generation component 126. In one embodiment, the display generation component controls the content output to the display and the background screen displayed on the display screen.
  • FIG. 1 also shows data from a viewpoint assessment sensors 140 a, 140 b and object tracking sensors 148 a, 148 b that input information into the display controller component 130 for use by the display generation component 126. In the embodiment shown, the background screen control sensor 114 is a separate sensor from the viewpoint assessment sensors and object tracking sensors used in the display system. However, in an alternative embodiment, one sensor type (for example, viewpoint assessment sensors or object tracking sensors) could be used to sense data for another sensor type (for example, the background screen control sensor). For example, for the case where an object's positioned behind the back surface of the display screen—a depth camera could be used both to perform the function required by the background control sensor and the object tracking sensor function. In other words, depth camera data could be used to determine whether an object was positioned behind the screen (the background control sensor condition) and the location of the object (the object tracking sensor function) . In this case, a single sensor might provide the data required—though the data gathered might be tested or processed differently dependent upon the requirements for the sensor.
  • In one embodiment, the display system is capable of switching between a first opaque background screen mode and a second transparent background mode. If it is desirable for the background screen to change between opaque and transparent background screens based on the position of the display screen—the background screen control sensors would measure conditions regarding the position of the display screen. Based on whether the predefined conditions occurred (i.e. display screen at certain angle relative to stand base, etc.), the background screen would change or remain static.
  • In one embodiment, switching between operational modes (and their corresponding opaque and transparent background screens) is controlled by moving the display screen from a first screen position to a second screen position. To further illustrate the example above, refer to the display systems shown in FIGS. 2A and 2B. FIG. 2A shows a side view of a desktop version of a display system with the display screen positioned in a first position—a position substantially vertical to the surface the display stand is resting on. For purposes of discussion only, assume the embodiments discussed with reference to FIGS. 2A and 2B operate in the viewing mode when in the first position (opaque screen) and the thru-screen mode (transparent screen) in the second position.
  • Referring to FIG. 23 shows a side view of a desktop version of the same display system with the display screen in a second position. Compared to the first position shown in FIG. 2A, the second position shown in FIG. 2B is more horizontal to the surface that the display stand is resting on. Referring to the xyz reference axis shown, the angle of the display screen in the first position shown in FIG. 2A is substantially vertical or at approximately 90 degree to the positive x axis, while the angle of the display screen in the second position shown in FIG. 23 is at an acute angle, say for purposes of example at an angle of approximately 75 degrees relative to the positive x axis. In one example, the first and/or second positions are at a defined angle, however, more commonly the first and second positions are defined within a range of angles. For example, it is common for a user to move the display screen of a laptop or desktop monitor to a position that is at an acute angle to the referenced positive axis. For example, the vertical range of the display screen might be between 80 and 95 degrees relative to the positive x axis for the first position. However, in one embodiment, whatever the angle range—the second position relative to the first viewing position has a smaller acute angle range. This smaller acute angle range occurs in the thru-screen mode as it is desirable to have additional room behind the screen to position an object behind the screen to interact with the object. To achieve this additional space, the user pulls the display screen out and angles the screen at a smaller acute angle range compared to the display screen first position.
  • In one embodiment, the background screen control sensors detect the movement of the hinges with respect to the display stand. For this case, the background control sensor 114 used when the display screen or object that is effected by, coupled to or is part of the display (for example, the hinge) reaches a predefined position. In an alternative embodiment, the background screen control sensors should sense the movement of the display hinges as it moves through a predefined range of motion. In one embodiment, background screen control sensors are coupled to or embedded in the hinges to determine whether so we know when the hinges are in a locked up or locked down position. In one example, the sensors coupled to the hinges are accelerometers that detect the motion of the hinges. In another alternative embodiment, the sensors are electrical components or devices, (i.e., a Hall effect device), which detect closing of a circuit. In one example, when the display screen moves from a first position to a second position a circuit is dosed resulting in a current flow. In one embodiment, the sensor could be a current sensing device which detects current flow when the hinge reaches the second position. In another example, the background control sensor could be any sensor that detects rotation, such as a rotary encoder. In one example, the rotation sensing device coupled to the hinge or bezel of the display senses when the display is at a first or second position or alternatively is moved from a first or second position.
  • As previously stated and as shown in FIGS. 2A-2B in one embodiment the predefined common usage assumption is that when the user changes the position of the display screen, the user wants to change the operational mode. In a second alternative embodiment, the user indicates that he wishes to change the operational mode by placing an object behind the display screen. For example, a user operating the display screen in an operational mode that has an opaque background screen could indicate the intent to switch to a transparent background screen by placing an object (such as a keyboard, etc.) behind the display screen.
  • Referring to FIG. 3 shows a perspective back view of the display system shown in FIG. 1 according to an embodiment of the invention where the sensors 114 a, 114 b can be more clearly seen. In one embodiment, sensors 114 a, 114 b act as background control sensors. The information sensed from sensors 114 a, 114 b from the back surface 158 of the display screen 112 provide information about whether the user has positioned a device or other object behind the display screen indicating the desire to change to the thru-screen mode (transparent background screen). In other words, if a display screen is in a mode which requires an opaque background and an object is positioned behind the display screen, the display screen would automatically change to a transparent screen.
  • As previously stated, the type of sensors used for the background control sensors 114 depends upon the type of interaction that the sensor is required to sense or detect. In this example, the background control sensors should be able to detect an object positioned behind the display screen. In one embodiment, the background control sensors 114 a, 114 b are motion sensors which detect movement of an object behind the screen. In another alternative embodiment, the background control sensors 114 a, 114 b could be an image capture devices. In one embodiment, image analysis software is used to detect changes from the previous image frames or in an alternatively is used to identify the type of object positioned behind the display screen. In one embodiment, the background control sensor can detect the addition or removal of an object from behind the display screen. For example, a camera with a capture region covering the region behind the display screen could be used for to detect the addition or removal of an object from behind the display screen. Thus in one embodiment, for example, an object being removed from behind the display screen could indicate that the user wished to return to the viewing mode or other mode where an opaque background display screen would be desirable.
  • In one embodiment the predefined common usage assumption upon which the predetermined background screen control conditions are based, is that when the user touches the back surface of the display (including the display screen and/or the frame of the display) that the user wants to interact with the display screen in the thru-screen operational mode (transparent display screen background). For example, a user operating the display screen in an operational mode that has an opaque background screen could indicate the intent to switch to a transparent background screen by touching the back surface 158 of the display screen or by touching the back of the frame 154 of the display screen.
  • As previously stated, the type of sensors used for the background control sensors 114 depends upon the type of interaction that the sensor is required to sense or detect. In this example, the background control sensors should be able to detect touch of the back surface of the display screen or display screen frame. In one embodiment, the background control sensors 114 are a plurality of sensors embedded in the bezel 154 around the entire periphery of the display screen that capable of detecting touch on the back surface 158 the display screen surface. In another alternative embodiment, the background control sensor could be a plurality of image capture device which captures a plurality of images behind the display screen surface. Image analysis software is used to determine when an object (including the user's hand) makes physical contact with the display screen.
  • In the embodiment where the background screen changes based on touching the back surface of the display screen, the user might be touching the back surface of the display screen frame merely to adjust the position of the display screen and not to change the operational mode. In one embodiment, an additional check may be made to verify that the intent was to change the mode and thus the background screen—and not just to adjust the position. For example, in one embodiment there may be sensors capable of sensing touch both on the bezel of the front surface of the display and the bezel on the back surface the display screen. In one embodiment, if both the back and front touch sensitive sensors on the bezel indicate that a touch has occurred, then it is assumed that the user is adjusting the display stand position. If only the sensor on the back bezel of the display screen indicates touch, then it is assumed that the user wished to interact in the thru screen mode with a transparent screen background.
  • As previously stated, the background screen control component 118 can be set by system or user controlled instructions that set background screen conditions of the background screen control component. In some cases, the user could set his preferences or additional preferences for automatically switching between background modes. For example, in addition to turning on the transparent mode by touching the back of the screen, touch could be used to turn off the transparent mode. In effect, touch could be used as an off/switch for a given background screen. For instance, a first touch on the back display screen bezel 154 could indicate that the user wishes to change to the thru-screen mode (transparent screen). A second touch on the back bezel of the display screen could indicate that the user wishes to change back to the original background screen. This could be useful when the user is in a mode that is uses an opaque background (i.e., displaying video), and wants to change to a transparent screen in order to interact with an object behind the screen. In an alternative embodiment, the user who might for example often interact with an opaque screen in the viewing mode, might indicate that he wishes to interact with a transparent background screen by touching the front surface of the display screen bezel a predetermined number of times.
  • In one embodiment, instead of a physical user interaction with the display by the user, the user interacts with a user interface that controls the background screen mode. In one embodiment, the user interface is an icon displayed on the display screen. For example, the icon could be a picture of a display with a dear or lighter colored display screen representing the transparent background mode and a darker colored display screen representing the opaque background mode. Alternatively, the display screen icon could represent a transparent background mode when an outline of the display frame is shown and the opaque background mode when the outline of the display frame is shown and in addition the display screen filled in . In one embodiment, the user's selection (clicking on the icon), could change the background screen to correspond to the background screen selected by the icon. In an alternative embodiment, the user interface is a pulldown menu where the user selects a transparent screen or transparent screen based on the user's desired mode of operation. In an alternative embodiment, the user interface is a button embedded in the frame of the display that is pressed in or released (on/off) to change background screen modes.
  • As previously discussed in some cases, the background screen is chosen indirectly by interpreting the sensed user physical interactions. In this case, common usage assumptions are associated with background screen control conditions 136 stored in the background screen control component 118. If background screen control conditions are sensed (for example, hinge movement), then based on the control conditions sensed and the background screen mode associated with it, a background screen is selected. In another case, instead of indirectly choosing the background screen based on the user's physical actions, the user selects the background screen mode directly using a background screen control user interface 141. For example, by simply clicking on the transparent screen icon, the user can change the background screen mode.
  • Referring to FIG. 1 shows a background screen control user interface 141, In one embodiment, based on the user selected background screen mode, the background screen is selected. For example, as previously discussed the user could select an icon representing the desirable background screen or use a pulldown menu to select the desired background screen. The user uses the background screen control user interface 141 to directly select whether the background screen is transparent or opaque.
  • FIG. 4 shows a flow diagram for a method of controlling the background screen display mode based on interaction with the display according to an embodiment of the invention. Referring to FIG. 4 shows the steps of: determining whether a predefined interaction with a display has occurred, wherein the display is capable of displaying at least a first background screen and a second background screen (step 410); and determining whether the predefined interaction with the display meets the background control conditions, wherein responsive to meeting the background control conditions, the background screen corresponding to the predefined conditions is displayed (steps 412, 414).
  • FIG. 5 shows a computer system for implementing the methods shown in FIG. 4 and described in accordance with embodiments of the present invention. It should be apparent to those of ordinary skill in the art that the method 400 represents generalized illustrations and that other steps may be added or existing steps may be removed, modified or rearranged without departing from the scopes of the method 400. The descriptions of the method 400 are made with reference to the system 100 illustrated in FIG. 1 and the system 500 illustrated in FIG. 5 and thus refers to the elements cited therein, It should, however, be understood that the method 400 is not limited to the elements set forth in the system 500. Instead, it should be understood that the method 400 may be practiced by a system having a different configuration than that set forth in the system 500.
  • Some or all of the operations set forth in the method 400 may be contained as utilities, programs or subprograms, in any desired computer accessible medium. In addition, the method 400 may be embodied by computer programs, which may exist in a variety of forms both active and inactive. For example, they may exist as software program(s) comprised of program instructions in source code, object code, executable code or other formats. Any of the above may be embodied on a computer readable medium, which include storage devices and signals, in compressed or uncompressed form.
  • FIG. 5 illustrates a block diagram of a computing apparatus 500 configured to implement or execute the methods 400 depicted in FIG. 4, according to an example. In this respect, the computing apparatus 400 may be used as a platform for executing one or more of the functions described hereinabove with respect to the display controller component 130.
  • The computing apparatus 500 includes one or more processor(s) 502 that may implement or execute some or all of the steps described in the methods 400. Commands and data from the processor 502 are communicated over a communication bus 504. The computing apparatus 500 also includes a main memory 506, such as a random access memory (RAM), where the program code for the processor 502, may be executed during runtime, and a secondary memory 508. The secondary memory 508 includes, for example, one or more hard drives 510 and/or a removable storage drive 512, representing a removable flash memory card, etc., where a copy of the program code for the method 500 may be stored. The removable storage drive 512 reads from and/or writes to a removable storage unit 514 in a well-known manner.
  • These methods, functions and other steps may be embodied as machine readable instructions stored on one or more computer readable mediums, which may be non-transitory. Exemplary non-transitory computer readable storage devices that may be used to implement the present invention include but are not limited to conventional computer system RAM, ROM, EPROM, EEPROM, hard drives, flash memory and magnetic or optical disks or tapes. Concrete examples of the foregoing include distribution of the programs on a CD ROM or via Internet download.
  • Although shown stored on main memory 506, any of the memory components described 506, 508, 514 may also store an operating system 530, such as Mac OS, MS Windows, Unix, or Linux; network applications 532; and a display controller component 130. The operating system 530 may be multi-participant, multiprocessing, multitasking, multithreading, real-time and the like. The operating system 530 may also perform basic tasks such as recognizing input from input devices, such as a keyboard or a keypad; sending output to the display 112 controlling peripheral devices, such as disk drives, printers, image capture device; and managing traffic on the one or more buses 504. The network applications 532 includes various components for establishing and maintaining network connections, such as software for implementing communication protocols including TCP/IP, HTTP, Ethernet, USB, and FireWire.
  • The computing apparatus 500 may also include an input devices 516, such as a keyboard, a keypad, functional keys, etc., a pointing device, such as a tracking ball, cursors, etc., and a display(s) 110, such as the display 110 shown for example in FIGS. 1-3. A display adaptor 522 may interface with the communication bus 504 and the display 112 and may receive display data from the processor 502 and convert the display data into display commands for the display 520.
  • The processor(s) 502 may communicate over a network, for instance, a cellular network, the Internet, LAN, etc., through one or more network interfaces 524 such as a Local Area Network LAN, a wireless 802.11x LAN, a 3G mobile WAN or a WiMax WAN. In addition, an interface 526 may be used to receive an image or sequence of images from imaging components 528 such as the image capture device.
  • The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that the specific details are not required in order to practice the invention. The foregoing descriptions of specific embodiments of the present invention are presented for purposes of illustration and description. They are not intended to be exhaustive of or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in view of the above teachings. The embodiments are shown and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents:

Claims (16)

1. A display system comprising:
a display, the display including a display screen for operating in at least a first background screen mode and a second background screen mode;
at least one background screen control sensor for sensing a physical user interaction with the display; and
a background screen control component for processing background control sensor data from the at least one background screen control sensor, wherein based on the background control sensor data, a determination is made as to whether a background screen control condition has been met, wherein responsive to the determination that the background screen control condition has been met, the background of the display screen is changed.
2. The display system recited in claim 1 wherein when the display is in the first background screen mode, the display screen displays an opaque background and the when the display is in the second background screen mode, the display screen displays a transparent background.
3. The display system recited 1 wherein the interaction with the display is touching the back surface of the display screen.
4. The display system recited 1 wherein the interaction with the display is placing an object within the capture region behind the back surface of the display screen.
5. The display system recited in claim I wherein the display screen is movable to a first screen position and a second screen position.
6. The display system recited in claim 5 wherein the interaction with display is moving the display screen between the first screen position and the second screen position.
7. A method executed on a processor, the method comprising the steps of:
capturing background screen control sensor data from at least one sensor for sensing interaction with a display, the display including a display screen for operating in at least a first background screen mode and a second background screen mode: and
determining based on the captured background screen control sensor data whether a background screen control condition has been met, wherein responsive to the determination that the background screen control condition has been met, the background of the display screen is changed.
8. The method recited ire claim 7 wherein when the display is in the first background screen mode, the display screen displays an opaque background and the when the display is in the second background screen mode, the display screen displays a transparent background.
9. The method recited 7 wherein the interaction with the display is touching the back surface of the display screen.
10. The method recited 7 wherein the interaction with the display is placing an object within the capture region behind the back surface of the display screen.
11. The method recited in claim 7 wherein the display screen is movable to a first screen position and a second screen position.
12. The method recited in claim 7 wherein the interaction with display is moving the display screen between the first screen position and the second screen position.
13. A non-transitory computer readable storage medium having computer readable program instructions stored thereon for causing a computer system to perform a method, the method comprising the steps of:
capturing background screen control sensor data from at least one sensor for sensing interaction with a display, the display including a display screen for operating in at least a first background screen mode and a second background screen mode; and
determining based on the captured background screen control sensor data whether a background screen control condition has been met, wherein responsive to the determination that the background screen control condition has been met, the background of the display screen is changed.
14. The non-transitory computer readable storage medium recited in claim 13 wherein when the display is in the first background screen mode, the display screen displays an opaque background and the when the display is in the second background screen mode, the display screen displays a transparent background.
15. The non-transitory computer readable storage medium recited in claim 13 wherein the display screen is movable to a first screen position and a second screen position, wherein the interaction with display is moving the display screen between the first screen position and the second screen position.
16. A display system comprising:
a display, the display including at least a display screen for operating in at least a first opaque background screen mode and a second transparent background screen mode; and
a background screen user interface control for controlling the background screen mode, wherein responsive to a user selection, the background screen changes to the selected background screen mode.
US13/004,193 2010-10-22 2011-01-11 Display system and method of display for supporting multiple display modes Abandoned US20120098761A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/004,193 US20120098761A1 (en) 2010-10-22 2011-01-11 Display system and method of display for supporting multiple display modes

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
PCT/US2010/053860 WO2012054063A1 (en) 2010-10-22 2010-10-22 An augmented reality display system and method of display
US13/004,193 US20120098761A1 (en) 2010-10-22 2011-01-11 Display system and method of display for supporting multiple display modes

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2010/053860 Continuation-In-Part WO2012054063A1 (en) 2010-10-22 2010-10-22 An augmented reality display system and method of display

Publications (1)

Publication Number Publication Date
US20120098761A1 true US20120098761A1 (en) 2012-04-26

Family

ID=45972595

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/004,193 Abandoned US20120098761A1 (en) 2010-10-22 2011-01-11 Display system and method of display for supporting multiple display modes

Country Status (1)

Country Link
US (1) US20120098761A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120099250A1 (en) * 2010-10-22 2012-04-26 Robinson Ian N Display with rotatable display screen
US8417376B1 (en) * 2011-01-28 2013-04-09 Christopher M. Smolen Method and means for viewing selecting and taking action relative to an item of a group of items
WO2014110567A2 (en) * 2013-01-14 2014-07-17 Andy C Lin Dual-Sided Display for Mobile Devices
US20140232630A1 (en) * 2013-02-15 2014-08-21 International Business Machines Corporation Transparent Display Field of View Region Determination
US9167147B2 (en) 2013-02-15 2015-10-20 International Business Machines Corporation Mobile device field of view region determination
CN106462869A (en) * 2014-05-26 2017-02-22 Sk 普兰尼特有限公司 Apparatus and method for providing advertisement using pupil tracking
US9720511B2 (en) 2012-07-13 2017-08-01 Panasonic Intellectual Property Management Co., Ltd. Hand and object tracking in three-dimensional space
US20180032143A1 (en) * 2011-11-04 2018-02-01 Tobii Ab Portable device
US10621809B2 (en) 2012-01-19 2020-04-14 Christopher M. Smolen Digital currency enabled vending machine

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020044152A1 (en) * 2000-10-16 2002-04-18 Abbott Kenneth H. Dynamic integration of computer generated and real world images
US20030151726A1 (en) * 2001-11-26 2003-08-14 Richard Perkins System and method for displaying physical objects in space
US20050002159A1 (en) * 2002-09-28 2005-01-06 Samsung Electronics Co., Ltd. Monitor
US20050041009A1 (en) * 2003-08-21 2005-02-24 Pioneer Corporation Display system and electronic appliance including the display system
US20060146047A1 (en) * 2003-07-16 2006-07-06 Marion Dale S Three dimensional display method, system and apparatus
US7177144B2 (en) * 2002-05-28 2007-02-13 Samsung Electronics Co., Ltd. Tilting apparatus of monitor
US20070200547A1 (en) * 2006-02-28 2007-08-30 Ming Chen Object detector
US20080084395A1 (en) * 2006-10-05 2008-04-10 Christopher James Dawson Motion based adjustment of display transparency
US20080166022A1 (en) * 2006-12-29 2008-07-10 Gesturetek, Inc. Manipulation Of Virtual Objects Using Enhanced Interactive System
US7724511B2 (en) * 2006-03-03 2010-05-25 Jacobs Matthew B Adjustable display screen for a laptop computer

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020044152A1 (en) * 2000-10-16 2002-04-18 Abbott Kenneth H. Dynamic integration of computer generated and real world images
US20030151726A1 (en) * 2001-11-26 2003-08-14 Richard Perkins System and method for displaying physical objects in space
US7177144B2 (en) * 2002-05-28 2007-02-13 Samsung Electronics Co., Ltd. Tilting apparatus of monitor
US20050002159A1 (en) * 2002-09-28 2005-01-06 Samsung Electronics Co., Ltd. Monitor
US7567436B2 (en) * 2002-09-28 2009-07-28 Samsung Electronics Co., Ltd. Monitor
US20060146047A1 (en) * 2003-07-16 2006-07-06 Marion Dale S Three dimensional display method, system and apparatus
US20050041009A1 (en) * 2003-08-21 2005-02-24 Pioneer Corporation Display system and electronic appliance including the display system
US20070200547A1 (en) * 2006-02-28 2007-08-30 Ming Chen Object detector
US7724511B2 (en) * 2006-03-03 2010-05-25 Jacobs Matthew B Adjustable display screen for a laptop computer
US20080084395A1 (en) * 2006-10-05 2008-04-10 Christopher James Dawson Motion based adjustment of display transparency
US20080166022A1 (en) * 2006-12-29 2008-07-10 Gesturetek, Inc. Manipulation Of Virtual Objects Using Enhanced Interactive System

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120099250A1 (en) * 2010-10-22 2012-04-26 Robinson Ian N Display with rotatable display screen
US8854802B2 (en) * 2010-10-22 2014-10-07 Hewlett-Packard Development Company, L.P. Display with rotatable display screen
US8417376B1 (en) * 2011-01-28 2013-04-09 Christopher M. Smolen Method and means for viewing selecting and taking action relative to an item of a group of items
US10061393B2 (en) * 2011-11-04 2018-08-28 Tobii Ab Portable device
US20180032143A1 (en) * 2011-11-04 2018-02-01 Tobii Ab Portable device
US10621809B2 (en) 2012-01-19 2020-04-14 Christopher M. Smolen Digital currency enabled vending machine
US9720511B2 (en) 2012-07-13 2017-08-01 Panasonic Intellectual Property Management Co., Ltd. Hand and object tracking in three-dimensional space
WO2014110567A3 (en) * 2013-01-14 2015-02-05 Andy C Lin Dual-Sided Display for Mobile Devices
WO2014110567A2 (en) * 2013-01-14 2014-07-17 Andy C Lin Dual-Sided Display for Mobile Devices
US9167147B2 (en) 2013-02-15 2015-10-20 International Business Machines Corporation Mobile device field of view region determination
US9285876B2 (en) 2013-02-15 2016-03-15 International Business Machines Corporation Transparent display field of view region determination
US9535498B2 (en) 2013-02-15 2017-01-03 International Business Machines Corporation Transparent display field of view region determination
US9569686B2 (en) 2013-02-15 2017-02-14 International Business Machines Corporation Mobile device field of view region determination
US9041645B2 (en) * 2013-02-15 2015-05-26 International Business Machines Corporation Transparent display field of view region determination
US20140232630A1 (en) * 2013-02-15 2014-08-21 International Business Machines Corporation Transparent Display Field of View Region Determination
CN106462869A (en) * 2014-05-26 2017-02-22 Sk 普兰尼特有限公司 Apparatus and method for providing advertisement using pupil tracking

Similar Documents

Publication Publication Date Title
US20120098761A1 (en) Display system and method of display for supporting multiple display modes
US9952663B2 (en) Method for gesture-based operation control
EP2405299B1 (en) Information processing device, information processing method, and program
US8665215B2 (en) Single-screen view in response to rotation
US9124686B2 (en) Portable device including automatic scrolling in response to a user's eye position and/or movement
JP5663916B2 (en) Information processing apparatus, display control method, program, and recording medium
JP6272502B2 (en) Method for identifying user operating mode on portable device and portable device
US20120102439A1 (en) System and method of modifying the display content based on sensor input
US20190034147A1 (en) Methods and apparatus to detect user-facing screens of multi-screen devices
US9544556B2 (en) Projection control apparatus and projection control method
US20150185833A1 (en) Display device, display method, and program
US20130002718A1 (en) Image display apparatus, image display control method and program
US8390649B2 (en) Electronic device input control system and method
JP2017211925A (en) Information processing device and image display method
US9400575B1 (en) Finger detection for element selection
US9958946B2 (en) Switching input rails without a release command in a natural user interface
JP2011248465A (en) Information processing apparatus and display control method
KR101890140B1 (en) A method for controlling a display apparatus using a camera device and mobile device, display apparatus, and system thereof
US20160328077A1 (en) Touch sensor
CN111880715A (en) Method and device for editing virtual control in interface, mobile terminal and storage medium
US10963097B2 (en) Method, electronic device, and apparatus for touch-region calibration
EP2634679A1 (en) Two-factor rotation input on a touchscreen device
JP2019096182A (en) Electronic device, display method, and program
KR101265296B1 (en) Apparatus and Method for Recognizing User Input using Camera
CN112684996B (en) Control method and device and electronic equipment

Legal Events

Date Code Title Description
AS Assignment

Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MITCHELL, APRIL SLAYDEN;ROBINSON, IAN N;SOLOMON, MARK C;REEL/FRAME:026855/0715

Effective date: 20110110

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION