CN102027463B - Use multi-touch based on the computer vision sensing of iraser - Google Patents

Abstract

Theme required for protection provides is easy to detect system and/or the method for multiple input simultaneously.Laser assembly is alternatively coupled to produce line generation (LG) Optical devices of laser rays from infrared ray (IR) laser spot, and wherein, laser assembly and line generate (LG) Optical devices and send IR optical plane.Camera installation can catch a part for the image in the region covered by optical plane.Camera installation is alternatively coupled to IR pass filter, and this IR pass filter can stop visible ray and allow IR light pass through, in order to the breach in the IR optical plane that detection sends.Image processing modules may determine that the position of the breach in the region covered by the IR optical plane sent.

Description

Use multi-touch based on the computer vision sensing of iraser

Background

The technical capability of calculating equipment is more and more higher, and wherein, such equipment can provide several functions in the limited device space.Calculating equipment can be, but it is not limited to, mobile communication equipment, desk computer, laptop computer, cell phone, PDA, pager, tablet PC, courier's equipment, portable equipment, pocket translater, barcode scanner, smart phone, scanner, portable hand-held scanner, and any other calculating equipment of data interaction can be carried out.Although each equipment uses the function that user is special, equipment has been developed to allow overlapping function, in order to attract the needs of consumer.In other words, calculating equipment has included multiple feature and/or application program so that each equipment has invaded function each other.Such as, cell phone can provide telephone service, telephone directory, calendar, game, voice mail, paging, web-browsing, video capture, picture catching, sound notepaper, speech recognition, high-end mobile phone (such as, smart phone is increasingly similar to portable computer/laptop computer in feature and function aspects) etc..

As a result, personal computing devices includes the various technology for inputting information and/or method.Personal computing devices is easy to use such as, but is not limited to, and keyboard, keypad, touch pad, touch screen, speaker, instruction pen (such as, identifying pen), the equipment of clipboard etc input information.But, the input equipment of such as keypad, speaker and clipboard etc produces each of which user cannot be similarly used the user individual defect of data entry teclmique (such as, sound, and/or write).Such as, use the consumer writing identification to write in English in the U.S., and there is different alphabetic variants.

Further, it is also possible to use calculating equipment to transmit the data or data interaction inputted via such technique described above.Such as, user can use Desktop Share application program, in order to shares his or her computer screen, and this allows other people to check substantially similar information and/or mutual.In another example, laptop computer can be used together with to lineup's transmission and/or the video data participating in meeting with projector.Furthermore it is possible to be used together PDA with display device (such as, projector, television set, flat faced display, monitor etc.) to carry out presentation slides, and use laser pointer to provide guide at specified point simultaneously.For available data volume, input technology and transmit such data or with data can be the basic aspect of function of equipment, application program, hardware etc. alternately.It addition, adjustment equipment or repacking calculating equipment can be difficult to strengthen input and data sharing technology and be expensive risks.

Summary of the invention

Presented below is the summary of the present invention, in order to provide the basic conception in terms of some described herein.This brief summary is not the detailed general introduction of theme required for protection.The most do not point out the critical elements of theme required for protection, the most do not describe the scope of the present invention.Unique purpose is some concept presenting theme required for protection in simplified form, as the prelude of the ratio presented after a while description in greater detail.

The present invention relates to be easy to utilize multiple touch to detect system and/or the method carrying out data interaction.Specifically, image processing modules can receive a part of data of the breach detected related in infrared ray (IR) optical plane, wherein, image processing techniques can be utilized to analyze such breach, in order to corresponding position, size and the shape being mapped on the image of display or projection.It is said that in general, the present invention can allow at least one breach in the IR optical plane sent by seizure to detect gesture, touch event, motion, and/or object.Laser assembly is alternatively coupled to line and generates (LG) Optical devices, in order to launch IR optical plane on the most essentially flat surface.Can catch or detect the breach in IR optical plane by the camera installation coupled with IR pass filter.Such breach can be received by image processing modules and change, in order to strengthen the data interaction with such as display.

According to a further aspect in the invention, image processing modules can be used together with application framework (such as, plug-in unit etc.).Application framework can utilize touch event that is that detect and that capture, gesture, motion etc., in order to carries out the data interaction strengthened.Such as, application framework can include surface computing technique, in order to can carry out data manipulation that is seamless and that flow.Other aspects at theme required for protection, it is provided that be easy to utilize multiple laser instrument to carry out data collection and mutual method based on the breach detected in infrared ray (IR) optical plane.

Explained below and accompanying drawing illustrating in detail some illustrative aspect of theme required for protection.But, these aspects are it is merely meant that can be to use some modes in the various modes of the principle of the present invention, and theme required for protection is intended to include all these aspect and equivalent.By reading the detailed description of the following present invention together with accompanying drawing, other advantages of theme required for protection and novel feature will become clear from.

Accompanying drawing is sketched

Fig. 1 shows the block diagram facilitating implementation the example system mutual with multiple touches of display.

Fig. 2 shows to be easy to utilize multiple laser instrument to carry out data collection and the block diagram of mutual example system based on the breach detected in infrared ray (IR) optical plane.

Fig. 3 shows the block diagram being easy to according to the touch event offer detected with the example system of the data interaction of display.

Fig. 4 shows to be easy to utilize at least one IR emitting laser to carry out the data collection with application framework and the block diagram of mutual example system.

Fig. 5 shows the block diagram being easy to realize the mutual example system of multiple touch by detecting the breach in the IR optical plane on surface projects.

Fig. 6 shows that be easy to use at least camera installation and laser assembly according to the present invention detect the block diagram of multiple mutual example system.

Fig. 7 shows to be easy to utilize at least one laser instrument to carry out data collection and the block diagram of mutual illustrative methods based on the breach detected in infrared ray (IR) optical plane.

Fig. 8 shows to be easy to utilize at least one IR emitting laser to carry out the data collection with application framework and the block diagram of mutual illustrative methods.

Fig. 9 is shown in which to use the example network environment of the novel aspect of theme required for protection.

Figure 10 shows the Illustrative Operating Environment that can use according to theme required for protection.

Detailed description of the invention

With reference now to accompanying drawing, describe theme required for protection, all accompanying drawings use identical reference to refer to identical element.In the following description, for the sake of explaining, numerous detail is illustrated to provide complete understanding of the present invention.It may be evident, however, that theme required for protection can be implemented in the case of not having these details.In other cases, show in block form structure known to each and equipment so that describing the present invention.

As used herein, term " assembly ", " system ", " data storage ", " engine ", " laser instrument ", " Optical devices ", " photographing unit ", " display ", " equipment " etc. are intended to refer to the entity relevant with computer, either hardware, software (such as, operating software) and/or firmware.Such as, assembly it may be that run on a processor process, processor, object, executable code, program, function, storehouse, subroutine, and/or computer or the combination of software and hardware.As explanation, the application program run on the server and server can be both assemblies.One or more assemblies may reside within process, and assembly may be located in a computer and/or is distributed between two or more computers.

Additionally, theme required for protection can use generation to control computer is implemented as method, device or goods with standard program and/or the engineering of the software of theme, firmware, hardware or its combination in any disclosed in realization." goods " can comprise the computer program that can conduct interviews from any computer-readable equipment, carrier or medium as the term is used herein.Such as, computer-readable medium can include but not limited to, magnetic storage apparatus is (such as, hard disk, floppy disk, magnetic stripe ...), CD (such as, CD, DVD ...), smart card and flash memory device (such as, card, rod, key drive ...).In addition, it is to be appreciated that carrier wave can be used to carry computer-readable electronic data, as those in transmission and receive Email or used when accessing the network of such as the Internet or LAN (LAN) etc.Certainly, it will be appreciated by those skilled in the art that on the premise of without departing substantially from the scope of theme required for protection or spirit, this configuration can be carried out many amendments.Additionally, use word " exemplary " to mean serving as example, example or explanation at this.It not necessarily is construed as preferred or advantageous over other aspects or design as any aspect described by " exemplary " or design herein.

Turning now to figure, Fig. 1 shows and facilitates implementation the system 100 mutual with multiple touches of display.System 100 can include the image processing modules 102 that can realize data interaction based on the touch event detected, wherein it is possible to detect touch event by infrared ray (IR) light utilizing camera installation 108 to follow the tracks of from laser assembly 104.It is said that in general, system 100 can detect motion, gesture, input, touch event, object etc. by detection breach from the IR optical plane that laser assembly 104 is launched.Can assess and process the breach in IR optical plane by image processing modules 102.It is appreciated that and can monitor, by camera installation 108, the such optical plane sent from laser assembly 104 in real time.The breach detected being based at least partially in IR optical plane, breach can be converted to the position of correspondence, shape, size etc. for use in data interaction by image processing modules 102.Specifically, laser assembly 104 is alternatively coupled to line and generates (LG) Optical devices 106, to produce laser rays from laser spot.LG Optical devices 106 and laser assembly 104 can send optical plane, wherein it is possible to detect and process the breach in optical plane by camera installation 108 and image processing modules 102 respectively.It is appreciated that camera installation 108 can include IR pass filter 110, in order to stop visible ray and allow IR light pass through.

System 100 may be used for catching touch event, e.g., but is not limited to, gesture, hand motion, and hands is mutual, object is mutual, and/or can disconnect any other suitable material object of IR optical plane.For example, it is possible to the hands being detected the IR optical plane that disconnection sends by system 100 in real time is mutual, in order to the corresponding data interaction being converted on display.So, touch event detection can be realized by configuring the laser assembly 104 with LG Optical devices 106, with the camera installation 108 of IR pass filter 110 and image processing modules 102.Being appreciated that touch event can be any suitable breach of the IR optical plane sent by laser assembly 104, wherein, breach is the obstruction of a part for the optical plane sent from LG Optical devices 106 and laser assembly 104.It will further be understood that system 100 efficient, cost-effective and the adaptable solution of legacy equipment that is small-sized, size, in order to touch event detection is provided.

Such as, IR optical plane can be broken by an object.Based on such breach being detected, image processing modules 102 can be with analytical data, in order to home position, size, shape, breach characteristic (such as, vestige, persistent period etc.), etc..Such data can be converted to by image processing modules 102, such as, and the corresponding position on display.Display (such as, being more thoroughly discussed below) can also present corresponding shape and/or the size of the object causing the breach in IR optical plane.Such as, slide demonstration can be shown, projection person can by means of system 100 (such as, the hand motion captured, and change to be sent to display etc.) interact (such as with such data, highlight information, utilize gesture to change lantern slide, open application program, touch screen input etc.).In another example, the desk surface of user can have the IR optical plane transmitted to abreast, in order to by desk surface transformation be for data input or the region of touch event detection.So, the data interaction of data on such as monitor or display is may be used for from the breach in the IR optical plane on desk surface.In another example, system 100 can utilize monitor to realize, wherein it is possible to launch IR optical plane abreast with monitor screen or surface.The detection of such breach can be changed by image processing modules 102, with the data corresponding to any display.

Although be by mean of from laser assembly 104 generate IR light make for discussing system 100, but, it is appreciated that, it is possible to use any suitable light source or laser assembly 104 generate the optical plane (such as, the breach in optical plane may be used for detection motion) that can be disconnected.Such as, the red laser of enough brightness can be used (such as, any other suitable laser instrument of green (light) laser and/or no matter color) and/or the darkest room, in order to detect and/or identify the breach allowing to carry out in such light source of motion detection.Although the light of visible laser or light source may make user divert one's attention, but, it can also be provided with by feedback in the case of optical plane is substantially parallel and above surface.Additionally, optical plane can be above display surface and leaves display surface (such as, no matter the wavelength of laser or light source how).Such as, such example may be used for game, game on line, control station game etc..

Additionally, system 100 can include any suitable and/or required interface module (not shown), these assemblies provide various adapters, adapter, channel, communication path etc., so that image processing modules 102, laser assembly 104, LG Optical devices 106, camera installation 108, and/or IR pass filter 110 are integrated in substantially any operating system and/or Database Systems and/or the most integrated.Additionally, these interface modules may be provided for various adapter, adapter, channel, communication path etc., for with image processing modules 102, laser assembly 104, LG Optical devices 106, camera installation 108, and/or IR pass filter 110, and any other equipment and/or the assembly that are associated with system 100 interact.

Fig. 2 shows to be easy to utilize multiple laser instrument to carry out data collection and mutual system 200 based on the breach detected in infrared ray (IR) optical plane.System 200 can include the image processing modules 102 that may determine that the position being associated with the breach detected in the IR optical plane launched.The position of the breach of IR optical plane can be converted to the position carrying out data interaction of correspondence by image processing modules 102.As discussed, available the camera installation 108 of IR pass filter 110 is employed to detect the breach in IR optical plane.It is further appreciated that camera installation 102 can be any suitable vision facilities that can catch the image being associated with IR light and/or data.Such as, camera installation 108 can be photographing unit, video camera, IR photographing unit, IR video camera, IP Camera, smart phone with photographing unit, and/or any other the suitable equipment with camera-enabled.

IR optical plane can be generated, such as laser assembly by the laser assembly 202 of any suitable quantity₁To laser assembly_N, wherein, N is positive integer.The bigger surface area of IR optical plane can be provided to cover it will further be understood that laser assembly quantity increases, this can strengthen touch event detection.As depicted, each laser assembly 202 may comprise corresponding LG Optical devices, but it is appreciated and understood that, it is possible to use the LG Optical devices (such as, two or more laser assemblies can share single LG Optical devices etc.) of any suitable quantity.Such as, in one configuration, it is being shaped as on foursquare essentially flat surface, first laser assembly can be placed in first jiao (such as, northwest corner), second laser assembly can be placed in second jiao (such as, northeast corner), to allow to be parallel to this to project IR optical plane outwardly.In another example, square surface can include laser assembly at each angle, in order to the surface area providing bigger covers (such as, bigger IR optical plane will cover the surface area of square surface).

In particular example, camera installation 108 can be installed and be incident upon surface (for example, it is possible to projecting the surface of IR optical plane in parallel on it).At least one laser assembly and corresponding LG Optical devices can be installed in projection surface, in order to the optical plane sent from which is parallel to the surface being projected.In other words, laser assembly is oriented so that the infrared light from LG Optical devices constitutes the infrared light plane skimming over projection surface.It is appreciated that and can adjust from projection surface to the height of IR optical plane based on the purposes planned or the preference of data interaction.

System 200 can also include data storage 204, and this data storage 204 can include any suitable data relating to image processing modules 102, at least one laser assembly 202, LG Optical devices, camera installation 108, IR pass filter 110 etc..nullSuch as，Data storage 204 can include，But it is not limited to include，Laser assembly configuration is arranged、Camera installation configuration is arranged、Laser assembly is towards setting、Laser assembly is towards definition、User preference、User password、User name、Map、Image processing data、Breach detection data、The sensitivity relating to breach detection is arranged、Position data、IR sensitivity is arranged、Relate to the sized data of the reason of breach in IR optical plane、Relate to the shape data of the reason of breach in IR optical plane、Image processing techniques、Image processing data、Image trace data、Binarization data、IR modulates data、LG Optical devices are arranged (such as，Development size、Intensity etc.)、Surface area data is (such as，The size on surface、The position on surface、Surface type etc.)，Gesture、Motion、Application framework data etc..

It is appreciated that data storage 204 it may be that such as, volatile memory or nonvolatile memory, or volatibility and nonvolatile memory can be included.As explanation, rather than limit, nonvolatile memory can include read only memory (ROM), programmable read-only memory (PROM), electrically programable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory.Volatile memory can include the random access memory (RAM) serving as external cache.As illustrating and noting limit, RAM can be there to be many forms, such as static RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), Double Data Rate SDRAM (DDRSDRAM), the SDRAM (ESDRAM) strengthened, synchronization link DRAM (SLDRAM), Direct-type Rambus RAM (RDRAM), Direct-type Rambus dynamic ram (DRDRAM) and Rambus dynamic ram (RDRAM).The data storage 204 of system and method is intended to include, is not limited solely to, the memorizer of these and any other suitable type.In addition, it is to be appreciated that data storage 204 can be server, data base, hard disk drive, pen type driver, external fixed disk drive, portable hard disc drives etc..

Fig. 3 shows the system 300 being easy to according to the touch event offer detected with the data interaction of display.System 300 can include the image processing modules 102 that can receive any suitable data for carrying out data interaction of the breach detected related in IR optical plane.Specifically, the touch event (such as, object, gesture, motion, hands, material object etc.) of the IR optical plane launched from laser assembly 104 (with LG Optical devices 106) can be disconnected by camera installation 108 (with IR pass filter 110) detection.It will further be understood that system 300 can follow the tracks of touch event in real time, in order to allow data interaction.For example, it is possible to the breach being monitored by system 300 and being detected in IR optical plane, in order to breach and the real-time video of movement associated there are provided.

Image processing modules 102 can utilize any suitable image processing techniques so that by the image alignment of the projection on the image (such as, from camera installation 108 and IR pass filter 110) captured and display 302.For example, it may be determined that point in the image captured (such as, the position of breach, the shape of breach, the size etc. of breach), and/or can be with the corresponding point in the image of projection on calculation display 302.In another example, the hands of user or finger can be placed in IR optical plane by surface the most projected thereon.In this example, finger or hands can catch a part of IR light (such as, disconnect IR optical plane), and this can be captured as the part (such as, utilizing IR pass filter 110) in clear zone by camera installation 108.Such a part of clear zone can be detected as followed the tracks of object by image processing modules 102.Such as, image processing modules 102 can use binarization, graphical analysis, the graphical analysis that captures etc..In addition, it is also understood that, display 302 can be any suitable display module, as, but it is not limited to, monitor, television set, liquid crystal display (LCD), plasma, rear projection display, front throwing display, cathode ray tube (CRT) monitor, flat faced display, the display of calculating equipment, portable digital-assistant (PDA) screen, laptop display, computer monitor, smart phone display, honeycomb equipment screen, mobile communication equipment display, portable game device display etc..

Fig. 4 shows that being easy to utilize at least one IR emitting laser to come with application framework carries out data collection and mutual system 400.System 400 can include image processing modules 102, and this assembly 102 allows corresponding position, the dimension and shape being converted to by the breach detected in IR optical plane on display 302.Specifically, laser assembly 104 is alternatively coupled to LG Optical devices 106, in order to generate IR optical plane.IR pass filter 110 can be utilized to detect the breach of such IR optical plane or to interrupt (such as, the object of a part for reflection IR optical plane) by camera installation 108.

The size of the mapping of the breach of IR optical plane, conversion, and correspondence, shape, and/or position is displayed for device 302.For example, it is possible to detected motion, gesture and/or object by system 300, wherein, such motion, gesture and/or object can be shown and/or copy on display 302.Specifically, instruction pen can disconnect IR optical plane when write, and system 300 can carry out the such motion of detect and track by breach based on IR optical plane, in order to such motion is presented on the display 302.Additionally, detect that a certain special exercise or gesture can trigger a certain function or output.Such as, detect that the motion such as drawing a question mark etc can trigger help adjustment or application program.

Being appreciated and understood that, the touch event (such as, motion, gesture, object etc.) detected can be used in combination with any suitable assembly, equipment, application program, software, hardware etc..Specifically, the touch event detected can be used together with application framework 402.Such as, application framework 402 can include with surface calculate be associated for the software interacted with data (such as, image, icon, file, video, sound clip, data etc.).Surface software for calculation can allow movement and/or gesture to control such as, but is not limited to, scaling, pan, the function of stretching, extension etc.Additionally, surface calculate application program can identify different shapes or object (such as, the video play in the rectangle detected of the size of definition, the cell phone image detected, image shown after specific pattern being detected, etc.).It is said that in general, surface component can be software and/or the part for hardware using special computer graphic user interface (GUI).Such as, user can directly interact with touch sensitive screen or display, wherein, system 400 catches touch event detection.In another example, application framework 402 can include relating to the software that digital content is mutual, and this software allows the hands utilizing user to capture data, and utilizes natural gesture and touch to move such information between objects.Specifically, tabletop display can be used together with system 400 in conjunction with the touch event detected, in order to data carry out seamless alternately.

Fig. 5 shows that the breach being easy to by the IR optical plane that projects from the teeth outwards of detection realizes the mutual system of multiple touch 500.System 500 can include image processing modules 102, and this assembly 102 can detect the live touch event disconnecting the IR optical plane utilizing LG Optical devices 106 to send from laser assembly 104.Specifically, surface 502 can be parallel to and project IR optical plane, wherein it is possible to utilized IR pass filter 110 to detect the breach in such IR optical plane by camera installation 108.Disconnecting IR optical plane based on object, detection allows camera installation 108 and IR pass filter 110 to catch the IR light of such touch event (breach in such as, motion, gesture, IR light etc.).It is appreciated that, surface 502 can be any essentially flat surface, such as, but it is not limited to, desk, desk, wall, display, television set, laptop computer, floor, ceiling, the part on ground, a part for concrete, one block of wood, blackboard, projection screen, windshield, automobile window, fascia, the essentially flat surface being associated with equipment, relate to automobile (such as, car, truck, all-terrain vehicle (ATV), motorcycle, autocycle, motor scooter etc.) essentially flat surface, or the pallet on aircraft etc..It is said that in general, the present invention can be used together with various daily surfaces, to carry out data interaction via the touch event detected.In another example, system 500 can include (such as, laser assembly 104 and/or LG Optical devices 106 are embedded in desk etc.) in surface 502.

Touch event detection system 500 can also use various setting for laser assembly 104 and/or LG Optical devices 106.For example, it is possible to be based at least partially on the area that in surface 502, IR optical plane should cover, adjust the development size of laser assembly 104.Bigger development size (such as, the angle of visual field of increase) can cover the bigger angle on surface 502, but scope will reduce.Less development size can cover with the angle of limiting surface 502 (such as, cone), but increases the scope distance of laser instrument (such as, with).It is appreciated that and the setting of laser assembly 104 can be carried out any suitable adjustment.

System 500 can also include IR modulation component 504, in order to compensates the IR light not generated by laser assembly 104 and/or LG Optical devices 106.The IR light that do not generated by laser assembly 104 and/or LG Optical devices 106 it may be that such as, dim (fray) daylight, ambient light, incandescence, halogen light etc..In order to compensate or remove this IR light, IR modulation component 504 can open and close laser assembly 104, and wherein, multiple images include that laser assembly 104 is in the image pair of on and off.In other words, one image can have ambient light (such as, the laser assembly 104 utilizing camera installation 108 and be in "Off" state detects, also referred to as first set), and another image can have ambient light and add the light of IR generation (such as, utilize camera installation 108 and be in " opening " state laser assembly 104 and detect, the also referred to as second set).The two image can be subtracted (such as, the second set being deducted the first set) to remove ambient light from detection.This is used as light filter to compensate the ambient light or the IR light of refraction not generated by laser assembly 104 and/or LG Optical devices 106.

Fig. 6 shows that utilize at least camera installation and laser assembly easy to use according to the present invention detect the system 600 of multiple mutual intelligence.System 600 can include image processing modules 102, laser assembly 104, LG Optical devices 106, camera installation 108, and IR pass filter 110, they are substantially similar to corresponding assembly, Optical devices, equipment and wave filter described in figure above.System 600 also includes intelligent assembly 602.Intelligent assembly 602 can be used for promoting that use system 600 configuration detects touch event by image processing modules 102.Such as, the breach position that intelligent assembly 602 may infer that IR optical plane, the size of breach related in IR optical plane, the shape of breach related in IR optical plane, the tracking of touch event related in IR optical plane, user preference, LG Optical devices 106 are arranged, camera installation 108 towards, laser assembly 104 towards, relate to the setting of display, modulating frequency/setting, ambient light detection, gesture assessment, locomotion evaluation, touch event prediction etc..

Intelligent assembly 602 can use information value (VOI) to calculate, in order to the breach in mark IR optical plane, and characteristic associated there (such as, size, position, shape etc.).Such as, by utilizing VOI to calculate, can identify and use the most preferable and/or suitable breach detection or touch event (the first touch event such as, being associated can be less than second touch event being associated with the second data interaction etc. in terms of priority) with the first data interaction.It will further be understood that intelligent assembly 602 can by via event and/or data capture to one group of observed result reasoning or infer the state of system, environment and/or user.Deduction can be used to identify specific context or operation, it is also possible to generate, such as the probability distribution in state.Deduction can be probabilistic, i.e. based on the consideration to data and event, calculates the probability distribution in states of interest.Infer the technology that may also mean that the event for constituting higher level from one group of event and/or data.Such deduction causes building new event or operation from the event data of one group of event observed and/or storage, and no matter the most whether event is closely related, and event and data are from one or multiple event and data source.Can be together with performing to relate to the automatic of theme required for protection and/or inference operations; use various classification (through explicitly and/or implicitly train) scheme and/or system (such as; support vector machine, neutral net, specialist system, bayesian belief networks, fuzzy logic, Data Synthesis engine, etc.).

Grader is that input property vector x=(x1, x2, x3, x4, xn) is mapped to the function that this input belongs to the confidence level of some class, i.e. f (x)=confidence level (class).Such classification can use analysis based on probability and/or statistics (such as, resolving into analysis utilities and cost) predict or infer that user wishes the action automatically carried out.Support vector machine (SVM) is an example of the grader that can use.SVM is operated by the hypersurface in the space of the input having found that it is likely that, wherein, hypersurface attempts to separate triggering criterion with non-trigger events.Intuitively, this makes classification to correct and is close to close with training data but incomplete same test data.Other oriented and undirected category of model methods include, such as, and naive Bayesian, Bayesian network, decision tree, neutral net, fuzzy logic model, it is possible to use the probabilistic classification models of different independence pattern is provided.Classification as used herein also includes the statistical regression being used to develop models of priority.

Image processing modules 102 can also utilize and present assembly 604, and this presents assembly 604 provides various types of user interface, in order to mutual between user and any assembly being coupled to image processing modules 102.As depicted, presenting assembly 604 is the corpus separatum can being used together with image processing modules 102.However, it is to be appreciated that present assembly 604 and/or similar check that assembly can be merged in image processing modules 102 and/or as separate unit.Present assembly 604 and one or more graphic user interface (GUI), Command Line Interface etc. can be provided.For example, it is possible to present provide a user with for data being loaded, importing, the region of reading etc. or means, and the GUI demonstrating the region of the result of such operation can be included.These regions can include known text and/or graphics field, including dialog box, static control, drop-down menu, list box, pop-up menu, edit control, combo box, radio button, check box, button and graphical boxes.Furthermore it is also possible to use the utility program being easy to present, if the vertical and/or HScrollBar for navigating and toolbar button are to judge whether a certain region can be checked.Such as, user can be coupled to and/or one or more interacting in the assembly processing assembly 102 that merges to image.In addition, it is to be appreciated that present assembly 604 can use surface software for calculation, hardware, and/or its any suitable combination.

User can also interact to select and offer information with each region with via various equipment (such as, such as, mouse, roller ball, touch pad, keypad, keyboard, touch screen, pen and/or voice activation, human motion detection).Generally, the mechanism of the enter key on such as button or keyboard etc can be used after entering the information to start search.However, it is to be appreciated that theme required for protection is not so limited.Such as, only highlight a check box and can start information reception and registration.In another example, it is possible to use Command Line Interface.Such as, Command Line Interface can prompt the user with information (such as, via the text message on display and audio tones) via providing text message.Then, user can provide suitable information, such as alphanumeric corresponding with the option provided in interface prompt input or the answer to the problem proposed in prompting.It is appreciated that Command Line Interface can be used in combination with GUI and/or API.It addition, Command Line Interface can also be with hardware (such as, video card) and/or have the display (such as, black and white, EGA, VGA, SVGA etc.) that limited figure is supported, and/or low-bandwidth communication channel is used in combination.

Fig. 7-8 shows the method according to theme required for protection and/or flow chart.For simplicity, describe as a series of actions and describe method.It is to be understood and appreciated that the present invention is not limited only to the order of shown action and/or action.Such as, some actions can be carried out in various orders and/or concurrently, and with other acts not presented and described herein.Additionally, the most all of shown action is all to realize according to necessary to the method for theme required for protection.It addition, those skilled in the art will understand and appreciate that, method can also be alternatively a series of states being mutually associated connection via state diagram or representations of events.It addition, it will be further understood that the method below and disclosed in entire disclosure can be stored in a kind of goods, to promote such method transmission and to be transferred in computer." goods " can comprise the computer program that can conduct interviews from any computer-readable equipment, carrier or medium as the term is used herein.

Fig. 7 shows to be easy to utilize at least one laser instrument to carry out data collection and mutual method 700 based on the breach detected in infrared ray (IR) optical plane.At Ref. No. 702, a part of data relating at least one breach in the plane of infrared ray (IR) light sent being detected can be received.For example, it is possible to the plane of the IR light sent from any suitable laser assembly that can launch IR optical plane.Specifically, laser assembly can use line to generate (LG) Optical devices, in order to provides IR optical plane.Can be by, for instance, it is possible to any suitable equipment of detection IR luminous reflectance catches a part of data relating at least one breach in the IR optical plane sent being detected.

At Ref. No. 704, the position of breach in the IR optical plane sent can be calculated.It is based at least partially on the breach detected in the IR optical plane sent, the corresponding position of the breach detected can be calculated.For example, it is possible to analyze relate to breach in IR optical plane detection data, in order to combine the surface area covered by IR optical plane to identify the position of breach.At Ref. No. 706, determine at least one in the shape of the breach in the size of breach in the IR optical plane sent or IR optical plane.For example, it is possible to the dimension and shape of the breach in mark IR optical plane, in order to present over the display and describe accurately.So, compared with the finger disconnecting IR optical plane, can calculate and/or determine the hands disconnecting IR optical plane exactly.In other words, it may be determined that disconnect the position of object of IR optical plane, dimension and shape.At Ref. No. 708, the position of breach, dimension and shape in the IR optical plane that can will send are mapped to display.

Although be by mean of making for discussion method 700 of IR light, however, it will be understood that any suitable light source or laser assembly 104 can be used to generate the optical plane (such as, the breach in optical plane may be used for detection motion) that can be disconnected.It is, for example possible to use have the red laser of enough brightness and/or the darkest room, in order to detect and/or identify the breach allowing to carry out in such light source of motion detection.Although the light of visible laser or light source may make user divert one's attention, but, it can also be provided with by feedback in the case of optical plane is substantially parallel and above surface.Additionally, optical plane can be above display surface and leaves display surface (such as, no matter the wavelength of laser or light source how).Such as, such example may be used for game, game on line, control station game etc..

Fig. 8 shows that being easy to utilize at least one IR emitting laser to come with application framework carries out data collection and mutual method 800.At Ref. No. 802, can be substantially parallel to surface and send infrared ray (IR) optical plane.It is appreciated that, surface can be, but it is not limited to, essentially flat surface that desk, desk, wall, display, television set, laptop computer, floor, ceiling, the part on ground, a part for concrete, one piece of wood, blackboard, projection screen, windshield, automobile window, fascia are associated with equipment, the essentially flat surface relating to automobile (such as, car, truck, all-terrain vehicle (ATV), motorcycle, autocycle, motor scooter etc.) etc..It is said that in general, the present invention can be used together with various daily surfaces, to carry out data interaction via the touch event detected.In another example, system 500 can be included surface 502 (such as, laser assembly 104 and/or LG Optical devices 106 are embedded in desk etc.), pallet on aircraft, and/or in any other the most smooth suitable surface.

At Ref. No. 804, at least one breach in IR optical plane can be followed the tracks of based on a part for real-time IR light detection.In other words, the tracking to breach can be provided to monitoring in real time of IR optical plane, also have the tracking to motion based on the continuous print tracking to IR optical plane.At Ref. No. 806, it is possible to use image processing techniques calculates at least one in the position of the breach in IR optical plane, size or shape.For example, it is possible to present breach that is that detect and that calculate or map that to the display of correspondence, wherein, such display can illustrate with corresponding position, the breach of the duplication of size and/or shape.

At Ref. No. 808, the tracked breach in IR optical plane can be used together with a part for surface software for calculation, to strengthen data interaction.Surface software for calculation can allow movement and/or gesture to control such as, but is not limited to, scaling, pan, the function of stretching, extension etc.Additionally, surface calculates application program can identify different shapes or object (video such as, play in the rectangle detected of the size of definition, the cell phone image detected, image shown after AD HOC being detected etc.).It is said that in general, surface component can be software and/or the part for hardware using special computer graphic user interface (GUI).In other words, the mode using IR light and described technology to detect gesture, motion and/or touch event can be used in combination with any suitable surface software for calculation, hardware and/or its any suitable combination.

In order to provide the additional context of various aspects for realizing theme required for protection, Fig. 9-10 and discussed below aim to provide brief, the general description of the suitable computing environment that wherein can realize various aspects of the invention.Such as, it is used to the image processing modules detecting touch event or interacting with IR optical plane, as described by figure above, can realize in such suitable computing environment.Although being to describe theme required for protection in the general context of the computer executable instructions of the computer program that can run on local computer and/or remote computer above; but; those skilled in the art is it will be recognized that the present invention can also realize in combination with other program modules.It is said that in general, program module includes performing particular task or realizing the routine of particular abstract data type, program, assembly, data structure etc..

In addition, those skilled in the art will be appreciated by, other computer system configurations can be utilized to the method implementing the present invention, including uniprocessor and/or multiprocessor computer system, minicomputer, mainframe computer, and personal computer, Handheld computing device, based on microprocessor and/or programmable consumption electronic product etc., each equipment above-mentioned can operationally communicate with one or more equipment being associated.The illustrated aspect of theme required for protection can also task be realized by the distributed computing environment performed by the remote processing devices of communication network links wherein.But, certain aspects of the invention, if all aspects, can implement on stand-alone computers.In a distributed computing environment, program module may be located at stored locally and/or remotely in device storage device.

Fig. 9 is the schematic block diagram of the example computing device 900 that theme required for protection can interact.System 900 includes one or more client computer 910.Client computer 910 can be hardware and/or software (such as, thread, process, calculating equipment).System 900 also includes one or more server 920.Server 920 can also be hardware and/or software (such as, thread, process, calculating equipment).Server 920 is permissible, such as, the application of the invention, preserve the thread performing conversion.

A kind of possible communication between client computer 910 and server 920 can be to present for the form of the packet of transmission between two or more computer processes.System 900 includes communications framework 940, and this communications framework 940 can be used to promote the communication between client computer 910 and server 920.Similarly, client computer 910 is operably connected to one or more client data store 950, it is possible to use the information of storage client 910 this locality is carried out in these client data store 950.Similarly, server 920 is operably connected to one or more server data storages 930, it is possible to use these server data storages 930 store the information of server 920 this locality.

With reference to Figure 10, the exemplary environments 1000 of the various aspects for realizing theme required for protection can include computer 1012.Computer 1012 includes processing unit 1014, system storage 1016 and system bus 1018.System bus 1018, by system component, include but not limited to, and system storage 1016 is coupled to processing unit 1014.Processing unit 1014 can be any one of various processor.Dual micro processor and other multiprocessor architecture can also be used as processing unit 1014.

If system bus 1018 can be any one of bus structures of dry type, including the memory bus or the Memory Controller that use any one of various available bus architecture, peripheral bus or external bus and local bus, bus architecture includes, but it is not limited only to, industry standard architecture (ISA), MCA (MCA), the ISA (EISA) of extension, Intelligent Drive Electronics (IDE), VESA local bus (VLB), periphery component interconnection (PCI), card bus, USB (universal serial bus) (USB), advanced graphics port (AGP), personal computer memory card international federation's bus (PCMCIA), live wire (IEEE 1394) and small computer system interface (SCSI).

System storage 1016 includes volatile memory 1020 and nonvolatile memory 1022.Basic input/output (BIOS) is generally stored inside in nonvolatile memory 1022, comprises the basic routine transmitting information between the element such as helped in start-up course in computer 1012.As explanation, rather than limit, nonvolatile memory 1022 can include read only memory (ROM), programmable read-only memory (PROM), electrically programable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory.Volatile memory 1020 includes the random access memory (RAM) serving as external cache.As illustrating and noting limit, RAM can be there to be many forms, such as static RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), Double Data Rate SDRAM (DDR SDRAM), the SDRAM (ESDRAM) strengthened, synchronization link DRAM (SLDRAM), Direct-type Rambus RAM (RDRAM), Direct-type Rambus dynamic ram (DRDRAM) and Rambus dynamic ram (RDRAM).

Computer 1012 also includes movably/immovable, and the computer-readable storage medium of volatile, nonvolatile.Figure 10 shows such as disk memory 1024.Disk memory 1024 includes but are not limited to, the equipment of such as disc driver, floppy disk, tape drive, Jaz driver, Zip drive, LS-100 driver, flash card or memory stick etc.Additionally, disk memory 1024 may also include storage medium, combine dividually or with other storage mediums, including, but it is not limited only to, the CD drive (CD-ROM) of such as compact disk ROM device etc, the recordable driver of CD (CD-R drive), CD recordable drive (CD-RW drive) or digital versatile disc ROM drive (DVD-ROM).It is connected to system bus 1018 for ease of disk storage equipment 1024, generally uses may move or irremovable interface of such as interface 1026 etc.

It is appreciated that Figure 10 describes the software of the intermediary in suitable operating environment 1000 between user and described basic computer resources.Such software includes operating system 1028.The operating system 1028 can being stored on disk memory 1024 is for controlling and distribute the resource of computer system 1012.System application 1030 utilizes by operating system 1028 by being stored in system storage 1016 or the program module 1032 being stored on disk memory 1024 and the routine data 1034 management to resource.It is appreciated that theme required for protection can utilize the combination of various operating system or operating system to realize.

User is ordered or information to computer 1012 input by input equipment 1036.Input equipment 1036 includes, but it is not limited to, the instruction equipment of such as mouse, trace ball, instruction pen, touch pad etc, keyboard, mike, joystick, game paddle, dish, scanner, TV tuner card, digital camera, digital camera, IP Camera etc..These and other input equipment is connected to processing unit 1014 by system bus 1018 via interface port 1038.Interface port 1038 includes, such as, and serial port, parallel port, game port and USB (universal serial bus) (USB).Outut device 1040 and input equipment 1036 use the port of some same types.So, it is, for example possible to use USB port to provide input to computer 1012, and information is exported from computer 1012 to outut device 1040.Provide o adapter 1042, with some outut devices 1040 shown with such as monitor, speaker and printer etc, also there is a need to other outut devices 1040 of private adapter.O adapter 1042 includes, as illustrating and noting limit, provides video card and the sound card of attachment means between outut device 1040 and system bus 1018.It should be noted that the system of other equipment and/or equipment provides the input of such as remote computer 1044 etc and exports two kinds of functions.

Computer 1012 can use the logic of one or more remote computers of such as remote computer 1044 etc to connect, and operates in networked environment.Remote computer 1044 can be personal computer, server, router, network PC, work station, electrical equipment based on microprocessor, peer device or other common network node etc., and generally includes above with reference to the many described by computer 1012 or whole element.For simplicity, together with remote computer 1044, merely illustrate memory devices 1046.Remote computer 1044 is logically connected to computer 1012 by network interface 1048, then, connects physically via communication connection 1050.Network interface 1048 comprises the wired or wireless communication network of such as LAN (LAN) and wide area network (WAN) etc.Lan technology includes Fiber Distributed Data Interface (FDDI), copper distributed data interface (CDDI), Ethernet, token-ring network etc..WAN technology includes, but not limited to point-to-point link, Circuit Switching Network, such as ISDN (ISDN) and its variant, packet switching network and Digital Subscriber Line (DSL).

Communication connection 1050 refers to the hardware/software for network interface 1048 is connected to bus 1018.Although communication connection 1050 is shown in computer 1012 inside for illustrative clarity, but, it is outside that it can also be positioned at computer 1012.Hardware/software needed for being connected to network interface 1048 includes, only makees example, inside and outside technology, and e.g., modem, including routine call hierarchical modulation demodulator, cable modem and DSL modem, ISDN adapter and ethernet nic.

The example including the present invention as described above.It is, of course, not possible to describe each possible combination of assembly or method for the purpose describing theme required for protection, but, it will be appreciated by the appropriately skilled person that many further combinations of the present invention and displacement are all possible.Therefore, theme required for protection is intended to all such change, the modifications and variations being included in the spirit and scope of appended claims.

Specifically; for the various functions performed by said modules, equipment, circuit, system etc.; except as otherwise noted; otherwise the term (including quoting " device ") for describing these assemblies is intended to corresponding to the appointment function performing assembly described by the function shown in this illustrative aspects being in theme required for protection described by performing (such as; functionally equivalent) any assembly, even if these assemblies structurally inequivalence is in disclosed structure.About this point, it will also be appreciated that system and the computer-readable medium that present invention includes the computer executable instructions with the action of the various methods for performing theme required for protection and/or event.

There is the multiple mode realizing the present invention, such as, make application program and the service suitable API of advertisement technology, tool kit, driver code, operating system, control, independence or the Downloadable software object that can use the present invention etc..Theme required for protection is also contemplated by from the viewpoint of API (or other software objects), and from the software operated according to advertisement technology according to the present invention or the purposes of hardware objects.So, the various realizations of the present invention described herein can have completely with hardware, partly with hardware partly with software, and the aspect realized with software.

Foregoing system is to utilize describing alternately between multiple assembly.It will be seen that, such system and assembly can include these assemblies or the sub-component wherein specified, some assembly specified or sub-component, and/or additional assembly, and according to the various displacements of aforesaid content and combination.Sub-component can also realize rather than be included in (level) in parent component as the assembly being communicably coupled to other assemblies.Additionally, it should be noted that, one or more assemblies can also be merged in the single component providing polymerizable functional, or multiple single sub-component can also be divided into, and, any one or more intermediate layer of such as management level etc can be provided, to be communicably coupled to such sub-component, in order to provide integrated function.Other assemblies that any assembly described herein can also be known widely with one or more not specifically described herein but those skilled in the art interact.

Although it addition, relative to the only one in some realizations one special characteristic of the present invention can realize disclose, but this feature can be as given or needed for application-specific and advantageously combined with other one or more further features realized any.In addition, with regard in " detailed description of the invention " or claims use term " include ", " having ", " with " or its variant for, these terms be intended to " include " to term similar in the way of as inclusive, as open transition word, without getting rid of any extra or other element.

Claims (15)

1. it is easy to a system for detection input, including:

The line being coupled to produce laser rays from infrared ray (IR) laser spot generates (LG) Optical devices Laser assembly, described laser assembly and line generate (LG) Optical devices and are arranged essentially parallel to a base In basis, even curface sends IR optical plane (104,106), described essentially flat surface relate to Under every at least one: desk, desk, wall, floor, ceiling, the part on ground, A part for concrete, one piece of wood, blackboard, projection screen, windshield, automobile window, automobile Instrument board, relate to the essentially flat surface of automobile, or the pallet on aircraft；

The camera installation (108) of a part for image in the region that seizure is covered by described optical plane；

Camera installation is coupled to IR pass filter, and described IR pass filter stops visible ray and allows IR light passes through, in order to the breach (108,110) in the IR optical plane sent described in detection；And

Determine the figure of position of described breach in the described region covered by the described IR optical plane sent As processing assembly (102), described image processing modules is further configured to be based at least partially on by described photograph Phase equipment capture described in the breach that detects to calculate size or the shape of described breach of described breach In at least one, and by the described position of the described breach in described IR optical plane, described size with And described shape map is to the correspondence position on display, dimension and shape, in order to in described correspondence The position data to presenting on the display carry out data interaction, the described correspondence on described display Position is the coordinate of the mapping of the position representing described breach.

2. the system as claimed in claim 1, wherein, described display is monitor, television set, liquid crystal Display (LCD), plasma, rear projection display, front throwing display, cathode ray tube (CRT) are supervised Visual organ, flat faced display, the display of calculating equipment, portable digital-assistant (PDA) screen, above-knee Escope, computer monitor, smart phone display, honeycomb equipment screen, mobile communication equipment At least one in display, portable game device display.

3. the system as claimed in claim 1, wherein, also includes wherein having coupled at least one LG optics Multiple laser assemblies of device, the plurality of laser assembly is directed to send described in covering the most flat The IR optical plane in one region on whole surface.

4. the system as claimed in claim 1, wherein, described image processing modules is followed the tracks of described in real time The breach detected described in IR optical plane.

5. the system as claimed in claim 1, wherein, the described breach in described IR optical plane be from Absorb the reflection of the object of a part for described IR optical plane, described breach be by touch event, gesture, At least one in motion or object is caused.

6. the system as claimed in claim 1, wherein, also includes infrared ray (IR) modulation component, described Infrared ray (IR) modulation component compensate not by described laser assembly or described LG Optical devices at least A part for one the IR light generated.

7. system as claimed in claim 6, wherein, described IR modulation component is caught at described camera installation Described laser instrument is controlled by the sequence of "Off" order and "ON" order when catching the following:

Oneth IR light set, is not generated by described laser assembly during being included in described "Off" order A part for IR light；And

2nd IR light set, during being included in described "ON" order described IR optical plane and not by described swash A part for the IR light that light device assembly generates.

8. system as claimed in claim 7, wherein, described IR modulation component is by making described 2nd IR Light set and a described IR light set subtract each other to identify the one of the IR light not generated by described laser assembly Part, described laser assembly a part for the described IR light not generated filters from described camera installation Go out.

9. the system as claimed in claim 1, wherein, farther includes to utilize in described IR optical plane The described defined location of described breach carries out the application of at least one in data input or data interaction Program frame.

10. system as claimed in claim 9, wherein, described application framework includes the one of software At least one partly, in the combination of the part of a part for hardware or a part for software and hardware, Wherein, described application framework receives a part for the breach detected related in described IR optical plane Data, in order to carry out data interaction.

11. systems as claimed in claim 10, wherein, described application framework is for touch-sensitive screen Curtain utilizes surface computing technique, wherein, touches detection by capturing described in described IR optical plane Breach is provided.

12. 1 kinds facilitate implementation the computer implemented method mutual with multiple touches of display, bag Include:

It is arranged essentially parallel to a surface and sends the laser plane with at least one in following properties: be described The visible gray scale of user of laser plane；Or described laser plane is to have the described laser pair of permission Generating in the environment of the visible darkness level of user, described surface relates at least one in the following: Desk, desk, wall, floor, ceiling, the part on ground, a part for concrete, one piece Wood, blackboard, projection screen, windshield, automobile window, fascia, relate to the base of automobile Pallet on even curface, or aircraft in basis；

Receive and relate to detecting a part of data of at least one breach in the described laser plane sent (702)；

The position (704) of the described breach in the laser plane sent described in calculating；

The size of the described breach in the laser plane sent described in determining or described in the laser plane that sends In described breach shape at least one (706)；And

By the described position of the described breach in the described laser plane sent, described size and described shape Shape is mapped to the correspondence position on described display (708), dimension and shape, for described right The position data to presenting on the display are answered to carry out data interaction.

13. methods as claimed in claim 12, also include:

Follow the tracks of at least one breach in described laser plane in real time.

14. methods as claimed in claim 13, also include:

Utilize application framework receive described in being mapped of described breach in the laser plane that sends Position, dimension and shape；And

The data input detected based on the described breach in the laser plane sent described in passing through, utilizes institute Stating application framework to interact with data, described application framework uses at least one surface to calculate skill Art.

15. 1 kinds of computer implemented systems, including:

Send for being arranged essentially parallel to a surface IR optical plane device (104,106,202,502, 802), described surface relates at least one in the following: desk, desk, wall, floor, Ceiling, the part on ground, a part for concrete, one piece of wood, blackboard, projection screen, automobile gear Wind glass, automobile window, fascia, relate to the essentially flat surface of automobile, or on aircraft Pallet；

For follow the tracks of in real time at least one breach in described IR optical plane device (108,110, 102、702、804)

Relate at least one breach being detected in described infrared ray (IR) optical plane sent for receiving The device (102,702,806) of a part of data；

For calculate the position of the described breach in the IR optical plane that sends device (102,108,110, 704、806)；

The size of the described breach in the IR optical plane sent described in determining or described IR optical plane In described breach shape in the device of at least one (102,108,110,706,806)；

For by the described position of described breach in the described IR optical plane sent, described size and Described shape map to the correspondence position on display (102,302,708,806), dimension and shape, Data interaction is carried out for the data to presenting on the display in described corresponding position；

For removing the dress of a part for the IR light not generated by the device being used for sending described IR optical plane Put (102,504)；

For the device utilizing a part for the described breach in described IR optical plane and data to interact (102、402、808)；And

For one with the data relating to detecting at least one breach described in described IR optical plane Divide the device (102,402,808) being used in combination surface computing technique.