Búsqueda Imágenes Maps Play YouTube Noticias Gmail Drive Más »
Iniciar sesión
Usuarios de lectores de pantalla: deben hacer clic en este enlace para utilizar el modo de accesibilidad. Este modo tiene las mismas funciones esenciales pero funciona mejor con el lector.

Patentes

  1. Búsqueda avanzada de patentes
Número de publicaciónCN102279670 B
Tipo de publicaciónConcesión
Número de solicitudCN 201110115071
Fecha de publicación14 Dic 2016
Fecha de presentación28 Abr 2011
Número de publicación201110115071.2, CN 102279670 B, CN 102279670B, CN 201110115071, CN-B-102279670, CN102279670 B, CN102279670B, CN201110115071, CN201110115071.2
InventoresD·L·S·吉蒙兹, N·P·奥兹, P·S·塔皮亚, D·E·卡姆皮罗
Solicitante波音公司
Exportar citaBiBTeX, EndNote, RefMan
Enlaces externos:  SIPO, Espacenet
基于手势的人机界面 Gesture-based human-machine interface traducido del chino
CN 102279670 B
Resumen  traducido del chino
本发明涉及基于手势的人机界面,例如用于控制在计算机上执行的程序的图形化用户界面。 The present invention relates to a gesture-based human-machine interface, for example for controlling a computer to execute a program on a graphical user interface. 用户的手势被监测并且提供基于检测到的手势的响应。 User's gesture is monitored and provide a response based on the detected gesture. 物体被用来指向显示在屏幕上的信息。 Object is used to point to information displayed on the screen. 不仅响应于对物体指向何处的确定而且响应于对物体离屏幕的距离的确定,显示在屏幕上的信息被修改。 Not only in response to determining the object and the point where the response to the determination of the distance of the object from the screen, the information displayed on the screen is modified.
Reclamaciones(17)  traducido del chino
1.一种通过基于手势的人机界面使用计算机系统的方法,所述方法包括: 使用用于指向的物体指向在所述计算机系统的屏幕上的位置,其中指向包括维持所述物体和所述屏幕之间的距离,所述距离为正值并且其中所述屏幕包括外角和中心; 用至少四个照相机来捕获所述屏幕前面的空间的图像,所述照相机被布置为连接在所述屏幕的外角处并且指向所述屏幕的中心,其中利用所述至少四个照相机捕获图像允许建立所述屏幕前面的空间的三维图,所述空间包括所述物体; 使用处理器来分析所述图像以识别所述物体,以便确定所述物体指向的所述屏幕上的所述位置,并确定所述物体与所述屏幕之间的所述距离;并且响应于对所述物体指向的所述屏幕上的所述位置和所述物体与所述屏幕之间的所述距离两者的确定,修改显示在所述屏幕上的信息; 其中修改显示在所述屏幕上的所述信息包括以某一放大率使所述物体正在指向的所述屏幕的一部分上显示的所述信息放大,该放大率取决于所述物体与所述屏幕之间的确定的距离,其中所述确定的距离处于最大值和最小值之间,并且其中随着所述确定的距离在所述最大值和所述最小值之间变化,放大率比例按指数规律变化。 1. A computer system by using the gesture-based human-machine interface, the method comprising: using a pointed object for pointing on the screen of the computer system of the position, which includes the maintenance of the object and point to the the distance between the screen and the distance is positive and wherein said screen comprises a central external corner; with at least of the four camera captures an image of the space in front of the screen, the camera is arranged to be connected to the screen outside corners and toward the center of the screen, wherein at least of the four camera using the captured image of the front of the screen allow the establishment of the three-dimensional map space, the space including the object; using a processor to analyze the image to identify the object and for determining the position of the object on the screen pointed to, and determining the distance between the object and the screen; and in response to the object on the screen of the point between the object and the position and the distance between the screen to determine, modify the information displayed on the screen; wherein modifying the information displayed on the screen includes a magnification the information enables the object being pointed to a portion of the screen displayed on the zoom magnification depends on the determination of the distance between the object and the screen, wherein the determined distance is at a maximum and minimum between the values, and wherein said determined as the distance between the maximum value and the minimum value changes, the magnification ratio change exponentially.
2.根据权利要求1所述的方法,其中确定所述物体指向的所述屏幕上的所述位置包括追踪所述物体的移动。 2. A method according to claim 1, wherein determining said position of said object on said screen comprising moving the tracking point to the object.
3.根据权利要求1所述的方法,其中确定所述物体指向的所述屏幕上的所述位置包括确定所述物体朝向所述屏幕的伸展。 3. The method according to claim 1, wherein determining said position of said object on the screen of the object point comprises determining extending toward the screen.
4.根据权利要求1所述的方法,其进一步包括: 使用所述处理器分析所述图像以确定所述物体朝向或远离所述屏幕移动的速度;和响应于对所述物体朝向或远离所述屏幕移动的速度的确定,修改显示在所述屏幕上的所述信息。 4. The method according to claim 1, further comprising: analyzing the image using the processor to determine whether the object toward or away from the moving speed of the screen; and in response to the object toward or away from the the speed of movement of said screen to determine, modify the information displayed on the screen.
5.根据权利要求1所述的方法,其进一步包括: 使用所述处理器分析所述图像以确定所述物体的移动速度;以及根据所述物体的不同移动速度不同地修改显示在所述屏幕上的所述信息。 5. The method according to claim 1, further comprising: using the processor analyzes the images to determine the speed of movement of the object; variously modified and displayed on the screen depending on the speed of the moving object the information on.
6.根据权利要求5所述的方法,其进一步包括: 使用所述处理器分析所述图像以确定所述物体在所述屏幕的前面的横向移动速度;以及响应于对所述物体的所述移动速度的确定修改显示在所述屏幕上的所述信息。 6. The method according to claim 5, further comprising: using the processor analyzes the image to determine the lateral movement speed of the object in front of said screen; and in response to the object of the moving speed is determined to modify the information displayed on the screen.
7.根据权利要求1所述的方法,进一步包括: 使用用于指向的两个物体来指向在所述计算机系统的所述屏幕上的位置; 使用所述处理器来分析所述图像以识别所述两个物体,从而确定所述两个物体指向的所述屏幕上的所述位置,并确定所述两个物体离所述屏幕的距离;并响应于对所述两个物体指向的所述屏幕上的所述位置和所述两个物体离所述屏幕的所述距离二者的确定来修改显示在所述屏幕上的所述信息。 7. The method according to claim 1, further comprising: using the two objects is used to point to point on the screen of the computer system's position; using the processor to analyze the image to identify the It said two objects, thereby determining the position on the screen of the two objects point and determine the distance between the two objects from the screen; and in response to the points of the two objects the position on the screen and the two objects from the screen of the distance between the two is determined to change the display on the screen of the information.
8.根据权利要求1所述的方法,其中所述物体是用户的手的伸展的手指。 8. The method according to claim 1, wherein said object is a user's hand extending fingers.
9.一种包括基于手势的人机界面的设备,所述设备包括: 可操作以显示信息的屏幕,其中所述屏幕包括外角和中心; 至少四个照相机,所述至少四个照相机被布置成连接在所述屏幕的外角处并且指向所述屏幕的中心以捕获所述屏幕前面的空间的图像,其中利用所述至少四个照相机捕获图像允许建立所述屏幕前面的空间的三维图;以及处理器,其被布置成接收所述图像, 分析所述图像以便识别指向所述屏幕上的一个位置处的物体,以确定相对于所述物体和所述至少四个照相机的屏幕位置,并且确定所述物体离所述屏幕的距离,所述屏幕是用于确定所述物体指向所述屏幕上的所述位置的参考平面,所述距离为正值,并且响应于对所述物体指向的所述屏幕上的所述位置和所述物体离所述屏幕的所述距离二者的确定来修改显示在所述屏幕上的所述信息; 其中修改显示在所述屏幕上的所述信息包括以某一放大率使所述物体正在指向的所述屏幕的一部分上显示的所述信息放大,该放大率取决于所述物体与所述屏幕之间的确定的距离,其中所述确定的距离处于最大值和最小值之间,并且其中随着所述确定的距离在所述最大值和所述最小值之间变化,放大率比例按指数规律变化。 A gesture-based human-machine interface comprising a device, the apparatus comprising: a screen operable to display information, and wherein the outer corner of the screen comprises a center; at least of the four camera, the camera is arranged at least four connecting the outer corners of the screen toward the center of the screen and to capture an image of the space in front of the screen, wherein at least of the four camera using the captured image of the front of the screen allow the establishment of the three-dimensional map space; and processing device, which is arranged to receive the image, analyzing the image to identify the location of a pointing object on the screen, the screen in order to determine the position of the object relative to the camera and said at least four, and determine whether the said object distance from the screen, the screen is used to determine the position of the object plane reference point on the screen, the distance is positive, in response to the pointing of the object and the position of the object on the screen from the screen, the distance between the two is determined to modify the information displayed on the screen; wherein modifying the information displayed on the screen includes a the information on a magnification of said object being pointed to a portion of the screen shows an enlarged, the magnification is determined depending on the distance between the object and the screen, wherein the determined distance is at a maximum between the value and the minimum value, and wherein the determined distance as between the maximum value and the minimum value changes, the magnification ratio change exponentially.
10.根据权利要求9所述的设备,其中所述处理器被布置成分析所述图像以追踪所述物体的移动从而确定所述物体指向的所述屏幕上的所述位置。 10. The apparatus according to claim 9, wherein said processor is arranged to analyze the image to the moving object tracking to determine the position of the object on the screen pointed.
11.根据权利要求9所述的设备,其中所述处理器被布置成分析所述图像以确定所述物体朝向所述屏幕的伸展从而确定所述物体指向的所述屏幕上的所述位置。 11. The apparatus according to claim 9, wherein said processor is arranged to analyze the image to determine the object extending towards the screen so as to determine the position of the object on the screen pointed.
12.根据权利要求9所述的设备,其中所述处理器还被布置成确定所述物体的移动速度并响应于对所述移动速度的确定来修改显示在所述屏幕上的所述信息。 12. The apparatus according to claim 9, wherein the processor is further arranged to determine a moving speed of said object and in response to determining that the moving velocity of modifying the information displayed on the screen.
13.根据权利要求9所述的设备,其中所述物体是用户的手的伸展的手指。 13. The apparatus according to claim 9, wherein said object is a user's hand extending fingers.
14.一种计算机系统的基于手势的人机界面,所述界面存储在所述计算机系统中的存储介质上,并且进一步包括: 被配置为使屏幕能够运行以显示信息的第一模块,其中所述屏幕包括外角和中心;被配置为向至少四个照相机发送指令的第二模块,所述至少四个照相机被布置成连接在所述屏幕的外角处并且指向所述屏幕的中心以捕获所述屏幕前面的空间的图像;以及被配置为控制处理器的第三模块,所述处理器被布置成接收所述图像, 分析所述图像以便识别指向在所述屏幕上的一个位置处的物体,以确定相对于所述物体和所述至少四个照相机的屏幕位置,确定所述物体离所述屏幕的距离,并确定所述物体的移动速度,所述屏幕是用于确定所述物体指向所述屏幕上的所述位置的参考平面,所述距离是正值,并且响应于对所述物体指向的所述屏幕上的所述位置、所述物体离所述屏幕的所述距离以及所述移动速度的确定来修改显示在所述屏幕上的所述信息; 其中修改显示在所述屏幕上的所述信息包括以某一放大率使所述物体正在指向的所述屏幕的一部分上显示的所述信息放大,该放大率取决于所述物体与所述屏幕之间的确定的距离,其中所述确定的距离处于最大值和最小值之间,并且其中随着所述确定的距离在所述最大值和所述最小值之间变化,放大率比例按指数规律变化。 Gesture-based human-machine interface 14. A computer system, said interface is stored on the storage medium in the computer system, and further comprising: a screen configured to enable the module to operate at the first display information, wherein said screen including the outer corner and center; configured to transmit instructions to at least a second of the four camera modules, at least of the four camera is arranged to be connected at the outside corners of the screen and the center point of the screen to capture the and a processor configured to control a third module, said processor being arranged to receive the image, analyzing the image to identify the point on the screen at a position of an object; the image the space in front of the screen with respect to the object and determining the position of at least of the four camera screen, determining the distance of the object from the screen, and determines the moving speed of the object, the screen is used to determine the point of the object the position of the reference plane on said screen, said distance is positive, in response to the position of the object on the screen of the point, the distance from the object and the screen of the moving speed is determined to modify the information displayed on the screen; wherein modifying the information displayed on the screen includes a magnification of a portion of the screen of the object being pointed to on the display amplifying said information, the magnification is determined depending on the distance between the object and the screen, wherein said maximum and minimum values determined at a distance, and wherein said determined distance with the varies between the minimum and said maximum magnification ratio by an exponential dependence.
15.根据权利要求14所述的界面,其中所述物体包括用于确定用户指向所述屏幕上的何处的手指伸展。 15. The interface of claim 14, wherein the object includes means for determining where a user's finger pointing on the screen stretch.
16.根据权利要求14所述的界面,其中所述处理器被布置成分析所述图像以确定所述物体朝向所述屏幕的伸展从而确定所述物体指向的所述屏幕上的所述位置。 16. The interface of claim 14, wherein said processor is arranged to analyze the image to determine the object extending towards the screen so as to determine the position of the object on the screen pointed.
17.根据权利要求14所述的界面,其中所述处理器被布置成分析所述图像以追踪所述物体的移动从而确定所述物体指向的所述屏幕上的所述位置。 17. The interface of claim 14, wherein said processor is arranged to analyze the image to the moving object tracking to determine the position of the object on the screen pointed.
Descripción  traducido del chino
基于手势的人机界面 Gesture-based human-machine interface

技术领域 TECHNICAL FIELD

[0001]本发明涉及基于手势的人机界面,例如可用于控制在计算机上执行的程序的图形化用户界面。 [0001] The present invention relates to a gesture-based human-machine interface, for example, can be used to control the execution of a computer program on a graphical user interface. 尽管适合于很多类型的程序,不过特别感兴趣于控制一个或更多个无人飞行器的飞行的程序。 Although suitable for many types of programs, but of particular interest for controlling one or more unmanned aircraft flight procedures.

背景技术 Background technique

[0002]人机界面已经在过去的数十年发生了很大的改变。 [0002] HMI has been in the past decades, great changes occurred. 即使在计算机控制的较窄领域中,界面已经从命令行演变成要求使用鼠标或类似指点装置以便选择显示给用户的图标的图形化用户界面。 Even in the narrow field of computer control, the command line interface has evolved into a requirement to use a mouse or similar pointing device to select an icon displayed to the user graphical user interface.

[0003]最近以来,触摸屏装置已经开始流行。 [0003] Recently, a touch screen device has become popular. 当触摸屏装置开启基于手势控制的可能性时,允许多点输入的触摸屏装置特别有利。 When the touch screen device is on the possibility of gesture-based control that allows multi-touch screen input device is particularly advantageous. 苹果的iPhone(TM)是触摸可被用来选择项目、向上或向下滚屏、放大或缩小并旋转项目的较好示例。 Apple iPhone (TM) is a touch can be used to select an item, up or down to scroll, zoom in or out and rotate the better examples of the project. 例如,屏幕倾向于具有较慢的反应时间、较差的精确性和较差的可靠性,而频繁使用触摸屏导致残余物和灰尘的积累,残余物和灰尘导致进一步的性能退化。 For example, the screen tends to have a slower reaction time, poor accuracy and poor reliability, frequent use of the touch screen and cause the accumulation of dust residues, residues and dust lead to further degradation.

[0004] 通过避免接触屏幕,避免触摸屏装置的一些问题的系统已经被提出。 [0004] By avoiding touching the screen, the system avoids some of the problems of the touch screen device has been proposed. 代之,用户的手势被监测,并且提供基于检测到的手势的响应。 Instead, the user's gesture is monitored, and provide a response based on the detected gestures. 例如,监测用户的手的系统已经被提出,使得用手产生的手势被用来选择、滚动、变焦、旋转等,类似于依赖于触摸屏幕的现有系统。 For example, a system to monitor a user's hand have been proposed, so that the hand gesture is used for generating the selection, scrolling, zooming, rotation, etc., similar to the existing systems rely on the touch screen.

发明内容 SUMMARY

[0005]在此背景下,本发明属于通过基于手势的人机界面使用计算机系统的方法。 [0005] In this context, the present invention pertains to the use of a computer system via the HMI gesture-based approach. 该方法包括使用物体指向显示在计算机系统的屏幕上的信息,并用至少两个照相机捕捉屏幕前的景象。 The method involves the use of an object, point to display information on the screen of a computer system, with at least two cameras to capture the scene in front of the screen. 处理器被用来分析用照相机捕捉的景象以识别物体,以确定物体指向在屏幕上的何处和物体离屏幕的距离。 The processor is used to analyze the scene captured by the camera to recognize objects, to determine the distance to the object point on the screen where and objects from the screen. 之后,处理器响应于物体指向何处和物体离屏幕的距离的确定来修改显示在屏幕上的信息。 After that, the processor in response to a point where the object and the object distance from the screen determined to modify the information displayed on the screen.

[0006]以此方式,可以避免触摸屏的缺点。 [0006] In this manner, the touch screen can be avoided drawbacks. 此外,可以利用关于物体离屏幕多远的信息。 In addition, you can use information about how far away from the screen object. 这个信息可以以不同的方式被使用。 This information can be used in different ways. 例如,可通过放大物体指向的屏幕部分来改变显示在屏幕上的信息,放大率取决于从屏幕到物体所确定的距离。 For example, you can change the information displayed on the screen through the screen partially enlarged object points, depending on the magnification of the object from the screen to the determined distance. 因此,靠近屏幕的指点可以被用来提供比在更远处的指点更大的放大率。 Therefore, pointing near the screen can be used to provide greater magnification than in more distant guidance. 可以设定限值,使得比某距离更远的指点产生一致的放大率,而在距屏幕设定距离处放大率到达最大值。 You can set limits, so that a distance farther than pointing to produce consistent magnification, and in the distance from the screen is set to reach the maximum magnification. 这些距离之间的放大率如何变化可以被控制,如,它可以线性地或按指数规律地变化。 How to change the magnification between the distance can be controlled, e.g., it can change linearly or exponentially.

[0007]该方法可以包括追踪物体的移动以确定物体指向屏幕上的何处。 [0007] The method may include tracking of moving objects to determine the point where the object on the screen. 该方法可以包括确定物体的纵向伸展以确定物体指向屏幕上的何处。 The method may include determining the longitudinal extension of the object to determine where on the screen the object point. 这两个可选的特征可以被用作可替换方案或者它们可以被用来相互增强。 These two optional features can be used as an alternative to or they may be used to reinforce each other.

[0008]通过确定在一段时间内物体离屏幕的距离,物体移动的速度可以被确定。 [0008] By determining the distance of objects from the screen for some time, the speed of moving objects can be determined. 这个速度可以被用作对显示在屏幕上的信息的进一步的控制。 This speed can be used as a further control displayed on the screen of information. 物体向屏幕的快速移动可以被解释得与物体向屏幕的逐渐移动不同。 Fast-moving objects to the screen can be interpreted to give the object gradually move the screen are different. 例如,逐渐移动可以被解释为单击,而快速移动可以被解释为双击。 For example, gradual movement may be interpreted as a click, and the rapid movement may be interpreted as a double-click.

[0009]可选地,该方法可以包括用两个物体指向显示在计算机系统的屏幕上的信息,并用至少两个照相机捕捉屏幕前面的景象。 [0009] Alternatively, the method may include two objects point to the information displayed on the screen of a computer system, with at least two cameras to capture the scene in front of the screen. 处理器可以被用来分析照相机所捕捉的景象以识别物体,从而确定物体指向屏幕上的何处和物体离屏幕的距离。 The processor may be used to analyze the camera captured the scene to identify the object to determine the point where the object and the object on the screen distance from the screen. 之后,处理器响应于物体指向屏幕上何处和物体离屏幕的距离的确定来修改显示在屏幕上的信息。 After that, the processor in response to a point on the screen where the object and the object distance from the screen determined to modify the information displayed on the screen. 这允许进一步的功能。 This allows for a further function. 例如,该对物体可以被用来独立地与屏幕上的不同控制交互作用,以调节音量控制并在某一区域上放大。 For example, the object can be used to independently control the interaction with the different on the screen to adjust the volume control, and zoom in on a particular area. 这两个物体还可以一起被使用。 These two objects can also be used together. 可以用物体操作显示在屏幕上的图像,例如通过旋转物体。 You can operate the object image is displayed on the screen, for example, by rotating the object. 例如,使左边的物体向屏幕移动并使右边的物体远离屏幕移动可以引起图像绕竖直轴线顺时针方向旋转,并且使上面的物体向屏幕移动并使下面的物体远离屏幕移动可以引起图像绕水平轴线旋转,根据物体的相对对齐和物体之间的相对移动也可以存在其它旋转。 For example, so that the left and right side of the screen to move the object to the object can be caused to move away from the screen image in a clockwise direction about a vertical axis, and the objects on top of the screen to move the object away from the screen and below the image can be caused to move around a horizontal axis, the relative alignment of the relative movement between the object and the object may also be present other rotary.

[0010]很多不同的物体可以被用来指在屏幕上。 [0010] Many different objects can be used to refer to on the screen. 例如,物体可以是用户的手。 For example, the object may be a user's hand. 优选地,物体可以是用户的手的伸展的手指。 Preferably, the object may be a user's hand extending fingers. 之后,手指的指尖可以是被用来确定离屏幕的距离的点。 Thereafter, the tip of the finger can be used to determine the distance from the screen. 手指伸展(extens1n)可以被用来确定用户指向屏幕上的何处。 Finger stretch (extens1n) can be used to determine where the user points on the screen.

[0011]本发明还在于包括基于手势的人机界面的计算机系统。 [0011] The present invention also includes a gesture-based human-machine interface of the computer system. 该计算机系统包括:(a)可被操作来显示信息的屏幕,(b)被布置成捕捉屏幕前面的景象的至少两个照相机,和(C)处理器。 The computer system comprising: (a) may be operable to display information, (b) is arranged in front of the screen to capture the scene of at least two cameras, and (C) a processor. 处理器被布置成接收照相机提供的图像并分析图像,以识别指向显示在屏幕上的信息的物体。 The processor is arranged to receive the image provided by the camera and image analysis, to identify the point information of the object displayed on the screen. 处理器还被布置成确定物体指向屏幕上的何处和物体离屏幕的距离。 The processor is further arranged to determine the point where the object and the object on the screen distance from the screen. 处理器还被布置成响应于物体指向何处和物体离屏幕的距离的确定来修改显示在屏幕上的信息。 The processor is further arranged to respond to the point where the object and the object distance from the screen determined to modify the information displayed on the screen.

[0012]可选择地,处理器被布置成追踪物体的移动以确定物体指向屏幕上的何处。 [0012] Alternatively, the processor is arranged to move the object tracking to determine where the object point on the screen. 此外或可替换地,处理器可以被布置成确定物体的纵向伸展,以确定物体指向屏幕上的何处。 Additionally or alternatively, the processor may be arranged to determine the longitudinal extension of the object, to determine where on the screen an object point. 处理器可以被布置成放大物体指向的屏幕部分来改变显示在屏幕上的信息,放大率取决于从屏幕到物体所确定的距离。 The processor may be arranged to magnify objects pointing screen section to change the information displayed on the screen, depending on the magnification of the object from the screen to the determined distance. 如以上关于发明的方法所述的,两个物体可以被用来修改显示在屏幕上的信息。 As described above with respect to the method of the invention, the two objects can be used to modify the information displayed on the screen. 物体可以是用户的手,例如用户的手的伸展的手指。 Object may be the user's hand, such as extending the user's hand fingers.

附图说明 BRIEF DESCRIPTION

[0013]为了使发明更容易被理解,仅通过示例参考附图,其中: [0013] To make the invention easier to understand, by way of example only with reference to the accompanying drawings, wherein:

[0014]图1是简化视图的透视图,其示出了根据本发明的实施例的包括人机界面的系统,该系统包括两个并排的屏幕和四个照相机; [0014] FIG. 1 is a simplified perspective view, which shows a man-machine interface system including an embodiment of the present invention, the system includes two screens side by side and four cameras;

[0015]图2是从用户视角观察的屏幕的透视图,其示出了用户通过指按钮来选择显示在屏幕上的按钮; [0015] FIG. 2 is a perspective view of the screen from the user's perspective observation, which shows the user by means button to select the button displayed on the screen;

[0016]图3a到图3d是系统的示意俯视图,其示出了根据本发明的人机界面的实施例,该系统包括屏幕和一个或更多个照相机,示出了照相机的视场如何结合; [0016] FIG. 3a to FIG. 3d is a system schematic plan view illustrating an embodiment of the display unit according to the present invention, the system includes a screen and one or more cameras, shows a field of view of the camera how to combine ;

[0017] 图3e到图3h是图3a到图3d中示出的系统的示意正视图,图3e、图3f、图3g和图3h分别对应于图3a、图3b、图3c和图3d; [0017] Figure 3e to Figure 3h is FIG. 3a schematically in FIG. 3d shows the system front view, Figure 3e, Figure 3f, Figure 3g and 3h, respectively, corresponding to Fig. 3a, to Figure 3b, 3c and 3d;

[0018]图4是示出根据本发明的人机界面的实施例的系统的示意图;以及 [0018] FIG. 4 is a schematic diagram illustrating an embodiment of the system according to the man-machine interface of the present invention; and

[0019]图5a到图5c是屏幕的简化正视图,其示出了根据本发明的由人机界面的实施例提供的变焦设备。 [0019] FIG. 5a to 5c is a simplified front view of the screen, which shows a zoom device provided by the embodiment of the man-machine interface of the present invention.

具体实施方式 detailed description

[0020]图中示出了包括基于手势的人机界面的计算机系统10。 [0020] is shown including gesture-based human-machine interface of the computer system 10. 计算机系统10包括被驱动来显示信息的一个或更多个屏幕12。 The computer system 10 includes a driver to display a message screen 12 or more. 信息的显示可以由用户通过用他的或她的手14在屏幕12前面做手势来控制。 Display of information can use his or her hand 14 in front of the screen 12 is controlled by the user through gestures. 这些手势用被布置在屏幕12周围的四个照相机16记录。 These gestures are arranged with four camera screen 12 around 16 records. 分析照相机16捕获的图像以确定用户的手14的三维位置并追踪手14的移动。 Analysis of 16 cameras captured images to determine the three-dimensional position of the user's hand 14 and 14 of the track hand movement. 手14的移动通过计算机系统10解释,例如以识别对应于显示在屏幕12上的图标选择或放大显示在屏幕12上的区域。 Moving the hand 14 of the computer system 10 via interpretation, e.g., to identify the screen corresponding to the display 12 or on the icon selection screen 12 displayed on the enlarged area. 计算机系统10响应于这些手势来改变显示在屏幕12上的信息。 The computer system 10 in response to these gestures to change the information displayed on the screen 12. 图2示出了用户将食指18向前朝显示在屏幕12上的按钮20移动的示例。 Figure 2 shows an example of a user finger 18 forward toward the display screen 12 on the button 20 moves. 这个移动模仿用户按压按钮20,并且计算机系统10将此解释为用户选择按钮20。 This movement to imitate the user presses the button 20, 10 and the computer system interprets this selection button 20 for the user. 这可以引起计算机系统10在屏幕12上显示新的图像。 This can cause the computer system 10 to display a new image on the screen 12.

[0021]尽管可以使用任意数量的屏幕12,不过图1和图2示出了使用两个并排的屏幕12的计算机系统10。 [0021] Although you can use any number of screen 12, but is shown in Figures 1 and 2 out of the use of two screens side by side computer system 10. 用户的手臂22被示意性地示出在屏幕12的前方。 User arm 22 is schematically shown in front of the screen 12. 手臂22的移动由被布置在屏幕12的四个外角处并朝向屏幕12的中心的四个照相机16捕获。 22 mobile arm 16 is arranged on the screen is captured by the four outside corners 12 and 12 toward the center of the screen's four cameras. 因此,当用户的手14在屏幕12的前面移动时,照相机捕捉用户的手14的移动。 Thus, when a user's hand 14 is moved in front of the screen 12, the camera captures a user's hand 14 of the mobile. 使用四个照相机16能够建立屏幕12前面的空间的三维图。 16 uses four cameras to establish three-dimensional map screen 12 in front of the space. 因此,在x、y、z坐标系统中,物体的位置(例如用户的手指18的指尖)可以被确定。 Thus, the x, y, z coordinate system, the position of an object (e.g., a user's fingertip 18) can be determined. 在图1和图3中指示了这些坐标轴。 It indicates these axes in Figure 1 and Figure 3. 根据所有三个x、y、z轴的空间信息可以被用在人机界面中。 According to all three x, y, z-axis spatial information can be used in the display.

[0022]图3a到图3h示出了每个照相机16的视场24如何结合以提供空间体积,在其中计算机系统1能够确定物体的位置。 [0022] FIG. 3a to FIG. 3h shows a field of view of each camera position 16 to 24 integrate to provide a volume of space in which a computer system is able to determine the object. 照相机16是相同的,并且也具有相同的视场24 O图3a和图3e分别是单个屏幕12的平面图和正视图,其仅示出了单个照相机16(为了清楚,照相机16被示意性地示为圆点)。 The camera 16 is identical and has the same field of view 24 O 3a and 3e are plan views of a single screen 12 and a front view, which shows only a single camera 16 (for clarity, the camera 16 is schematically illustrated as dots). 因此,它们较好地示出了从每个照相机16获得的视场24。 Thus, they are better illustrates the field of view obtained from each camera 1624. 图3b和图3f分别是同一屏幕12的平面图和正视图,这次示出了被布置在屏幕12的右手边缘上的两个照相机16。 Figures 3b and 3f are a plan view and a front view of the same screen 12, this shows two are arranged on the right-hand edge of the screen 12 of the camera 16. 这个图示出了两个照相机16的视场如何结合。 This figure shows two camera field of view of 16 how to combine. 图3c和图3g是屏幕12的平面图和正视图,示出了所有四个照相机16和它们的视场24。 3c and 3g is a plan view and a front screen of FIG. 12, shows all four cameras 16 and 24 of their field of view. 如果物体被捕捉在至少两个照相机16的视场24以内,则物体在屏幕12前面的位置可以被确定。 If the object is captured within the field of view of at least two cameras 16 to 24, the object 12 can be determined at a position in front of the screen. 因此,无论在图3c和图3g内的何处存在视场24的重叠,物体的位置可以被确定。 Thus, no matter where in Figure 3c and 3g of overlapping fields of view 24, the position of the object can be determined. 有用的核心区26被示出在图3d和图3h中,在该核心区中可以确定物体的位置。 Useful core area 26 is shown in Fig. 3d and 3h, it is possible to determine the position of an object in the core area.

[0023]图4更详细地示出了计算机系统10。 [0023] Figure 4 illustrates in greater detail the computer system 10. 计算机系统10具有作为其集线器(hub)的计算机40。 The computer system 10 has a computer 40 as its hub (hub) of. 计算机40可以包括很多不同的部分,例如主处理器42、其中包括储存在其内的程序的存储器,例如用于类似于屏幕12的外围设备的驱动器和用于操作类似于屏幕12的外围设备的卡。 Computer 40 may comprise many different parts, such as the main processor 42, which includes a program memory stores therein, for example, similar to the screen driver of the peripheral device 12 and a screen similar to the operation of the peripheral device 12 card.

[0024] 如所见的,输入端(feed)44将四个照相机16连接到图像处理器46。 [0024] As can be seen, the input (feed) 44 four camera 16 is connected to the image processor 46. 图像处理器46可以是主处理器42的一部分,或者图像处理器46可以被提供为单独的处理器。 The image processor 46 may be part of the main processor 42 or the image processor 46 may be provided as a separate processor. 无论是两者中的哪一种形式,图像处理器46都接收来自照相机16的图像。 No matter which of the two is one form, the image processor 46 receives the image from the camera 16. 图像处理器46使用通常可用的软件来处理图像以改善它们的质量。 The image processor 46 using commonly available software to process the image in order to improve their quality. 例如,可以改善亮度、对比度和清晰度以使得产生更高质量的图像。 For example, it can improve the brightness, contrast and sharpness so that produce higher quality images. 被处理的图像被传到主处理器42。 A processed image is transmitted to the main processor 42. 储存在存储器中的图像分析软件由主处理器43检索到并运行,以分析处理的图像,从而确定用户指向屏幕12上的何处。 Stored in the memory image analysis software by the host processor 43 to retrieve and run to analyze the image processing to determine the screen where the user points on 12. 应当知道,这种图像分析软件是常用的。 It should be understood that the image analysis software is used.

[0025] 一旦主处理器42已经确定用户指向屏幕上的何处,主处理器42确定屏幕12上呈现出的图像是否需要改变。 [0025] Once the main processor 42 determines that the user has to point where on the screen, showing the main processor 42 determines whether the image on the screen 12 needs to be changed. 如果确定需要,则主处理器42产生必要的信号以引起显示在屏幕12上的信息的必要改变。 If it determines that, the main processor 42 to generate the necessary signals to cause the necessary change information displayed on the screen 12. 这些信号被传到屏幕驱动器/卡48,屏幕驱动器/卡48提供被供应到屏幕12的当前信号。 These signals are transmitted to the screen drive / 48 card, the screen drive / card 48 provides the current signal is supplied to the screen 12.

[0026]如图4所示,计算机40可包括用于接收来自屏幕12的触摸屏输入的输入装置50,即,意味着允许用户通过触摸屏幕12来选择显示在屏幕12上的图标。 [0026] FIG. 4, the computer 40 may include means for receiving input from the touch-screen input screen 12 of 50, that means 12 allows the user to select the screen by touching an icon displayed on the screen 12. 提供这个特征在某些情况下可能是有用的。 This feature is provided in some cases may be useful. 例如,关键的选择可能要求用户触摸屏幕12作为进一步的步骤,以确保用户确定他们想做出那个选择。 For example, the critical choices may require the user to touch the screen 12 as a further step to ensure that the user determines that they want to make that choice. 例如,这可以被用于引起系统紧急关闭的按钮:此动作明确是极端情况并且要求用户触摸屏幕12可以反映这一点。 For example, this can cause the system to be used for emergency shut-off button: This action is clearly an extreme case and require the user to touch the screen 12 may reflect this. 因此提供输入装置50。 Thereby providing the input device 50.

[0027]如以上提及的,主处理器42得到由图像处理器46提供的仍被处理的图像,并分析这些图像以确定用户是否指向屏幕12。 [0027] As mentioned above, the host processor 42 to obtain an image provided by the image processor 46 is still processing and analyzing the images to determine whether the user points to the screen 12. 这可以用常规的图像识别技术(例如使用被用来识别与具有伸向一个屏幕12的食指18的手14相关的形状的软件)来进行。 This can be a conventional image recognition technology (for example, be used to identify and use a screen having stretched hand forefinger 12 18 14 related to the shape of the software) to carry out. 之后主处理器42确定手指18指向屏幕12上的何处。 After the main processor 42 determines where 12 of the 18 finger pointing screen. 主处理器42可以针对一只手发挥此功能或针对被认为适当的多只手来发挥此功能。 Main processor 42 may play a hand for this feature or is considered appropriate for the more hands to play this feature. 例如,主处理器42可以针对指在屏幕上的所有手来对其进行确定。 For example, the host processor 42 can refer to on the screen for all hands to be determined. 后面的说明针对单个手指18的示例,如将被逐步理解的,对于被期望或者被确定指在屏幕12上的多个手指18,该方法可以被重复。 For example the description which follows of the single finger 18, as will be appreciated gradually, or is expected to be determined for a plurality of finger means 12 on the screen 18, the method may be repeated.

[0028] 主处理器42如何确定手指18指在屏幕12上的何处可以以不同的方式进行。 [0028] the main processor 42 refers to how to determine where the finger 18 on the screen 12 can be in a different manner.

[0029]在一个实施例中,主处理器42识别食指18的指尖在x、y、z坐标系统内的位置。 [0029] In one embodiment, the main processor 42 identifies the position of index finger 18 in the x, y, z coordinate system. 这可以通过对被四个照相机16捕获的图像进行三角测量来进行。 This can be through the four images captured by the camera 16 to perform triangulation. 在根据一组四个图像识别了食指18的指尖的位置后,下一组四个图像可以以相同的方式被处理,以便确定食指18的指尖的下一个位置。 In the recognition of a set of four image index finger position 18, the next set of four images can be processed in the same manner, in order to determine the next position of the index finger 18. 以此方式,食指18的指尖可以被追踪,并且如果它的运动继续,那么它的移动随时间重复向前以确定它将碰屏幕12的位置。 In this way, index finger 18 can be traced, and if it continues to exercise, then it's time to move forward to determine if it will repeat the touch screen position 12.

[°03°]在可替换的实施例中,图像被分析以确定食指18的伸展(extens1n)和手指18指向的方向。 [° 03 °] In an alternative embodiment, the image is analyzed to determine the index finger extended (extens1n) 18 and the finger 18 is pointing. 当然,这个技术可以与例如以上描述的实施例相结合,以识别何时手指18沿着它指向的方向移动,因为这可以被解释为手指18“按压”显示在屏幕12上的物体。 Of course, this technique can be used with the example embodiments described above combined to identify when the finger 18 moves along the direction it points, as this may be interpreted as a finger 18 "pressed" is displayed on the screen 12 of the object.

[0031]图5a到图5c示出了根据本发明所提供的变焦设备的实施例。 [0031] 5a to 5c show an embodiment of apparatus according to the present invention is to provide a zoom of. 提供侧边有四个照相机16的单个屏幕12,每个都已经被描述过。 There are provided side edges 16 of the four camera single screen 12, each of which has been described. 照相机16和屏幕12被连接到计算机系统10,如之前描述的,计算机系统10操作以提供基于手势的人机界面。 10 Operating the camera 16 and the screen 12 is connected to the computer system 10, as previously described, the computer system to provide gesture-based human-machine interface.

[0032]在图5a到图5c中所示的示例中,屏幕12显示了图80和相关信息。 [0032] In the example shown in FIG. 5a to FIG. 5c, FIG. 12 shows a screen 80 and related information. 屏幕12的顶部具有标题信息82,一列四个可选按钮84设在屏幕12的左手边缘。 Top of the screen 12 having header information 82, a four selectable buttons 84 provided in the left-hand edge of the screen 12. 按钮84可以带有文本86以表示可以选择的新的信息屏或改变显示在图80上的信息。 86 buttons 84 may be provided with a text screen to show new or changed information that can be selected to display information on the FIG. 80. 图80占据了屏幕12的大部分并且被布置为偏向屏幕12的右下方。 Figure 80 occupies most of the screen 12 and arranged to bias the bottom right of the screen 12. 图80将飞机88示为具有说明其当前飞行方向的箭头的圆点。 Figure 80 shows the dots 88 aircraft have illustrate its current flight direction of the arrow. 识别飞机88的信息还可以被显示在圆点旁边,如90处所示。 88 aircraft identification information may also be displayed, as shown in the next 90 dots. 进一步的信息被提供在沿图80的底部边缘的一行框92中。 Further information is provided in the bottom edge 80 along a line 92 in the block.

[0033]用户可能想放大例如图80上的感兴趣的飞机88,以例如更详细地示出显示在图80上的地理信息。 [0033] for example, a user might want to zoom in on the interest 80 Figure 88 aircraft, for example, shown in more detail on the display geographic information in FIG. 80. 为了这么做,用户可以指向一个按钮84以选择缩放模式,并且之后可以指向图80上的感兴趣的飞机88。 To do this, the user can point to a button 84 to select the zoom mode, and then you can point interested in Figure 80 on the plane 88. 如图5b和图5c所示,这引起用户指向的区域以更大的放大率被显示在圆94中。 As shown in Figure 5b and 5c, which causes the area indicated by the user with greater magnification is displayed on the circle 94. 圆94被显示为覆盖在背景图80上。 Round 94 is shown as FIG. 80 overlaid on the background. 如本领域中已知的,变焦的圆94的边缘和背景图80在需要时可以合并。 As it is known in the art, zooming round edges and the background image 94 80 can be combined as needed. 为了调节放大率因子,用户仅使他或她的食指18向着或远离屏幕12(即在z方向上)移动。 In order to adjust the magnification factor, only the user of his or her finger 18 toward or away from the screen 12 (i.e. in the z-direction) to move. 将食指18移向屏幕导致更大的放大率。 The index finger toward the screen 18 resulting in greater magnification.

[0034]因此,用户的手指18的x、y位置被用来确定图80上被放大的区域,手指18的z位置被用来确定放大率。 [0034] Thus, the user's finger x 18, y 80 is used to determine the position of the magnified area map, z position of the finger 18 is used to determine the magnification. Z位置的上限值和下限值可以被设置成对应于上限放大率因素和下限放大率因素。 Upper limit and lower limit Z position may be set to correspond to the upper and lower magnification factor magnification factor. 例如,放大率可以设置成I,而用户的指尖18至少是离屏幕12的某距离(如,30厘米)。 For example, the magnification can be set to I, and the user's finger 18 is at least a certain distance away from the screen 12 (e.g., 30 cm). 而且,离屏幕的最小间隔(如,5厘米)可以被设置为最大放大率,使得如果用户的手指18比5厘米更近地靠近屏幕12,放大率不再增加。 Moreover, the minimum interval from the screen (e.g., 5 cm) may be set to the maximum magnification, so that if the user's finger 18 closer than 5 cm closer to the screen 12, not increase the magnification. 可以根据需要选择这些距离之间的放大率如何变化。 You can select how the distances between the magnification change. 例如,放大率可以随距离线性地变化或者它可以遵循一些其它的关系,例如指数关系。 For example, the magnification can be varied linearly with distance or it may follow some other relations, such as exponential.

[0035]图5b和图5c反映了如下情况,即用户从图5b中的开始位置将他们的食指18移动得更靠近屏幕12同时指在感兴趣的飞机88处,使得放大率增加,如图5c所示。 [0035] FIG. 5b and 5c reflects a situation in which the user from a starting position in Fig. 5b their index finger 18 is moved closer to the screen 12 while 88 means the aircraft of interest, so that the magnification increases as shown FIG 5c. 和朝向屏幕12移动一样,用户横向移动他们的手指18,则放大率将增加并且放大的区域将移动以跟随手指18的横向移动。 And 12 is moved toward the screen, users lateral move their fingers 18, will increase the magnification and the enlarged area will move to follow the lateral movement of the finger 18.

[0036]如本领域中的技术人员将意识到的,可以对上述实施例进行修改,而不脱离于由随附的权利要求限定的本发明的范围。 [0036] As one skilled in the art will appreciate, the embodiments described above can be modified without departing from the scope in the appended claims define the invention.

[0037]例如,屏幕12的数量可以从一到任意数自由变化。 [0037] For example, the number of screen 12 can vary from one to any number of freedom. 此外,屏幕12的类型可以改变。 In addition, the type of the screen 12 can be changed. 例如,屏幕12可以是像等离子屏幕、LCD屏幕、OLED屏幕的平屏幕,或者它可以是电子射线管或仅是图像被投射到其上的表面。 For example, the screen 12 can be like plasma screens, LCD screens, OLED screen, flat screen, or it may be an electronic ray tube or merely an image is projected onto the surface thereon. 当使用多个屏幕12时,他们不需要共同的类型。 When using multiple screens 12, they do not need a common type. 尽管CCD照相机是优选的,但使用的照相机16的类型还可以变化。 Although a CCD camera is preferred, but the type used by the camera 16 may also be changed. 照相机16可以用可见光操作,但可以使用其它波长的电磁辐射。 The camera 16 can operate with visible light, but can use other wavelengths of electromagnetic radiation. 例如,红外线照相机可以被用在低光条件下。 For example, an infrared camera can be used in low light conditions.

[0038]软件可以被设置为监测任何物体并确定从屏幕12选择什么物体。 [0038] software can be set to monitor any object and determine the choice of what the object from the screen 12. 例如,上文描述的用户的手指18。 For example, the user's finger 18 described above. 可替换地,可以使用例如棒状物或棍的指点装置。 Alternatively, you can use such as a rod or stick pointing device.

[0039]本发明可以被用来非常有效地访问被布置成树状结构的菜单。 [0039] The present invention can be used very effectively access are arranged in a tree structure menu. 例如,手指18可以指向屏幕12上呈现出的按钮或菜单选项以在屏幕12上产生新的信息显示。 For example, 18 can point a finger on the screen showing the 12 buttons or menu options to generate new information on the display screen 12. 之后,用户可以移动它们的手指18以指向另一个按钮或菜单选项以在屏幕12上产生另一个新的信息显示,等等。 Thereafter, the user can move their fingers 18 to point to another button or menu option on the screen 12 to produce another new information display, and so on. 因此,通过仅移动手指18使得它指向屏幕12的不同部分,允许用户非常快速地巡览树形菜单结构。 Therefore, only by moving the finger 18 so that it points to a different part of the screen 12, allows the user to very quickly navigate the menu tree structure.

[0040]例如可以通过追踪手指18的指尖来连续确定用户的手指18的位置。 [0040] for example, by tracking the fingertip of the finger 18 continuously determines the position of the finger 18 of the user. 这能够使手指18的移动速度被确定。 This enables the moving speed of the finger 18 is determined. 这个速度之后可以被用来控制屏幕12上的信息。 After this speed can be used to control the on-screen information 12. 例如,朝向屏幕12移动的速度可以被使用,使得逐渐移动引起与快速移动不同的反应。 For example, the speed of movement toward the screen 12 can be used, such movement causes the fast moving progressively different response. 还可以使用横向移动,使得不同的速度产生不同的结果。 You can also use lateral movement, so that different speeds to produce different results. 例如,较慢的横向移动可以引起显示在屏幕12上的物体在屏幕内来回移动,即,从左到右较慢移动可以将物体从中心位置移动到屏幕12的右手边缘上的位置。 For example, a slow lateral movement can cause the object displayed on the screen 12 to move around the screen, that is, from left to right slower moving object can be moved from the center position to the right-hand edge of the screen position 12 on. 相反,快速移动可也引起物体从屏幕12被移除,S卩,从左到右快速移动可以引起物体飞出屏幕12的右手边缘。 On the contrary, can also cause fast moving objects to be removed from the screen 12, S Jie, from left to right can cause fast moving objects flying out of the screen right-hand edge 12.

[0041]如上提及的,主处理器42可以监测多于一个类似于用户手指18的物体。 [0041] As mentioned above, the main processor 42 can monitor more than one finger of the user object 18 is similar. 这能够使多个物体被用来控制屏幕12上的信息。 This enables a plurality of objects 12 is used to control information on the screen. 一对物体可以被用来与屏幕上的不同控制独立地交互,例如以调节选择新项目并改变与选择的项目相关的信息的类型。 One pair of objects can be used to interact independently with the different controls on the screen, for example, to adjust to change and select a new project and select the type of project-related information. 两个物体还可以一起使用。 Two objects can also be used together. 显示在屏幕上的图像可以用两个手14操作。 An image displayed on the screen 14 can use two hands to operate. 显示在屏幕12上的物体可以旋转。 Displayed on the screen 12 of the object can be rotated. 例如,用户可以将他们的手放在相同的高度,每个手的手指18指向显示在屏幕12上的物体的左手和右手边缘。 For example, users can place their hands on the same height, each finger of the hand 18 pointing left and right edges of the display screen 12 of the object. 通过使左手14向屏幕12移动并使右手14远离屏幕12移动,可以使物体绕垂直轴线顺时针旋转。 14 by left and right-hand screen 12 is moved away from the 14 to the screen 12 moves, so that the object can be rotated clockwise around the vertical axis. 如果一个手14放在另一个手上,物体可以绕水平轴线旋转。 If a hand 14 on the other hand, the object can rotate about a horizontal axis. 旋转轴线可以被限定为对应于手指18的指尖之间的线。 The rotation axis may be defined as corresponding to the fingertip of the finger 18 between the lines.

Citas de patentes
Patente citada Fecha de presentación Fecha de publicación Solicitante Título
CN101040242A *13 Oct 200519 Sep 2007皇家飞利浦电子股份有限公司System for 3D rendering applications using hands
CN101636207A *3 Mar 200827 Ene 2010科乐美数码娱乐株式会社Game device, progress control method, information recording medium, and program
*US2002/0036617 Título no disponible
*US2007/0146347 Título no disponible
*US2009/0254855 Título no disponible
US7348963 *5 Ago 200525 Mar 2008Reactrix Systems, Inc.Interactive video display system
Clasificaciones
Clasificación internacionalG06F3/01, G06F3/042