WO2015081705A1 - Air touch-control system - Google Patents

Air touch-control system Download PDF

Info

Publication number
WO2015081705A1
WO2015081705A1 PCT/CN2014/082897 CN2014082897W WO2015081705A1 WO 2015081705 A1 WO2015081705 A1 WO 2015081705A1 CN 2014082897 W CN2014082897 W CN 2014082897W WO 2015081705 A1 WO2015081705 A1 WO 2015081705A1
Authority
WO
WIPO (PCT)
Prior art keywords
processing module
person
display screen
camera
angle
Prior art date
Application number
PCT/CN2014/082897
Other languages
French (fr)
Chinese (zh)
Inventor
刘美鸿
高炜
徐万良
Original Assignee
深圳市亿思达科技集团有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市亿思达科技集团有限公司 filed Critical 深圳市亿思达科技集团有限公司
Publication of WO2015081705A1 publication Critical patent/WO2015081705A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Definitions

  • the present invention relates to the field of handwriting input technology, and in particular to an air touch system.
  • the handwriting input device generally comprises: a stylus and a tablet or a touch screen.
  • the stylus and the tablet or the touch screen are close-range input devices, and the remote input cannot be completed.
  • the user is beyond a certain distance from the electronic device or the user's hand is occupied by other things, it is inconvenient to complete the manipulation of the electronic device.
  • the present invention provides an air touch system.
  • the present invention provides an air touch control system that acquires a position image of a person by using a 3D camera provided at a front end of the display screen, and generates a corresponding operation of the finger position and the motion track of the person according to the position image.
  • the instruction adjusts the tilt angle of the dynamic grating of the display screen according to the angle between the human eye and the display screen, and rotates the 3D scene, so that the air touch system provided by the invention can not only input in the air, but also can make the user move during the movement. Still able to get a holographic 3D effect experience.
  • the invention provides an air touch system, comprising:
  • the 3D camera is disposed on a front end surface of the display screen, and is used for real-time acquiring a position image of a person in a visual field;
  • a first processing module electrically connected to the 3D camera, configured to calculate a position of a human finger on the screen and a motion track according to the position image
  • a first execution module electrically connected to the first processing module, configured to generate a corresponding operation instruction according to a position of the finger on the screen and a motion track, and execute the operation instruction;
  • a second processing module electrically connected to the 3D camera, for calculating an angle of the human eye relative to the display screen according to the position image
  • a second execution module electrically connected to the second processing module, configured to adjust an inclination angle of the dynamic grating of the display screen according to the angle to adapt to a position of the human eye;
  • the third execution module is electrically connected to the second processing module, and is configured to rotate the virtual 3D scene according to the angle, so that the human eye can view the scene from different angles to obtain a holographic 3D experience.
  • the present invention provides an air touch system that acquires a position image of a person by using a 3D camera provided at a front end of the display screen, and generates a person according to the position image, which is different from the prior art.
  • the operation command corresponding to the finger position and the motion track adjusts the tilt angle of the dynamic grating of the display screen according to the angle between the human eye and the display screen, and rotates the 3D scene, so that the air touch system provided by the present invention can not only realize
  • the operational commands are entered in the air and can enable the user to obtain a holographic 3D effect experience during exercise.
  • FIG. 1 is a schematic structural view of an embodiment of an air touch system of the present invention.
  • FIG. 1 is a schematic structural diagram of an embodiment of an air touch system according to the present invention.
  • the air touch system provided by the present invention includes a 3D camera 11 , a first processing module 12 , a first execution module 13 , a second processing module 14 , a second execution module 15 , and a third execution module 16 .
  • the 3D camera 11 is disposed on the front end surface of the display screen as an input device for acquiring the position image of the person in the field of view in real time;
  • the first processing module 12 is electrically connected to the 3D camera 11 , and the first processing module is 12 is used to calculate the position of the human finger on the screen according to the position image of the person acquired by the 3D camera 11, and the continuous position coordinates outline the movement track of the human hand;
  • the first execution module 13 and the first processing module 12 are electrically The first execution module 13 is configured to generate a corresponding click or swipe action according to the position of the finger on the screen and the motion track, and execute the operation instruction;
  • the second processing module 14 is electrically connected to the 3D camera 11 , and second The processing module 14 is configured to calculate an angle of the human eye relative to the display screen according to the position image of the person.
  • the second processing module 14 can obtain the connection of the human eye to the center of the display according to the position image of the person acquired by the 3D camera 11 .
  • the tilt angle of the grating is adapted to the position of the human eye; the third execution module 16 is electrically connected to the second processing module 14 for rotating the virtual 3D scene according to the angle, so that the human eye can view the scene from different angles to obtain the holographic 3D experience. .
  • the present invention provides an air touch system that acquires a position image of a person by using a 3D camera disposed at a front end of the display screen, and generates an operation instruction corresponding to a finger position and a motion track of the person according to the position image. Adjusting the tilt angle of the dynamic grating of the display screen and rotating the 3D scene according to the angle between the human eye and the display screen, so that the air touch control system provided by the invention can not only realize the air input operation command, but also enable the user to Get a holographic 3D effect experience during exercise.

Abstract

An air touch-control system comprises: a 3D camera (11), configured to acquire in real time a location image of a person in a view range; a first processing module (12), configured to compute a location and a movement trajectory above a screen of a finger of the person according to the location image; a first execution module (13), configured to generate a corresponding operation instruction according to the location and the movement trajectory above the screen of the finger, and execute the operation instruction; a second processing module (14), configured to compute an included angle from an eye of the person relative to a display screen according to the location image; a second execution module (15), configured to adjust an inclining angle of dynamic gratings of the display screen according to the included angle to adapt to a location of the eye of the person; and a third execution module (16), configured to rotate a virtual 3D scene according to the included angle, so that the eye of the person may see the scene from different angles to acquire holographic 3D experience. By means of the foregoing manner, air input is implemented, and a user can still be provided with experience of a holographic 3D effect during movement.

Description

一种空中触控系统 Air touch system
【技术领域】[Technical Field]
本发明涉及手写输入技术领域，特别是涉及一种空中触控系统。The present invention relates to the field of handwriting input technology, and in particular to an air touch system.
【背景技术】 【Background technique】
现有的电子设备中比较常见的输入方式包括键盘输入、语音输入和手写输入。其中，手写输入装置一般包括：手写笔和手写板或触摸屏。然而，手写笔和手写板或触摸屏是近距离输入设备，无法完成远距离输入，在用户距离电子设备超出一定的距离或者用户的手被其他东西占用，将不便于完成对电子设备进行操控。The more common input methods in existing electronic devices include keyboard input, voice input, and handwriting input. Wherein, the handwriting input device generally comprises: a stylus and a tablet or a touch screen. However, the stylus and the tablet or the touch screen are close-range input devices, and the remote input cannot be completed. When the user is beyond a certain distance from the electronic device or the user's hand is occupied by other things, it is inconvenient to complete the manipulation of the electronic device.
为解决上述技术问题，本发明提供一种空中触控系统。To solve the above technical problems, the present invention provides an air touch system.
【发明内容】 [Summary of the Invention]
为了至少部分解决以上问题，本发明提供一种空中触控系统，利用在显示屏的前端设置的3D摄像头获取人的位置图像，并根据所述位置图像生成人的手指位置及运动轨迹对应的操作指令，根据人眼与显示屏的夹角对显示屏的动态光栅的倾斜角度进行调整以及对3D场景进行旋转，使得本发明提供的空中触控系统不但可以空中输入，而且可以使得用户运动过程中仍然能够获得全息3D效果体验。In order to at least partially solve the above problems, the present invention provides an air touch control system that acquires a position image of a person by using a 3D camera provided at a front end of the display screen, and generates a corresponding operation of the finger position and the motion track of the person according to the position image. The instruction adjusts the tilt angle of the dynamic grating of the display screen according to the angle between the human eye and the display screen, and rotates the 3D scene, so that the air touch system provided by the invention can not only input in the air, but also can make the user move during the movement. Still able to get a holographic 3D effect experience.
本发明提供一种空中触控系统，包括：The invention provides an air touch system, comprising:
3D摄像头，所述3D摄像头设置于显示屏的前端面上，用于实时获取视野范围内人的位置图像；a 3D camera, the 3D camera is disposed on a front end surface of the display screen, and is used for real-time acquiring a position image of a person in a visual field;
第一处理模块，与所述3D摄像头电连接，用于根据所述位置图像计算出人的手指在屏幕上空的位置以及运动轨迹；a first processing module, electrically connected to the 3D camera, configured to calculate a position of a human finger on the screen and a motion track according to the position image;
第一执行模块，与所述第一处理模块电连接，用于根据手指在屏幕上空的位置以及运动轨迹生成对应的操作指令，并执行所述操作指令；a first execution module, electrically connected to the first processing module, configured to generate a corresponding operation instruction according to a position of the finger on the screen and a motion track, and execute the operation instruction;
第二处理模块，与所述3D摄像头电连接，用于根据所述位置图像计算人眼相对于显示屏幕的夹角；a second processing module, electrically connected to the 3D camera, for calculating an angle of the human eye relative to the display screen according to the position image;
第二执行模块，与所述第二处理模块电连接，用于根据所述夹角调整显示屏的动态光栅的倾斜角度以适应人眼的位置；a second execution module electrically connected to the second processing module, configured to adjust an inclination angle of the dynamic grating of the display screen according to the angle to adapt to a position of the human eye;
第三执行模块，与所述第二处理模块电连接，用于根据所述夹角对虚拟3D场景进行旋转，使得人眼可从不同角度观看场景获得全息3D体验。The third execution module is electrically connected to the second processing module, and is configured to rotate the virtual 3D scene according to the angle, so that the human eye can view the scene from different angles to obtain a holographic 3D experience.
本发明的有益效果是：区别于现有技术的情况，本发明提供的一种空中触控系统，利用在显示屏的前端设置的3D摄像头获取人的位置图像，并根据所述位置图像生成人的手指位置及运动轨迹对应的操作指令，根据人眼与显示屏的夹角对显示屏的动态光栅的倾斜角度进行调整以及对3D场景进行旋转，使得本发明提供的空中触控系统不但可以实现空中输入操作指令，而且可以使得用户能够在运动过程中获得全息3D效果体验。The present invention provides an air touch system that acquires a position image of a person by using a 3D camera provided at a front end of the display screen, and generates a person according to the position image, which is different from the prior art. The operation command corresponding to the finger position and the motion track adjusts the tilt angle of the dynamic grating of the display screen according to the angle between the human eye and the display screen, and rotates the 3D scene, so that the air touch system provided by the present invention can not only realize The operational commands are entered in the air and can enable the user to obtain a holographic 3D effect experience during exercise.
【附图说明】 [Description of the Drawings]
图1是本发明的空中触控系统的一实施例的结构示意图。1 is a schematic structural view of an embodiment of an air touch system of the present invention.
【具体实施方式】 【detailed description】
下面结合附图和实施例对本发明进行详细说明。The invention will now be described in detail in conjunction with the drawings and embodiments.
请参见图1，图1是本发明的空中触控系统的一实施例的结构示意图。如图1所示，本发明提供的空中触控系统包括3D摄像头11、第一处理模块12、第一执行模块13、第二处理模块14、第二执行模块15和第三执行模块16。Please refer to FIG. 1. FIG. 1 is a schematic structural diagram of an embodiment of an air touch system according to the present invention. As shown in FIG. 1 , the air touch system provided by the present invention includes a 3D camera 11 , a first processing module 12 , a first execution module 13 , a second processing module 14 , a second execution module 15 , and a third execution module 16 .
在本实施例中，3D摄像头11设置于显示屏的前端面上，作为输入设备，用于实时获取视野范围内人的位置图像；第一处理模块12与3D摄像头11电连接，第一处理模块12用于根据3D摄像头11获取到的人的位置图像计算出人的手指在屏幕上空的位置，而连续的位置坐标勾勒出了人手的运动轨迹；第一执行模块13与第一处理模块12电连接，第一执行模块13用于根据手指在屏幕上空的位置以及运动轨迹生成对应的点击或者划动的动作，并执行所述操作指令；第二处理模块14与3D摄像头11电连接，第二处理模块14用于根据人的位置图像计算人眼相对于显示屏幕的夹角，具体的，第二处理模块14根据3D摄像头11获取到的人的位置图像可以得到人眼到显示屏中心的连线与显示屏之间的夹角；第二执行模块15与第二处理模块14电连接，第二执行模块15用于根据夹角调整显示屏的动态光栅的倾斜角度以适应人眼的位置；第三执行模块16与第二处理模块14电连接，用于根据夹角对虚拟3D场景进行旋转，使得人眼可从不同角度观看场景获得全息3D体验。In this embodiment, the 3D camera 11 is disposed on the front end surface of the display screen as an input device for acquiring the position image of the person in the field of view in real time; the first processing module 12 is electrically connected to the 3D camera 11 , and the first processing module is 12 is used to calculate the position of the human finger on the screen according to the position image of the person acquired by the 3D camera 11, and the continuous position coordinates outline the movement track of the human hand; the first execution module 13 and the first processing module 12 are electrically The first execution module 13 is configured to generate a corresponding click or swipe action according to the position of the finger on the screen and the motion track, and execute the operation instruction; the second processing module 14 is electrically connected to the 3D camera 11 , and second The processing module 14 is configured to calculate an angle of the human eye relative to the display screen according to the position image of the person. Specifically, the second processing module 14 can obtain the connection of the human eye to the center of the display according to the position image of the person acquired by the 3D camera 11 . An angle between the line and the display screen; the second execution module 15 is electrically connected to the second processing module 14, and the second execution module 15 is configured to adjust the movement of the display screen according to the angle The tilt angle of the grating is adapted to the position of the human eye; the third execution module 16 is electrically connected to the second processing module 14 for rotating the virtual 3D scene according to the angle, so that the human eye can view the scene from different angles to obtain the holographic 3D experience. .
通过上述方式，本发明提供的一种空中触控系统，利用在显示屏的前端设置的3D摄像头获取人的位置图像，并根据所述位置图像生成人的手指位置及运动轨迹对应的操作指令，根据人眼与显示屏的夹角对显示屏的动态光栅的倾斜角度进行调整以及对3D场景进行旋转，使得本发明提供的空中触控系统不但可以实现空中输入操作指令，而且可以使得用户能够在运动过程中获得全息3D效果体验。In the above manner, the present invention provides an air touch system that acquires a position image of a person by using a 3D camera disposed at a front end of the display screen, and generates an operation instruction corresponding to a finger position and a motion track of the person according to the position image. Adjusting the tilt angle of the dynamic grating of the display screen and rotating the 3D scene according to the angle between the human eye and the display screen, so that the air touch control system provided by the invention can not only realize the air input operation command, but also enable the user to Get a holographic 3D effect experience during exercise.
在上述实施例中，仅对本发明进行了示范性描述，但是本领域技术人员在阅读本专利申请后可以在不脱离本发明的精神和范围的情况下对本发明进行各种修改。 In the above-described embodiments, the present invention has been exemplarily described, and various modifications of the present invention may be made without departing from the spirit and scope of the invention.

Claims (1)

  1. 一种空中触控系统，其特征在于，所述空中触控系统包括：An air touch system, wherein the air touch system comprises:
    3D摄像头，所述3D摄像头设置于显示屏的前端面上，用于实时获取视野范围内人的位置图像；a 3D camera, the 3D camera is disposed on a front end surface of the display screen, and is used for real-time acquiring a position image of a person in a visual field;
    第一处理模块，与所述3D摄像头电连接，用于根据所述位置图像计算出人的手指在屏幕上空的位置以及运动轨迹；a first processing module, electrically connected to the 3D camera, configured to calculate a position of a human finger on the screen and a motion track according to the position image;
    第一执行模块，与所述第一处理模块电连接，用于根据手指在屏幕上空的位置以及运动轨迹生成对应的操作指令，并执行所述操作指令；a first execution module, electrically connected to the first processing module, configured to generate a corresponding operation instruction according to a position of the finger on the screen and a motion track, and execute the operation instruction;
    第二处理模块，与所述3D摄像头电连接，用于根据所述位置图像计算人眼相对于显示屏幕的夹角；a second processing module, electrically connected to the 3D camera, for calculating an angle of the human eye relative to the display screen according to the position image;
    第二执行模块，与所述第二处理模块电连接，用于根据所述夹角调整显示屏的动态光栅的倾斜角度以适应人眼的位置；a second execution module electrically connected to the second processing module, configured to adjust an inclination angle of the dynamic grating of the display screen according to the angle to adapt to a position of the human eye;
    第三执行模块，与所述第二处理模块电连接，用于根据所述夹角对虚拟3D场景进行旋转，使得人眼可从不同角度观看场景获得全息3D体验。The third execution module is electrically connected to the second processing module, and is configured to rotate the virtual 3D scene according to the angle, so that the human eye can view the scene from different angles to obtain a holographic 3D experience.
PCT/CN2014/082897 2013-12-02 2014-07-24 Air touch-control system WO2015081705A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310638317.3 2013-12-02
CN201310638317.3A CN103995620A (en) 2013-12-02 2013-12-02 Air touch system

Publications (1)

Publication Number Publication Date
WO2015081705A1 true WO2015081705A1 (en) 2015-06-11

Family

ID=51309806

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/082897 WO2015081705A1 (en) 2013-12-02 2014-07-24 Air touch-control system

Country Status (2)

Country Link
CN (1) CN103995620A (en)
WO (1) WO2015081705A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105704469A (en) * 2015-11-26 2016-06-22 中国传媒大学 Holographic video playback method
CN105704470A (en) * 2015-11-26 2016-06-22 中国传媒大学 Holographic video preview method
CN106055108B (en) * 2016-06-10 2020-11-13 北京行云时空科技有限公司 Virtual touch screen control method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981616A (en) * 2012-11-06 2013-03-20 中兴通讯股份有限公司 Identification method and identification system and computer capable of enhancing reality objects
CN103180893A (en) * 2011-08-23 2013-06-26 索尼公司 Method and system for use in providing three dimensional user interface
CN103176605A (en) * 2013-03-27 2013-06-26 刘仁俊 Control device of gesture recognition and control method of gesture recognition
CN103279191A (en) * 2013-06-18 2013-09-04 北京科技大学 3D (three dimensional) virtual interaction method and system based on gesture recognition technology
US20130249786A1 (en) * 2012-03-20 2013-09-26 Robert Wang Gesture-based control system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8368753B2 (en) * 2008-03-17 2013-02-05 Sony Computer Entertainment America Llc Controller with an integrated depth camera
CN102253713B (en) * 2011-06-23 2016-10-12 康佳集团股份有限公司 Towards 3 D stereoscopic image display system
CN102508546B (en) * 2011-10-31 2014-04-09 冠捷显示科技（厦门）有限公司 Three-dimensional (3D) virtual projection and virtual touch user interface and achieving method
CN103248905A (en) * 2013-03-22 2013-08-14 深圳市云立方信息科技有限公司 Display device and visual display method for simulating 3D scene

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103180893A (en) * 2011-08-23 2013-06-26 索尼公司 Method and system for use in providing three dimensional user interface
US20130249786A1 (en) * 2012-03-20 2013-09-26 Robert Wang Gesture-based control system
CN102981616A (en) * 2012-11-06 2013-03-20 中兴通讯股份有限公司 Identification method and identification system and computer capable of enhancing reality objects
CN103176605A (en) * 2013-03-27 2013-06-26 刘仁俊 Control device of gesture recognition and control method of gesture recognition
CN103279191A (en) * 2013-06-18 2013-09-04 北京科技大学 3D (three dimensional) virtual interaction method and system based on gesture recognition technology

Also Published As

Publication number Publication date
CN103995620A (en) 2014-08-20

Similar Documents

Publication Publication Date Title
US11221730B2 (en) Input device for VR/AR applications
TWI546725B (en) Continued virtual links between gestures and user interface elements
US9395821B2 (en) Systems and techniques for user interface control
US9658695B2 (en) Systems and methods for alternative control of touch-based devices
JP2023515525A (en) Hand Gesture Input for Wearable Systems
JP6619003B2 (en) Removable input / output module with adjustment mechanism
US11853527B2 (en) Devices, methods, and graphical user interfaces for providing computer-generated experiences
WO2019034038A1 (en) Vr content capturing method, processing device and system, and storage medium
WO2016123816A1 (en) Aiming method and apparatus for shooting game
JP7382994B2 (en) Tracking the position and orientation of virtual controllers in virtual reality systems
Pietroszek et al. Watchcasting: Freehand 3D interaction with off-the-shelf smartwatch
Kharlamov et al. TickTockRay: smartwatch-based 3D pointing for smartphone-based virtual reality
WO2011137226A1 (en) Spatial-input-based cursor projection systems and methods
KR20170107357A (en) Multi-modal gesture based interactive system and method using one single sensing system
US20150002475A1 (en) Mobile device and method for controlling graphical user interface thereof
TW202105129A (en) Artificial reality systems with personal assistant element for gating user interface elements
WO2015081705A1 (en) Air touch-control system
TWI486815B (en) Display device, system and method for controlling the display device
JP2019051297A5 (en) Programs and information processing equipment
Edelmann et al. The dabr-a multitouch system for intuitive 3d scene navigation
Arslan et al. E-Pad: Large display pointing in a continuous interaction space around a mobile device
US20200057501A1 (en) System, devices, and methods for remote projection of haptic effects
WO2012114791A1 (en) Gesture operation system
Mendes Manipulation of 3d objects in immersive virtual environments
TW201913298A (en) Virtual reality system capable of showing real-time image of physical input device and controlling method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14868120

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14868120

Country of ref document: EP

Kind code of ref document: A1