Description GESTURE COGNITIVE DEVICE FOR UBIQUITOUS NETWORK Technical Field
[1] The present invention relates to a gesture cognitive device for transmitting command signal that is used to control the networked electronic components in a ubiquitous network, and more particularly to a gesture cognitive device for the ubiquitous network using a hand-held device. Background Art
[2] A term 'ubiquitous' originated from the Latin language means being present everywhere at the same time like water or air. The term 'ubiquitous' is combined with information telecommunication technology so that a term 'ubiquitous computing' or 'ubiquitous network' has appeared. In the ubiquitous network, a computing device is mounted in almost things including a car, a classroom or a cloth, glasses and a watch which one wears, and the things with built-in computing device are networked by LAN and Internet infrastructure, so the ubiquitous network makes the computers to live out here in the world with people and information networks are accessible at any time and from anywhere without conscious recognition of the computer system or network. For example, in the ubiquitous network, the access of people can be detected by an electric light and in responsive thereto the electric light is automatically turned on or by using a portable computing device such as PDA or lap-top computer people can control the operation of a door or a household electrical appliances including a washing machine and refrigerator, which are located long distance.
[3] As a command transmission means for remote-controlling various computing devices including the household electrical appliances in the ubiquitous network, a desk-top computer or the portable computer such as the PDA or the lap-top computer is now used. In order to control the operation of the computing devices, the user must list all computing devices networked and the operation modes thereof via the desktop computer or the portable computer and then select a given computing device and operation mode thereof. Accordingly, several times button handling inevitably accompanies the operation of the computing devices in the ubiquitous network.
[4] Recently, the hand-held device such as the PDA or the hand-phone is world-widely used along with the development of the wireless telecommunication technology. So, it is expected that the hand-held device becomes the most promising command
transmission means to quickly and conveniently transmit command signal to the ubiquitous -networked computing device, in the next-coming ubiquitous network. However, the button handling of the hand-held device is more difficult or inconvenient than that of the desktop computer or the laptop computer because of their small size.
[5] Therefore, the present inventor has contrived the hand-held device suitable for the ubiquitous network where people can communicate with the ubiquitous-networked computing devices based on a gesture cognition of hand which grips the hand-held device, not button handling of the hand-held device. Disclosure of Invention Technical Problem
[6] Accordingly, an object of the present invention is to provide a gesture cognitive device for transmitting command signal that is used to control the networked electronic components in a ubiquitous network. Technical Solution
[7] The ubiquitous gesture cognitive device in accordance with the present invention was built in a hand-held device. The ubiquitous gesture cognitive device comprises a means for detecting a motion of a hand which grasps said hand-held device and for producing signal corresponding to said motion of the hand; a hexa-code generator for producing hexa-code corresponding to said motion of the hand based on signal produced from said means for detecting and producing signal corresponding to said motion of the hand; a data decision part for receiving said hexa-code from said hexa- code generator, accessing a database having data-type file containing hexa-code fields and data type fields mapped to said hexa-code fields and having command file containing hexa-code fields and command contents fields mapped to said hexa-code fields, and for checking whether the received hexa-code is identical with a hexa-code in said data-type file or said command file ; and a command signal generating and transmitting part for producing control signal for carrying out said command contents in said command file and then transmitting said control signal to said ubiquitous- networked electronic device. Here, data mapped to the identically searched hexa-code may be a key data or a command data. The key data is for indicating commencement of control on the ubiquitous-networked electronic devices and the command data is for indicating any operations of the ubiquitous-networked electronic device except for the control commencement thereof and the database is prepared in said hand-held device.
[8] Further the ubiquitous gesture cognitive device comprises a controller for making said command signal generating and transmitting part produce said control signal, in
case where message informing a successive receipt of hexa-codes representing said key data and said command data, is transmitted from said data decision part.
[9] Also, the ubiquitous gesture cognitive device further comprises an indicator for informing, under control of said controller, operation state of said ubiquitous gesture cognitive device according to whether said controller receives message informing a successive receipt of hexa-code representing said key data and hexa-code representing said command, from said data decision part; and wherein said controller forces said indicator to inform user of operation error of said ubiquitous gesture cognitive device in case where message informing a successive receipt of hexa-codes representing said key data and said command data is not received from said data decision part.
[10] In detail, the hexa-code generator comprises an operation module for dividing output of means for detecting the motion of the hand into three-level value, an operation module for allotting a positive value or a negative value to the three-level values, an operation module for producing average value of output of the allotting operation module, an operation module for producing binary data responsive to output of the average value producing operation module and an operation module for producing a hexa-code from the binary data received. And, the data decision part accesses the data-type file containing hexa-code field in the database, checks whether a hexa-code identical with the received hexa-code is included in the data-type file, and when the identical hexa-code is searched, examines whether type of data mapped to the identical hexa-code in the data-type file is the key data. Also the data decision part monitors whether another hexa-code is successively received within a given time after receipt of hexa-code mapped to the key data, if so, accesses said command file, checks whether a hexa-code identical with the successively received hexa-code is included in the command file and when the identical hexa-code is searched, transmits the identically searched hexa-code in the command file and command content mapped to the identically searched hexa-code in the command file to the controller. Description of Drawings
[11] Fig. 1 is a block diagram for schematically illustrating a gesture cognitive device for a ubiquitous network according to an embodiment of the present invention;
[12] Fig. 2 is a drawing for illustrating a hand-held device in which there is mounted an accelerometer module that is one of components of a ubiquitous gesture cognitive device according to the present invention;
[13] Fig. 3 is a pulse diagram of binary data produced from the hand-held device of Fig. 2 in case where the hand-held device rotates around Y-axis in the counterclockwise
direction by 360 degrees;
[14] Fig. 4 is a pulse diagram of binary data produced from the hand-held device of Fig. 2 in case where the hand-held device rotates around X-axis in the counterclockwise direction by 90 degrees and returns to its original position and then the above- mentioned motion of rotation and return is repeated one more ; and
[15] Fig. 5 is a pulse diagram of binary data produced from the hand-held device of Fig. 2 in case where the hand-held device rotates around Y-axis in the counterclockwise direction by 90 degrees and returns to its original position and then rotates around Y- axis in the clockwise direction by 90 degrees and returns to its original position. Mode for Invention
[16] With reference to the attached drawings the spirit of the present invention will be explained.
[17] Fig. 1 shows a block diagram for schematically illustrating a gesture cognitive device for a ubiquitous network (which is referred to as a ubiquitous gesture cognitive device hereinafter) according to an embodiment of the present invention. The ubiquitous gesture cognitive device is installed in a hand-held device and detects a motion of hand which grasps the hand-held device. The ubiquitous gesture cognitive device includes an accelerometer module 11, an analogue/digital (A/D) converter 12 for converting the output of the accelerometer module 11 into digital signal, a hexa- code generator 13, a data decision part 14 and a command signal generating and transmitting part 18. In case where the accelerometer module 11 producing digital signal is used, the A/D converter 12 is not necessary.
[18] The accelerometer module 11 senses the motion of the hand and produces an analogue signal corresponding to the motion of the hand or the hand-held device. The hexa-code generator 13 receives the output of the A/D converter 12 and then produces the hexa-codes corresponding to the motion of the hand. The data decision part 14 receives the output of the hexa-code generator 13 and searches whether there is a key data or a command data corresponding to the received hexa-code. The command signal generating and transmitting part 18 generates a control signal for operational action corresponding to the command data when the command data is searched, and then transmits the generated control signal to the ubiquitous -networked computing device. The key data is a data indicative of commencement of control on the ubiquitous- networked computing device, and the command data produced after the key data indicates any operations of the ubiquitous-networked computing device except for the control commencement thereof.
[19] The search of the key data or the command data in the data decision part 14 is carried out by accessing a database 15 in which there are contained a data-type file including hexa-code fields and data kind fields mapped to the hexa-code fields and a command file including hexa-code fields and command content fields mapped to the hexa-code fields. User of the ubiquitous gesture ccgnitive device may variously and freely set up the mapping of the hexa-code (or moving pattern of the hand) to the key data and the command data and the mapping of the hexa-code to the commend content.
[20] More detail, the hexa-code generator 13 includes an operation module for dividing output of the A/D converter 12 into three-level value, an operation module for allotting the positive value or negative value to three-level values of each axis, an operation module for producing average value of output of the allotting operation module, an operation module for producing binary data responsive to the output of average value producing operation module and an operation module for producing the hexa-code from the binary data received. The command signal generating and transmitting part 18 can be easily constructed by using a well-known technique employed for generating and transmitting command signal in the desktop computer or the laptop computer.
[21] Also, the ubiquitous gesture cognitive device has a controller 16 for controlling the operation of all elements therein, a timer 17 and a warning signal indicator 19 for producing an error message of the ubiquitous gesture ccgnitive device. The exemplary warning signal indicator 19 includes a speaker or a monitor.
[22] The operating error of the ubiquitous gesture cognitive device is produced when the successive hexa-code is not generated or the command data corresponding to the hexa-code representative of the motion of the hand is not detected, for a given time after the input of the key data. If the operating error of the ubiquitous gesture cognitive device is produced two times, the controller 16 initializes the ubiquitous gesture ccgnitive device and the hand-held device returns to the typical communication apparatus.
[23] In operation, if one moves one's hand which holds the hand-held device with the ubiquitous gesture cognitive device, the accelerometer module 11 of the ubiquitous gesture ccgnitive device produces analogue signals indicating the motion of the hand and the output of the accelerometer module 11 is converted into digital signals at the A/D converter 12.
[24] Thereafter, the hexa-code generator 13 receives the output of the A/D converter 12 and produces the hexa-code representative of the motion of the hand-held device. The hexa-code is transmitted to the data decision part 14. The data decision part 14
accesses the data-type file containing hexa-code field in the database 15 and checks whether the hexa-code identical with the transmitted hexa-code is included in the datatype file. When the identical hexa-code is detected, the data decision part 14 checks whether the type of the data mapped to the identical hexa-code in the data-type file is the key data. If there is no hexa-code identical with the received hexa-code in the datatype file of the database 15 or even if present, the data mapped to the identical hexa- code is not key data, the ubiquitous gesture cognitive device does not operate.
[25] In case where there is the identical hexa-code and the data mapped to the identical hexa-code is key data, the data decision part 14 transmits the key data to the controller 16. The controller 16 informs user of an operation commencement of ubiquitous gesture ccgnitive device. The controller 16 operates the timer 17 and checks whether the command data is successively received within a given time. The command data is data indicative of hand motion performed after the generation of key data. The hexa- code generator 13 produces the hexa-code corresponding to the hand motion via the accelerometer module 11 and the A/D converter 12. The data decision part 14 receives the following hexa-code to access the command file containing hexa-code field in the database 15, thereby checking whether the hexa-code identical with the received hexa- code is included in the command file. When the identical hexa-code is detected, the data decision part 14 transmits the command key mapped to the hexa-code to the controller 16. The controller 16 drives the command signal generating and transmitting part 18 so as to produce a control signal for carrying out command corresponding to the command key and then to transmit the same to the ubiquitous-networked computing devices. The command signal generating and transmitting part 18 may transmit the control signal using one manner of CDMA ( code-division multiple access ), blue tooth and 802.1 IB, according to types of the hand-held device and the ubiquitous -networked computing devices.
[26] In case that there is no hexa-code following the key data within in a given time, or in case that, even if there is following hexa-code, there is no command key corresponding to the hexa-code, the controller 16 informs the user of the operation error of the ubiquitous gesture cognitive device via the warning signal indicator 19. When the warning information is occurred two times, the controller 16 notifies the user of the operational end of the ubiquitous gesture cognitive device and an initialization of the hand-held device having built n the ubiquitous gesture cognitive device via the warning signal indicator 19, so that the hand-held device returns to the conventional communication apparatus.
[27] Now, referring to Figs. 2-3 and Figs. 4-5, the operation of an embodiment of the ubiquitous gesture cognitive device according to the present invention will be explained wherein the ubiquitous gesture cognitive device is installed in a hand-phone and an acceleration sensor serving as the accelerometer module is provided in the ubiquitous gesture cognitive device.
[28] Referring to Fig. 2, the hand-phone is comprised of a first body 21, a second body 22 pivotally connected to the first body 21 so as to open the first body 21, a displaying part 23 formed on the first body 21 and button part 24 (in which buttons are not shown) provided on the second body 22. The acceleration sensor 25 for detecting motion of hand which grasps the hand-phone is equipped within the second body 22. The acceleration sensor 25 is a 3-axis acceleration sensor wherein X-axis represents right and left direction of the hand-phone, Y-axis represents top and bottom direction of the same and Z-axis represents upper and lower direction of the same with respect to second body 22. In Fig. 2, the 3-axis acceleration sensor 25 is shown as the accelerometer module. However, the accelerometer module of the ubiquitous gesture cognitive device cannot be limited thereto and the two-axis acceleration sensor can be also used as the accelerometer module.
[29] The exemplary data- type file and command file recorded in the database 15 of the ubiquitous gesture cognitive device are expressed in table 1 and table 2, respectively.
[30] [Table 1] [31]
[32] [Table 2]
[33]
[34] The hand holding the hand-phone of Fig. 2 moves so that hexa-code corresponding to the motion of the hand is produced at the acceleration sensor 25. In detail, acceleration value from the acceleration sensor 25 becomes the gravitational acceleration value, as maximized value in case where positive direction of each axis is identical with the direction of gravitational acceleration. While acceleration value from the acceleration sensor 25 becomes the minus value of gravitational acceleration value, as minimized value in case where positive direction of each axis is at 180° with the direction of gravitational acceleration, and acceleration value from the acceleration sensor 25 becomes zero in case where positive direction of each axis is parallel to the direction of gravitational acceleration. Accordingly, the analogue acceleration value from the acceleration sensor 25 ranges between -1G through +1G. The analogue acceleration value is converted into digital signal at the A/D converter 12 in Fig. 1 which is then transmitted to the hexa-code generator 13 of Fig. 1.
[35] The hexa-code generator 13 divides the received value into three-level value. In order to definitely discriminate a motion of hand for the ubiquitous gesture cognitive device from the general and conventional motion of hand, the three-level value (X', Y', Z') of digital value ( X,Y, Z) according to one embodiment of the present invention is determined based on +0.7G and -0.7G. That is, if X is more than 0.7G (X > 0.7G) X' is '2', if X is less than -0.7G (X < -0.7G) X' is T and if X is between -0.7G and +0.7G (-0.7G < X < +0.7G) X' is 0. In the same way, if Y is more than 0.7G (Y > 0.7G) Y' is '2', if Y is less than -0.7G (Y < -0.7G) Y' is T, if Y is between -0.7G and +0.7G (-0.7G < Y < +0.7G) Y' is 0, and if Z is more than 0.7G (Z > 0.7G) Z' is '2', if Z is
less than -0.7G (Z < -0.7G) Z' is T and if Z is between -0.7G and +0.7G (-0.7G < Z < +0.7G) Z' is O. [36] Thereafter the three-level value (X',Y\ Z') is classified into two classes, the positive three-level value X , Y or Z and the negative three-level value X , Y or P P P N N Z . That is, if X' is '2', X is T and X' is O, X is T. Then if X' is T, X and X N P N P N both are 0. In the same way if Y' is '2', Y is 'l'. Y' is O, Y is T and if V is T, Y P N P and Y both are 0. Also, if Z' is '2', Z is . Z'is O, Z is T and if Y' is T, Y and N P N P Y are both 0. N
[37] For increasing operational reliability of the hand motion for ubiquitous gesture cognitive device, the three-level value are measured n times during one type motion of hand is performed to produce average values X , Y and Z of the positive three - AP AP AP level values X , Y and Z and average values X , Y and Z of the negative P P P AN AN AN three-level values X , Y and Z . That is average value X is represented by (X N N N AP PI + X + X + + X )/n and the average value X is represented by (X + X P2 P3 Pn AN Nl N2 + X + + X )/n. Similarly, the average value Y is represented by (Y +Y N3 Nn AP PI P2 + Y + + Y )/n,the average value Y is represented by (Y +Y + Y + P3 Pn AN Nl N2 N3 + Y )/n, the average value Z is represented by (Z +Z + Z + + Z Nn AP PI P2 P3 Pn )/n and the average value Z is represented by (Z +Z + Z + + Z )/ n. AN Nl N2 N3 Nn Then the average values X , Y and Z are classified into two class, the positive AP AP AP average values X p. , Y p. and Z p. and the negative average values X N. , Y N. and Z N.. That is, if X > 0.5, X = and X = 0 in the other cases, and if X > 0.5, X AP P' P' NP N' = and X = 0 in the other cases. Similarly, if Y > 0.5, Y = T and Y =0 in N' AP P' P' the other cases, and if Y > 0.5, Y = T and Y = 0 in the other cases. Also, if Z NP N' N' > 0.5, Z = T and Z = 0 in the other cases, and if Z > 0.5, Z = T and Z AP P' P' NP N' N' = 0 in the other cases. The binary data is produced from X p. , Y p. , Z p. X N. , Y N. and Z . The hexa-code is generated from the binary data. N'
[38] In detail, with reference to Fig. 3, the hand motion which produces hexa-code 01 10 02 20 in Table 1 will be explained. The hand-phone of Fig. 2, which is positioned on the palm of hand, is rotated around Y-axis in the counter clockwise direction (from right direction of the hand-phone to the left direction thereof) by 360 ° . Then pulse diagram signals X', Y' and Z' at the upper part of Fig. 3 are produced by the operation module for dividing output of the A/D converter 12 into three-level value. Wherein pulse signal in the first section tl indicates the 90 ° -rotation of the hand-phone in the counterclockwise direction and pulse signal in the second section t2 indicates the further 90 ° -rotation of the hand-phone in the counterclockwise direction so that the
rear side of the second body 22 of the hand-phone shown in Fig.2 is appeared. The pulse signal in the third section t3 indicates the 90 ° -rotation of the hand-phone of the second section t2 in the counterclockwise direction and pulse signal in the fourth section t4 indicates the further 90 ° -rotation of the hand-phone in the counterclockwise direction so that the posture of hand-phone returns to its initial state as shown in Fig. 2. Positive value or negative value is assigned to pulse signals X',Y' and Z' at the first section tl through the fourth section t4, according to their value by the operation module for allotting the positive value or negative value to three-level values of each axis and average values of positive value and negative value are produced by the operation module for producing average value of output of the allotting operation module. Thereby pulse signals X p. , X N. , Y p. Y N. , Z p. and Z N. at the lower part of Fig. 3 are produced. The binary data of pulse signals X p. , X N. , Y p. Y N. , Z p. and Z N. at the first section tl through the fourth section t4 are produced as 000001(tl), 010000(t2), 000010(t3) and 100000(t4), and then produced binary data is finally transformed to the hexa-codes of 01 , 10 , 02 and 20 HEX HEX HEX HEX
[39] In the same way, when the hand-phone of Fig. 2 rotates around X-axis in the counterclockwise direction by 90 degrees and returns to its original position and then the above-mentioned motion of rotation and return is repeated one more s, there are produced pulse signals of binary data of OOOOOl(tl'), 000100(t2'), 00000 l(t3') and 000100(t4') as shown in Fig. 4 and hexa-codes of 01 , 04 , 01 and 04 . HEX HEX HEX HEX Further, when the hand-phone of Fig. 2 rotates around Y-axis in the counterclockwise direction by 90 degrees and returns to its original position and then rotates around Y- axis in the clockwise direction by 90 degrees and returns to its original position, there are pulse signals of binary data of OOOOOl(tl'), 010000(t2'), 00000 l(t3') and 100000( t4') as shown in Fig. 5 and hexa-codes of 01 , 10 , 01 and 20 HEX HEX HEX HEX
[40] Motion examples of hand for producing hexa-codes corresponding to key data and command data effective for the ubiquitous gesture ccgnitive device, in relation with Table 1 and Table 2, are explained in Table 3, wherein the hand-held device is grasped in the hand and serves as the ubiquitous gesture ccgnitive device under a specific condition.
[41] [Table 3]
[42]
[43] Therefore, if the hand-phone is rotated around Y axis in the counterclockwise direction by 360 ° , the hexa-code generator 13 produces a hexa code of 01 10 02 20 and the data decision part 14 access the data-type file of the database 15 as shown in Table 1 and checks whether there is a hexa-code identical with the received hexa-code and the identical hexa-code is mapped to the key data of Table 1. If so, the signal indicative of the operation commencement of the ubiquitous gesture ccgnitive device is transmitted to the controller 16.
[44] Thereafter, the user rotates the hand-phone around X axis in the counterclockwise by 90°, returns the same to initial state, again rotates the hand-phone around X axis in the counterclockwise by 90°and then returns the same to initial state. Following the above-mentioned hand motion, the user rotates the hand-phone around Y axis in the clockwise direction by 90°, returns it to initial state, then rotates around Y axis in the clockwise direction by 90 ° and returns it to initial state. After some time passes, the user rotates the hand-phone around Y-axis in the clockwise direction by 360 °.
[45] Then the hexa-code generator 13 successively generates hexa-codes of 01 04 01 04', 01 10 01 20 and 01 20 02 10.
[46] The data decision part 14 sequentially receives the hexa-codes from the hexa-code generator 13 and the controller 16 drives the command signal generating and transmitting part 18 so as to produce a control signal for carrying out command corresponding to the command key mapped to the produced hexa-code and then to transmit the same to the ubiquitous-networked computing devices.
[47] Therefore, in the present embodiment, by the above-mentioned motion of hand in which the hand-phone serving as the ubiquitous gesture cognitive device is grasped, the power of device A is turned on, the door of the device A is open and then after the time passed the power of the device A is turned off.
[48] However, if there is no following motion of the hand or there is no hexa-code corresponding to the command data contained in the database 15 during the operation of the ubiquitous gesture ccgnitive device, the controller 16 informs the user of the operation error of the ubiquitous gesture cognitive device or initialization of the hand- phone serving as the ubiquitous gesture cognitive device via the warning signal indicator 19. In addition, the warning signal information can be transmitted to user by using the vibration mode of the hand-phone.
[49] As described-above, the accelerometer module for detecting the hand motion pattern is installed in the hand-held device so that the hand-held device can be effectively used as a device for transmitting control signal for controlling various ubiquitous networked-electronic devices, that is ubiquitous gesture cognitive device. Accordingly, user could communicate with the ubiquitous networked-electronic devices by using the hand-held device without button handling of the hand-held device
[50] While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.