WO2016209039A1

WO2016209039A1 - Method, system and non-temporary computer-readable recording medium for supporting communication

Info

Publication number: WO2016209039A1
Application number: PCT/KR2016/006796
Authority: WO
Inventors: 김석중; 김정훈
Original assignee: 주식회사 브이터치
Priority date: 2015-06-24
Filing date: 2016-06-24
Publication date: 2016-12-29
Also published as: KR20170000809A; KR101879349B1

Abstract

A method for supporting communication, according to an aspect of the present invention, comprises the steps of: obtaining information about a user's posture and information about voice provided by the user; and, in reference to the obtained information, determining a field of communication to which the voice is to be transmitted among the surrounding fields in which exist are one or more objects that can interact with the user.

Description

Methods, systems, and non-transitory computer readable recording media for supporting communication

The present invention relates to a method, system and non-transitory computer readable recording medium for supporting communication.

Recently, a technique for supporting a user in performing a voice-based communication with another user in a real space or a virtual reality space or performing a voice-based control on a device, a virtual object, or the like has been introduced.

In recent years, as interest in Augmented Reality (AR) or Virtual Reality (VR) has increased and research and development in related technical fields have been actively progressed, users interact in real or virtual reality. There is an increasing demand for a technology that enables the user to communicate with objects as smoothly as possible.

Conventionally, by allowing a user or a system to transmit a voice spoken by a user only to a target object previously designated by the user or the system, or by allowing the user to speak the name of the target object separately or by explicitly instructing the target object (for example, "Turn off the lights in the living room," "Reduce the air conditioning temperature of room 301", etc.) has been introduced to the target object to which the user's voice is to be specified.

1 is a diagram illustrating a communication support method according to the prior art by way of example. According to the prior art shown in FIG. 1, the user moves the pointer 11 to indicate the target object 10 so as to specify the target object 10 as a communication target (see FIG. 1 (a)), The user can specify the target object 10 as a communication target by aligning his gaze with the target object 10 (see (b) of FIG. 1).

However, according to the prior art as described above, it becomes very difficult for a user to flexibly or adaptively determine the range of a target object to which his voice is to be delivered, or a target to which his voice is delivered whenever the user communicates. Since it is inevitably experienced to be very specific, the inconvenience of communication has a limit in communication flexibility or user convenience.

Accordingly, the present inventors determine a region (aka, a field of communication) to which the user's voice is transmitted by referring to the posture of the user and the strength of the voice spoken by the user, whereby the user's voice is intended by the user. We propose a technique that can be delivered accurately to a domain or object.

The object of the present invention is to solve all the above-mentioned problems.

In addition, the present invention obtains information about the user's posture and information about the voice spoken by the user, and the surrounding area in which at least one object that can interact with the user exists with reference to the obtained information. Method, system, and non-transitory computer readable recording that determines the field of communication to which the voice of the intermediate user is to be delivered, so that the user's voice can be accurately delivered to the intended area or object of the user. It is another object to provide the medium.

Representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, a method for supporting communication, comprising: obtaining information about a user's posture and information about a voice spoken by the user, and the obtained information with reference to And determining a field of communication to which the voice is to be transmitted among the surrounding areas in which at least one object that can interact with the user exists.

According to another aspect of the present invention, there is provided a system for supporting communication, comprising: an information obtaining unit for obtaining information about a user's posture and information about a voice spoken by the user, and the obtained information; As a result, there is provided a system including a communication area determiner for determining a field of communication to which the voice is to be transmitted from among the surrounding areas in which at least one object capable of interacting with the user exists.

In addition, there is further provided a non-transitory computer readable recording medium for recording another method, system, and computer program for executing the method for implementing the present invention.

According to the present invention, a user's voice is accurately transmitted to an area or object intended by the user based on a field of communication determined based on a direction in which the user is headed and the strength of the voice spoken by the user. As a result, the user can communicate naturally with the intended object without the inconvenience of specifying the target of communication separately in the actual use environment.

In addition, according to the present invention, by further utilizing the context information such as the filter set by the user, the user's gesture, the user's voice, the context information about the user and the like to meet the user's intention among the plurality of objects existing in the communication area. Since the object can be specified specifically, the effect that communication between the user and the objects around the user can be made more precisely and simply is achieved.

1 is a diagram illustrating a communication support method according to the prior art by way of example.

2 is a diagram illustrating an internal configuration of a communication support system according to an embodiment of the present invention.

3 to 5 are diagrams exemplarily illustrating a configuration for determining a communication area according to an embodiment of the present invention.

6 and 8 illustrate various embodiments in which a configuration for determining a communication area may be utilized according to an embodiment of the present invention.

9 to 12 are diagrams exemplarily illustrating a configuration for specifying a target object among objects existing in a communication area according to an embodiment of the present invention.

100: user device

200: communication support system

210: information acquisition unit

220: communication area determination unit

230: target specific part

240: database

250: communication unit

260: control unit

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

Composition of communication support system

Hereinafter, the internal structure of the communication support system 200 performing important functions for the implementation of the present invention and the function of each component will be described.

2 is a diagram illustrating an internal configuration of a communication support system according to an embodiment of the present invention by way of example.

Referring to FIG. 2, the communication support system 200 according to an embodiment of the present invention may include an information acquisition unit 210, a communication area determination unit 220, a target specifying unit 230, and a database 240. It may include a communication unit 250 and a control unit 260. According to an embodiment of the present invention, the information acquisition unit 210, the communication area determination unit 220, the target specifying unit 230, the database 240, the communication unit 250 and the control unit 260 at least Some may be program modules in communication with an external system (not shown). Such program modules may be included in the communication support system 200 in the form of operating systems, application modules, and other program modules, and may be physically stored on various known storage devices. In addition, these program modules may be stored in a remote storage device that can communicate with the communication support system 200. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

On the other hand, the communication support system 200 has been described as above, but this description is exemplary, and at least some of the components or functions of the communication support system 200 are carried by the user as needed or the body part of the user. It will be apparent to those skilled in the art that the device may be implemented within the user device 100 or worn within the user device 100 (eg, head, eyes, etc.). In some cases, all the functions and all the components of the communication support system 200 may be executed in the user device 100 or may be included in the user device 100.

According to an embodiment of the present invention, the user device 100 may include a head mounted display (HMD) capable of implementing Augmented Reality (AR) or Virtual Reality (VR); It may be a near eye display, and the user device 100 may include a microphone module (not shown) for recognizing a voice of a user. In addition, according to an embodiment of the present invention, the user device 100 may be a wearable voice recognition device that does not include a display module, and more specifically, a microphone module (not shown) disposed around the mouth or Ear plug-type microphone module (not shown) may be included.

On the other hand, as will be described later, according to an embodiment of the present invention, the user device 100, the information necessary to determine the communication area of the user (that is, information about the posture of the user and the voice spoken by the user) Information) may be included in the sensing module (not shown).

First, according to an embodiment of the present invention, the information acquisition unit 210 may perform a function of obtaining information about a posture of a user. Specifically, the information acquisition unit 210 according to an embodiment of the present invention, may obtain information about the posture of the body part of the user related to communication through voice, for example, the head of the user or Information about the direction in which the mouth is facing can be obtained.

To this end, according to an embodiment of the present invention, the user device 100 possessed or worn by the user, or an external device (not shown) that can observe the user's posture separately from the user, may be a specific body of the user. It may include at least one sensing module (not shown) that can obtain physical information about the position or movement of the site. Examples of such sensing modules include known components such as motion sensors, acceleration sensors, gyroscopes, magnetic sensors, positioning modules (GPS modules, beacon-based positioning (identification) modules, etc.), barometers, cameras, and the like. Can be.

In addition, according to an embodiment of the present invention, the user device 100 possessed or worn by the user may acquire physical information about the posture or movement of the user based on biometric information obtained from the human body of the user. It may include at least one sensing module. As an example of such a sensing module, the EMG signal measuring apparatus etc. are mentioned.

In addition, according to an embodiment of the present invention, the information acquisition unit 210 may perform a function of obtaining information about a voice spoken by a user. More specifically, the information acquisition unit 210 according to an embodiment of the present invention may obtain information about the strength of the voice spoken by the user, the voice recognition result of the voice spoken by the user, and the like.

Next, according to an embodiment of the present invention, the communication area determiner 220 may interact with the user with reference to the information about the posture of the user and the information about the voice spoken by the user. The user may perform a function of determining a communication area to which the user's voice is to be transmitted among the surrounding areas in which at least one object exists. More specifically, the communication area determiner 220 according to an embodiment of the present invention may determine the communication area with reference to the direction of the head or mouth of the user and the intensity of the voice spoken by the user. have.

In addition, according to an embodiment of the present disclosure, the communication area determiner 220 may perform at least some of the attributes of the communication area by further referring to the setting information input from the user.

First, referring to FIG. 3, the communication area 300 according to an embodiment of the present invention may be determined as a region having a quadrangular pyramid (ie, pyramid) shape having a vertex as a reference point of a user's mouth. More specifically, according to an embodiment of the present invention, the height (or depth) direction 310 of the communication region 300 having a quadrangular pyramid shape may be determined according to the direction of the head or mouth of the user. As a result, the communication area 300 may face the direction intended by the user. In addition, according to an embodiment of the present invention, the height 320 of the communication region 300 having a quadrangular pyramid shape may be determined according to the intensity of the voice spoken by the user. ) Can be made smaller (ie, the communication area 300 is confined only close to the user), while if the user speaks louder, the height of the communication area 300 can be made larger (ie, the communication area). 300 extends farther away from the user).

3, the horizontal and vertical lengths of the bottom surface of the communication area 300 having a quadrangular pyramid shape may be determined according to a value arbitrarily set by a user. For example, a user who wants to broaden the range of communication can set the above arbitrary setting value to widen the communication area 300, and a user who want to narrow the range of communication can set the above arbitrary setting. By setting the value small, the communication area 300 can be narrowed.

Next, referring to FIG. 4, the communication area 400 according to an embodiment of the present invention may be determined as an area having a cone shape having a vertex as a reference point of a user's mouth. More specifically, according to an embodiment of the present invention, the height direction 410 and the height 420 of the communication area 400 are respectively the direction that the head part or mouth part of the user looks and the voice spoken by the user It can be determined based on the strength of. In addition, according to an embodiment of the present invention, the

widths

430 and 440 of the bottom surface of the communication area 400 having a conical shape may be determined according to a value arbitrarily set by a user.

Next, referring to FIG. 5, the communication area 500 according to an embodiment of the present invention may be determined as an area having an ellipsoid shape focusing on a reference point of a user's mouth. More specifically, according to an embodiment of the present invention, the long axis direction 510 and the long axis length 520 of the communication area 500 having an ellipsoid shape are respectively seen by the head part or mouth part of the user. It may be determined based on the direction and the intensity of the voice spoken by the user. Further, according to an embodiment of the present invention, the ratio between the length 520 of the long axis, the length 530 of the short axis, and the length 550 of the width of the communication area 500 having an ellipsoid shape (ie, 520: 530: 550 or the length 520 of the major axis, the length 530 of the minor axis, the length 540 of the height and the length 550 of the width (ie, 520: 530: 540: 550) It can be determined according to the value set arbitrarily.

However, the communication area determination method according to the present invention is not necessarily limited to those listed above, and it can be understood that the present invention can be changed as much as possible within the scope of achieving the object of the present invention. For another example, some attributes of the communication area (ie, the width or length of the base, the width of the base, the ratio between the width and the length) of the communication area that were determined according to the values arbitrarily set by the user in the embodiments of FIGS. Etc.) may include various situations such as information about a filter preset by the user, information about a gesture input from the user, information about the user's intention derived from the voice recognition result of the user's voice, and context information about the user. It may be determined adaptively (ie, automatically) based on the information.

First, referring to FIG. 6, a user 610 is included in a communication area 600 that is determined based on a direction that a user views among a plurality of other users who are present and the intensity of a voice spoken by the user. Two

different users

620 and 630 can forward their voice messages.

Next, referring to FIG. 7, each of the plurality of

users

711, 721, 731, 741, 751 is a communication area 712 that is determined based on the direction in which the user is looking and the intensity of the voice spoken by the user. , 722, 723, 724, and 725 may transmit their voice commands to the

devices

731, 732, 733, 734, and 735, respectively.

Next, referring to FIG. 8, in the virtual reality VR provided through the head mounted display device 100, a communication area based on the direction that the user 810 is looking at and the intensity of the voice spoken by the user. 830 may be determined, and the user 810 may transmit his or her voice command to the virtual object included in the communication area 830.

Next, according to an embodiment of the present invention, the target specifying unit 220 is derived from information on a filter preset by the user, information on a gesture input from the user, and a voice recognition result of the user's voice. A function of specifying at least one target object to which the user's voice is to be transmitted from among objects existing in the communication area with reference to various contextual information including information about the intention of the user and contextual information about the user. Can be done.

Here, according to an embodiment of the present invention, the target object is implemented by a virtual object or a real object such as another user, a real device capable of receiving a voice message or a voice command from the user, and a virtual reality or augmented reality from the user. A virtual object such as an avatar or a virtual device capable of receiving a voice message or a voice command may be included.

Specifically, according to an embodiment of the present invention, the filter preset by the user may be set based on various conditions that must be satisfied in order to become a target object.

For example, in the embodiment of FIG. 9, the setting is based on the relationship between the user and the object (family, friends, etc.), the language used by the user or the object, the age of the user or the object, the gender of the user or the object, and the like. Based on the filter, the object 940 that meets the criteria of the above filter among the plurality of objects (ie, other users) 920 to 950 existing in the communication area 900 of the user 910 is the user. It may be determined to be a target object to receive the voice message from 910.

In addition, according to an embodiment of the present invention, the information about the user's intention derived from the voice recognition result of the user's voice, the keyword (for example, the name of the object, the command to be delivered to the object) included in the voice Information on the type, keyword, location of the object, etc.), whether or not to use the idiom, and the user's identity recognized from the voice.

For example, in the embodiment of FIG. 10, the communication area 1100 of the user 1110 when the user 1110 speaks directly to the location (“Top right”) of the object that he / she wants to specify. It may be determined that the object 1150 existing at the location directly spoken by the user 1110 among the plurality of objects 1120 to 1150 existing in the Cs is a target object to receive a voice message from the user 1110.

In another example, when a user calls an object's name, it may be determined that the object corresponding to that name is the target object, and that another user who is older or higher than the user when the user uses the idiom It may be determined to be a target object.

In addition, according to an embodiment of the present invention, the gesture input from the user may include various operations that the user may perform to communicate with a real object or a virtual object that exists in the vicinity of the user, and the gesture may move It can be recognized by various sensing modules such as sensors, acceleration sensors, gyroscopes, magnetic sensors, positioning modules (GPS modules, beacon-based positioning (identification) modules, etc.), barometers, and cameras.

For example, in the embodiment of FIG. 11, when the user 1210 makes a gesture that looks at an object that he or she wishes to specify, the plurality of objects 1220 that exist in the communication area 1200 of the user 1210. In operation 1230, the object 1220 facing the user 1210 may be determined to be a target object to receive a voice message from the user 1210.

Further, according to an embodiment of the present invention, the contextual information about the user includes demographic information of the user, information on the behavior pattern of the user, information on the place or time zone where the user uttered the voice, and the user's voice. Information about the voice spoken before the speech, information about the user's preferences for the object, information about the user's activities on social network services (SNS), information about the conversations the user has had with other users, and notes that the user has recorded Information about a social relationship or ownership relationship between a user and an object.

For example, in the embodiment of FIG. 12, when a user 1310 showing a sleep pattern every night at 12 o'clock utters a voice command “Turn off” at 11:30 one night In some of the home appliances 1320 to 1350 in the communication area 1300 of the user 1310, some of the

home appliances

1320 and 1330, which may interfere with sleep, may receive the above voice command. It can be determined to be an object.

In another example, when a user utters a voice command "unlocked", a smartphone owned by the user among the various smartphones existing in the communication area of the user will receive the voice command. Can be determined to be.

On the other hand, according to an embodiment of the present invention, the database 240, various information that can be used in determining the communication area according to the present invention and various that can be used in specifying the target object according to the present invention Information can be stored. Although the database 240 is illustrated as being included in the communication support system 200 in FIG. 2, according to the needs of those skilled in the art of implementing the present invention, the database 240 may be connected to the communication support system 200. It may be configured separately. On the other hand, the database 240 in the present invention is a concept that includes a computer-readable recording medium, and may be a broad database including not only a negotiated database but also a data record based on a file system. If the set can be retrieved to extract data, it can be the database 240 in the present invention.

Next, the communication unit 250 according to an embodiment of the present invention performs a function to enable the communication support system 200 to communicate with an external device.

Finally, the control unit 260 according to an embodiment of the present invention is the data between the information acquisition unit 210, the communication area determination unit 220, the target specifying unit 230, the database 240 and the communication unit 250 To control the flow of the. That is, the controller 260 controls the flow of data from the outside or between the respective components of the communication support system 200, whereby the information obtaining unit 210, the communication area determining unit 220, and the object specifying unit ( 230, the database 240 and the communication unit 250 control to perform a unique function, respectively.

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded on a non-transitory computer readable recording medium. The non-transitory computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the non-transitory computer readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of non-transitory computer readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, magnetic-optical media such as floppy disks ( magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the like, but the embodiments and the drawings are provided to assist in a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the appended claims, fall within the scope of the spirit of the present invention. I will say.

Claims

As a way to support communication,

Obtaining information about a posture of a user and information about a voice spoken by the user, and

Determining a field of communication to which the voice is to be transmitted from among surrounding areas in which at least one object capable of interacting with the user exists with reference to the obtained information;

How to include.
The method of claim 1,

In the communication area determination step,

And determining the communication area based on the direction in which the head or mouth of the user is facing and the intensity of the voice spoken by the user.
The method of claim 1,

And determining at least some of the attributes of the communication area with further reference to setting information input from the user.
The method of claim 1,

Determining at least one target object to which the voice is to be transmitted among objects existing in the communication area with reference to context information;

How to include more.
The method of claim 4, wherein

The contextual information may include information about a filter preset by the user, information about a gesture input from the user, information about the intention of the user derived from a voice recognition result for the voice, and context information about the user. At least one of which is included.
A non-transitory computer readable recording medium having recorded thereon a computer program for executing the method according to claim 1.
As a system to support communication,

An information obtaining unit for obtaining information about a posture of a user and information about a voice spoken by the user, and

A communication area determiner for determining a field of communication to which the voice is to be transmitted from among surrounding areas in which at least one object capable of interacting with the user exists with reference to the obtained information.

System comprising.