WO2008156263A2

WO2008156263A2 - Method for recognizing sensibility using musical elements.

Info

Publication number: WO2008156263A2
Application number: PCT/KR2008/003319
Authority: WO
Inventors: Gil Ho Kim
Original assignee: Gil Ho Kim
Priority date: 2007-06-16
Filing date: 2008-06-13
Publication date: 2008-12-24
Also published as: WO2008156263A3

Abstract

Disclosed is a method of recognizing sensibility, and more particularly to a method of recognizing a speaker's sensibility status and sensibility statuses of noise and music around a speaker by using musical elements of harmony and counterpoint, such as chord information, beat per minute information, volume information, and overtone and register information. The sensibility recognition method using musical elements, the method including: previously establishing a mapping table obtained by mapping sensibility status information regarding a speaker and musical element information invoking the sensibility status information and recording the mapping table; detecting musical element information and harmonic disharmony information from input voice signals; reading and computing the sensibility status information corresponding to the detected musical element information from a previously established mapping table, and computing a harmonic disharmony index from the harmonic disharmony information; and visually/audibly displaying the computed sensibility status information and harmonic disharmony index. The present invention can increase a rate of recognizing speaker's sensibility, recognize a sensibility status of noise and music around a speaker, provide the speaker with sensibility status information, and provide the speaker with elements favorable for a speaker's sensibility status in response to the sensibility status information, thereby providing a bi-directional effect of improving the speaker's sensibility status.

Description

[DESCRIPTION]

[Invention Title]

METHOD FOR RECOGNIZING SENSIBILITY USING MUSICAL ELEMENTS

[Technical Field]

The present invention relates to a method of recognizing sensibility, and more particularly to a method of recognizing a speaker's sensibility status by using musical elements of the laws of harmony and counterpoint, recognizing a sensibility status of noise and music around a speaker, and providing the speaker with the sensibility status information.

[Background Art] In general, a sensibility recognizing method is not very reliable due to a great personal deviation, such as a speaker's dialecticism, age, sex, degree of obesity, temporal period, personality, sense of values, or the like.

To address the above problems, methods of increasing a rate of recognizing a speaker's sensibility by recognizing the speaker through self- learning have been suggested. However, such methods require a long period of time for self-learning to occur and have their limits in terms of learning efficiency.

[Disclosure] [Technical Problem]

The present invention has been made in view of the above-mentioned problems, and the present invention provides a method of recognizing a speaker's sensibility status by using musical elements related to the laws of harmony and counterpoint.

Also, the present invention provides a method of taking statistics on a plurality of pieces of mapping information renewed based on speaker information, such as a speaker's dialecticism, age, sex, degree of obesity, temporal period, or the like, and renewing mapping table information based on the mapping information statistics, thereby increasing a rate of recognizing speaker's sensibility.

Also, the present invention provides a method of recognizing a sensibility status of noise and music around a speaker, providing the speaker with sensibility status information, and providing the speaker with elements favorable for a speaker's sensibility status in response to the sensibility status information.

[Technical Solution] In accordance with an aspect of the present invention, there is provided a sensibility recognition method using musical elements, the method comprising: previously establishing a mapping table obtained by mapping sensibility status information regarding a speaker and musical element information invoking the sensibility status information and recording the mapping table; detecting musical element information and harmonic disharmony information from input voice signals; reading and computing the sensibility status information corresponding to the detected musical element information from a previously established mapping table, and computing a harmonic disharmony index from the harmonic disharmony information; and visually/audibly displaying the computed sensibility status information and harmonic disharmony index.

The detecting of the musical element information and the harmonic disharmony information from input voice signals may comprise: identifying and separating an externally input voice signal into a noise signal, a music signal, and a speaker voice signal; and comparing the musical element information stored in the mapping table based on the separated noise signal, music signal, and speaker voice signal, detecting information that is consistent for more than an established rate, and giving a weight to harmony information detected based on the consistency degree and frequency of the detected information.

The comparing of the musical element information may comprise: detecting each radical of the separated noise signal, the music signal, and the speaker voice signal according to time, comparing pitch information of at least two of the detected adjacent radicals with chord codes stored in the mapping table, and detecting chord information that is consistent for more than a predetermined rate; giving a weight to harmony information detected based on the consistency degree and frequency of the detected chord information within an established temporal section, and detecting the harmonic disharmony information that is not consistent with the chord information; comparing beat per minute values stored in the mapping table based on temporal variation of each beat and radical of the separated noise signal, music signal, and speaker voice signal, and detecting beat per minute information that is consistent for more than the predetermined rate; giving a weight to beat per minute information detected based on the consistency degree and frequency of the detected beat per minute information within an established temporal section; detecting each volume of the separated noise signal, music signal, and speaker voice signal in real-time, comparing volume values stored in the mapping table with the detected volumes of the separated noise signal, music signal, and speaker voice signal, and detecting volume information that is consistent for more than the predetermined rate; giving a weight to volume information detected based on the consistency degree and frequency of the detected volume information within the established temporal section; detecting each overtone structure and register range values of the separated noise signal, music signal, and speaker voice signal in real-time, comparing overtone structure and register range values stored in the mapping table with the detected overtone structure and register range values, and detecting overtone structure and register range information that is consistent for more than the predetermined rate; and giving a weight to overtone structure and register range information detected based on the consistency degree and frequency of the detected overtone structure and register range information within the established temporal section.

The method may further comprise: if the established temporal section has passed, detecting period information from the variation of each chord, beat per minute, volume, and overtone structure and register range information; and renewing mapping information stored in the mapping table based on the speaker and period information.

The method may further comprise^; if there is no speaker voice signal, predicting the speaker's sensibility status based on the detected period information; and selectively providing the speaker with at least one of established noise, automatic noise, established music, and automatic music including the musical element information corresponding to the speaker's sensibility status based on the predicted speaker's sensibility status.

The reading and computing of the sensibility status information may comprise: detecting the sensibility status information corresponding to the detected musical element information from the mapping table based on the musical element information; summing weights of the detected sensibility status information excluding the sensibility status in a section opposite to the sensibility status corresponding to a maximum weight and computing the sensibility status information; and calculating the harmonic disharmony index at a ratio of a time of occurrence of disharmony of the detected harmonic disharmony information with regard to the total time required to input the voice signals. The visually/audibly displaying of the computed sensibility status information and the harmonic disharmony index may comprise : reading noise information and music information including the computed sensibility status information from the mapping table; providing the speaker with established noise and established music corresponding to the read noise information and music information in real-time; and playing automatic noise and automatic music including the computed sensibility status information and the harmonic disharmony information based on the computed sensibility status information and the harmonic disharmony information, and providing the speaker with the automatic noise and automatic music.

The method may further comprise: renewing the visually/audibly displayed sensibility status information by the speaker.

[Advantageous Effects] The present invention renews mapping information based on speaker information, such as a speaker's dialecticism, age, sex, degree of obesity, temporal period, or the like, thereby increasing a rate of recognizing speaker's sensibility.

In addition, the present invention recognizes a sensibility status of noise and music around a speaker, provides the speaker with sensibility status information, and provides the speaker with elements favorable for a speaker's sensibility status in response to the sensibility status information, thereby providing a bi-directional effect of improving the speaker's sensibility status.

[Description of Drawings]

The foregoing and other objects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which: FIG. 1 is a flowchart illustrating the operation of a method of the present invention;

FIG. 2 illustrates a configuration according to an embodiment of the present invention; FIG. 3 illustrates a process of detecting chord information according to an embodiment of the present invention;

FIG. 4 is a table obtained by mapping musical elements to sensibility statuses according to an embodiment of the present invention;

FIG. 5 illustrates that the arrangement characteristics of sensibility statuses are classified according to an embodiment of the present invention;

FIG. 6 illustrates a sensibility status classification based on the arrangement characteristics of FIG. 5 according to an embodiment of the present invention;

FIG. 7 illustrates chord mapping information obtained by mapping chord codes to the sensibility statuses of FIG. 6 according to an embodiment of the present invention;

FIG. 8 illustrates sensibility statuses according to the Ekman classification method based on the sensibility status classification shown in FIG.

6; FIG. 9 illustrates a sensibility status classification according to the Ekman classification method based on the arrangement characteristics of FIG. 5;

FIG. 10 illustrates beat per minute mapping information obtained by mapping beat per minute values to the sensibility statuses of FIG. 6;

FIG. 11 illustrates volume mapping information obtained by mapping volume values to the sensibility statuses of FIG. 6;

FIG. 12 illustrates arrangement characteristics according to an increase and decrease of the beat per minute values of FIG. 10 and the volume values of FIG. 11 compared to the sensibility status classification of FIG. 6; and FIG. 13 illustrates overtone and register mapping information obtained by mapping overtone structure and register range values to the sensibility statuses of FIG. 6.

[Best Mode]

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

[Mode for Invention] Harmony is the use of different pitches simultaneously, and chords, actual or implied, in music. Counterpoint is the relationship between two or more voices that are independent in contour and rhythm.

Harmony and counterpoint are indispensable to musical composition.

Composers have generally used a specific musical component value corresponding to a person's specific sensibility in order to evoke the person's specific sensibility. The specific musical component value has been proved for a long time, as there is a statistical rule causing the person's specific sensibility.

Musical elements include a chord (a harmonic structure of a keynote during a specific temporal section and a harmonic structure of overtones during a specific temporal section), a melody (a temporal variation of a keynote), a beat per minute (beat), volume (a sound volume), or the like.

In more detail, a musical element having a specific value evokes a person's specific sensibility, and inversely, a speaker's voice in a specific sensibilty status has the musical element having the specifica value. In an opera artificially expressing a speaker's sensibility status by music, and a daily conversational pattern of a speaker, the speaker's voice generates regular characteristics of a musical element corresponding to the speaker's sensibility status. The present invention relates to a method of reading specific values of musical elements mapped to speaker's voice information from the speaker's voice information, and computing the read specific values of the musical elements, thereby detecting sensibility status information regarding a speaker. A method of classifying a speaker's sensibilty status according to the present invention will now be described before describing the operation of the method of the present invention.

Although a person's emotion is classified according to various methods, the Ekman classification method, that is, a conventional representative method, classifies the person's emotion into six emotional statuses of happiness, sadness, disgust, surprise, anger, and fear. Although these six emotions are common to persons, only happiness is a positive emotion, whereas the others are negative emotions, and thus such classification is unequal.

Therefore, since the positive emotion happiness is very comprehensive, the present invention classifies happiness into concord, joy, love, comfort, easiness, and self-confidence, and disgust and anger are substituted for indifference and tension, respectively, that are more comprehensive than disgust and anger.

Meanwhile, since persons and the universe have ambilaterality, the opposites of the above emotions are: concord <-> solitude, joy <-> sorrow, love <-> indifference, comfort <-> surprise, easiness <-> tension, and self-confidence <-> fear

As described above, the present invention redefines the six basic emotions to twelve emotions to comprehend person's various emotions. The twelve emotions are referred to as sensibility. The above twelve sensibilities are intended to depict an exemplary embodiment and should not be interpreted to limit the embodiments set forth herein. Therefore, it will be understood by one of ordinary skill in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

The twelve sensibility statuses are arranged in a ring shape as shown in FIG. 5 that illustrates that the arrangement characteristics of sensibility statuses are classified according to an embodiment of the present invention. In more detail, emotions of a static and cold status and a dynamic and warm status are arranged in a vertical axis of the ring, and emotions of a bright and positive status and a dark and negative status are arranged in a horizontal axis of the ring. Also, tension is placed in the most dynamic sensibility status, easiness is placed in the most static sensibility status, concord is placed in the most bright sensibility status, and solitude is placed in the most dark sensibility status. In view of the placement of the four sensibility statuses, the other eight sensibility statuses are arranged. In more detail, joy, which is more dynamic than concord, the brightest sensibility status, and self-confidence, which is more static than concord, are placed adjacent to concord. Fear, which is more dynamic than solitude, the darkest sensibility status, and sorrow, which is more static than solitude, are placed adjacent to solitude. Love, which is brighter than tension, the most dynamic sensibility status, and surprise, which is darker than tension, are placed adjacent to tension. Comfort, which is brighter than easiness, the most static sensibility status, and indifference, which is darker than easiness, are placed adjacent to easiness.

As a result, the opposite sensibility statuses face each other in view of a center point of the ring.

Such arrangement of the sensibility statuses excludes an opposite sensibility status in estimating a main sensibility status, in order to increase the reliability of a sensibility recognition rate. In addition, as shown in FIG. 6, twelve sensibility statuses belonging to the sub-range of the basic twelve sensibility statuses are arranged three times in the ring by using the above method of arranging the sensibility statuses, so that thirty-six sensibility statuses are additionally arranged such that the opposite sensibility statuses face each other in view of the center point of the ring (see FIG. 6).

FIG. 7 illustrates chord mapping information obtained by mapping chord codes to the sensibility statuses of FIG. 6 according to an embodiment of the present invention. Referring to FIG. 7, the chord mapping information is an example of chord codes that are mainly used by composers in order to invoke person's sensibility statuses. Although the chord codes corresponding to the sensibility statuses may be different according to each composer, the chord codes are generally consistent according to composers.

FIG. 8 illustrates sensibility statuses according to the Ekman classification method based on the sensibility status classification shown in FIG. 6. Referring to FIG. 8, the sensibility statuses are each arranged in a direction.

FIG. 9 illustrates a sensibility status classification according to the Ekman classification method based on the arrangement characteristics of FIG. 5. The sensibility status classification is another embodiment of that shown in FIG. 6. That is, another embodiment of chord mapping information can be obtained by mapping the chord codes shown in FIG. 7 to the sensibility statuses of the present embodiment.

FIGS. 10, 11, and 13 illustrate beat per minute mapping information, volume mapping information, and overtone and register mapping information obtained by mapping beat per minute values, volume values, and an overtone structure and register range values, respectively, to the sensibility statuses of

FIG. 6. The tempo, volume, and overtone and register mapping information include values of musical elements that are mainly used to evoke a person's specific sensibility status.

Meanwhile, FIG. 12 illustrates arrangement characteristics according to an increase and decrease of the beat per minute values of FIG. 10 and the volume values of FIG. 11 compared to the sensibility status classification of FIG. 6. The increase and decrease of the beat per minute and volume of sound are arranged in the left and right of the ring so that a maximum value is mapped to the sensibility status, tension, and a minimum value is mapped to the sensibility status, easiness. Thereafter, the operation of the method of the present invention will now be described.

FIG. 1 is a flowchart illustrating the operation of the method of the present invention. FIG. 2 illustrates a configuration according to an embodiment of the present invention. FIG. 3 illustrates a process of detecting chord information according to an embodiment of the present invention. FIG. 4 is a table obtained by mapping musical elements to sensibility statuses according to an embodiment of the present invention.

Referring to FIG. 2, a plurality of local terminals 220 that are locally distributed is wire-or wireless-connected to a server 200. A mapping table DB server 210 that stores data is connected to the server 200. The local terminals 220 detect sensibility status information regarding the speaker, sensibility status information regarding noise around a speaker, and status information regarding music around the speaker from externally input voice signals, compute the above sensibility status information and a disharmony index, provide the speaker with the computed sensibility status information, and transmit mapping information renewed by the speaker and speaker information to the server 200. The server 200 stores speaker information that is input from each local terminal 220 and the renewed mapping information based on the received speaker information to a database, takes statistics on the database, renews and stores the statistics in the mapping table DB server 210 (SlOO).

The local terminals 220 receive and store a plurality of pieces of speaker information from the outside in order to recognize the speaker. The local terminals 220 identify and separate the externally input voice signals into a noise signal, a music signal, and a speaker voice signal (SI lO). The voice signal separating method is well-known to one skilled in the art and thus the detailed description thereof is not repeated herein.

Meanwhile, the local terminals 220 detect musical element information (chord information, beat per minute information, volume information, and overtone structure/register rang information) and harmonic disharmony information from the noise signal, the music signal, and the speaker voice signal

(S120).

If the speaker voice signal separated from the externally input voice signals is input from a plurality of speakers, specific speaker information is read from a plurality of previously input speaker information based on the input speaker voice signal to recognize the speaker.

The local terminals 220 transmit the read speaker information to the server 200 and requests the server 200 to send mapping information. The server 200 reads mapping information corresponding to the received speaker information from the mapping table DB server 210 and transmits the read mapping information to the local terminals 220.

The local terminals 220 compute sensibility status information corresponding to the detected musical element information based on the mapping information received from the server 200, and computes a harmonic disharmony index corresponding to the harmonic disharmony information (S130).

Steps S 120 and S 130 will be described in detail later. The local terminals 220 visually/audibly provide the speaker with the computed sensibility status information and harmonic disharmony index that is selectively corrected by the speaker (S 140). When the speaker corrects the computed sensibility status information, the local terminals 220 transmit corrected mapping information corresponding to the corrected sensibility status information and the speaker information to the server 200. The server 200 takes statistics on the corrected mapping information received from the local terminals 220 based on the speaker information, stores the statistics to the database, renews and stores the database in the mapping table DB server 210 (SlOO).

The local terminals 220 guide the speaker's sensibility status in a favorable way or compute the musical element information suitable for the speaker's sensibility status, and transmit the computed musical element information to the server 200. The server 200 reads the noise information and the musical information including the musical element information received from the local terminals 220 from the mapping table DB server 210, and transmits the read noise information and the musical information to the local terminals 220. The local terminals 220 provide the speaker with established noise and established music corresponding to noise information and music information, respectively, received from the server 200 in real-time. The local terminals 220 play automatic noise and automatic music including the computed musical element information and provides the speaker with the automatic noise and automatic music in real-time. In more detail, one of the established noise and music or the automatic noise/music is provided to the speaker according to the speaker's selection (S 150), and the established noise is a recorded sound of nature. The automatic noise is created by a computer, and the established music is created by a musical composer. The automatic music is created by the computer. Although not shown, noise information DB and music information DB corresponding to various kinds of musical element information are mapped and stored in the mapping table DB server 210. The speaker can selectively renew the noise information DB and music information DB.

The local terminals 220 temporarily store temporal variation information of the sensibility status information and the harmonic disharmony index with respect to the noise signal, the music signal, and the speaker voice signal (S160), and determines if a predetermined period of time has passed (S 170). If it is determined that the predetermined period of time has not passed, the local terminals 220 proceed back to step SIlO. Otherwise, if it is determined that the predetermined period of time has passed, the local terminals 220 detect daily, weekly, and monthly period information from the stored temporal variation information (S 180), and transmit the detected period information and the speaker information to the server 200.

The server 200 renews the mapping information stored in the mapping table DB server 210 based on the speaker information and the period information that are input from the local terminals 220 (S190).

If the period information regarding the speaker is generated, the local terminals 220 predict the speaker's sensibility status based on the period information if the speaker voice signal is not received. The local terminals 220 guide the speaker's sensibility status in a favorable way based on the predicted speaker's sensibility information or selectively provide the speaker with musical element information including established noise, automatic noise, established music, and automatic music suitable for the speaker's sensibility status, and proceeds to step S 150. Step S120 will now be described in detail. In step S120, the local terminals 220 detect musical element information (chord information, beat per minute information, volume information, and overtone structure/register rang information) and harmonic disharmony information from the noise signal, the music signal, and the speaker voice signal.

The operation of the local terminals 220 in terms of detecting chord information and disharmony information from the noise signal, the music signal, and the speaker voice signal will now be described. FIG. 7 illustrates chord mapping information obtained by mapping chord codes to the sensibility statuses of FIG. 6 according to an embodiment of the present invention. Referring to FIG. 7, the chord mapping information is an example of chord codes that are mainly used by composers in order to invoke persons sensibility statuses. Although the chord codes corresponding to the sensibility statuses of FIG. 7 may be different according to each composer, the chord codes are generally consistent according to composers. The basic twelve pieces of information are mapped as shown in the mapping table of FIG. 4 among the forty-eight sensibility statuses and chord codes that are inter-mapped as shown in FIGS. 6 and 7. FIG. 4 shows the mapping information table received on the weekend afternoon by a 37-year-old male who speaks a Pusan dialect and is overweight from the server 200 via the local terminals 220. The local terminals 220 detect each radical of the noise signal, the music signal, and the speaker voice signal. The local terminals 220 compare sound pitch information regarding at least two of the detected adjacent radicals with the chord codes stored in the mapping information table, and detect chord information that is consistent for more than a predetermined rate. The local terminals 220 give a weight to the detected chord information based on the consistency degree and frequency of the detected chord information within a predetermined temporal section. The local terminals 220 detect disharmony information that is not consistent with the chord information. In more detail, as shown in FIG. 3, the local terminals 220 detect the chord information, the weight, and disharmony information 310 based on variation information 300 of the adjacent pitches according to time. The operation of the local terminals 220 in terms of detecting beat per minute information from the noise signal, the music signal, and the speaker voice signal will now be described. FIG. 10 illustrates beat per minute mapping information obtained by mapping beat per minute values to the sensibility statuses of FIG. 6. The beat per minute mapping information is mapped as in the mapping table shown in FIG. 4. The beat per minute mapping information is an example of beat per minute values that are mapped to the sensibility statues and are mainly used by composers in order to invoke person's sensibility statuses. The local terminals 220 compare beat per minute values stored in the mapping information table based on the temporal variation of each beat and radical of the noise signal, the music signal, and the speaker voice signal, and detect beat per minute information that is consistent for more than a predetermined rate. The local terminals 220 give a weight to the detected beat per minute information based on the consistency degree and frequency of the detected beat per minute information within a predetermined temporal section.

The operation of the local terminals 220 in terms of detecting volume information from the noise signal, the music signal, and the speaker voice signal will now be described. FIG. 11 illustrates volume mapping information obtained by mapping volume values to the sensibility statuses of FIG. 6. The volume mapping information is mapped as in the mapping table shown in FIG. 4. The volume mapping information is an example of volume values that are mapped to the sensibility statues and are mainly used by composers in order to invoke person's sensibility statuses. The local terminals 220 detect volume of the noise signal, the music signal, and the speaker voice signal in real-time, compare the detected volume with volume values stored in the mapping information table, and detect volume information that is consistent for more than a predetermined rate. The local terminals 220 give a weight to the detected volume information based on the consistency degree and frequency of the detected volume information within a predetermined temporal section.

The operation of the local terminals 220 in terms of detecting overtone structure information from the noise signal, the music signal, and the speaker voice signal will now be described. FIG. 13 illustrates overtone and register mapping information obtained by mapping overtone structure and register range values to the sensibility statuses of FIG. 6. The overtone and register mapping information is mapped as in the mapping table shown in FIG. 4. The local terminals 220 detect each overtone structure and register range values of the noise signal, the music signal, and the speaker voice signal in real-time, compare the detected overtone structure and register range values with overtone structure and register range values stored in the mapping information table, and detect overtone and register range information that is consistent for more than a predetermined rate. The local terminals 220 give a weight to the detected overtone and register range information based on the consistency degree and frequency of the detected overtone and register range information within a predetermined temporal section.

Step S 130 will now be described in more detail. In step S 130, the local terminals 220 detect and compute sensibility status information corresponding to the detected musical element information from the mapping information table received from the server, and compute a harmonic disharmony index from the detected harmonic disharmony information.

The local terminals 220 detect the sensibility status information corresponding to the chord information, the beat per minute information, the volume information, and the overtone and register range information that are detected from the noise signal, the music signal, and the speaker voice signal within a predetermined temporal section, from the mapping information table. The local terminals 220 sum weights, excluding a sensibility status in a section opposite to a sensibility status corresponding to a maximum weight from a computing section among the detected sensibility status information, and compute the sensibility status information.

As an example, weights of the musical element information detected from the speaker voice signal and the sensibility status information corresponding to the weights are assumed as listed below. chord information: self-confidence 40%, concord 20%, tension 20%, solitude 20% beat per minute information: self-confidence/sorrow 40%, concord/solitude 30%, comfort/indifference 30% volume information: concord/solitude 60%, self-confidence/sorrow 20%, joy/fear 20% overtone and register range information: comfort 35%, joy 25%, self- confidence 20%, tension 20% The detected sensibility status information and weights are compared with the sensibility status classification shown in FIG. 6. Since self-confidence detected from the chord information has a maximum weight, the sensibility status information (fear, solitude, surprise, sorrow, and tension) around fear that is opposite to self-confidence is excluded from the computing section. Since two sensibility statuses are detected from the beat per minute information and the volume information, each weight of the two sensibility statuses has a value of a half.

The computing of the sensibility status information is summarized based on the sensibility status information and the weight as shown below. Self-confidence 90, concord 65, comfort 50, joy 35, and indifference 15

That is, the computed sensibility status information is described above, and the main sensibility status is self-confidence Therefore, with regard to the summarized sensibility status information regarding the speaker, the speaker information is the 37-year-old male who on the weekend afternoon speaks the Pusan dialect and is overweight, the speaker's sensibility status is self-confidence and the disharmony index is about 27 %. The harmonic disharmony index is computed according to a ratio of a time of occurrence of the harmonic disharmony with respect to the total time required to input the noise signal, the music signal, and the speaker voice signal.

[Industrial Applicability] Although several exemplary embodiments of the present invention have been described for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Claims

[CLAIMS]

[Claim 1]

A sensibility recognition method using musical elements, the method comprising the steps of: a) previously establishing a mapping table obtained by mapping sensibility status information regarding a speaker and musical element information invoking the sensibility status information and recording the mapping table; b) detecting musical element information and harmonic disharmony information from input voice signals; c) reading and computing the sensibility status information corresponding to the detected musical element information from a previously established mapping table, and computing a harmonic disharmony index from the harmonic disharmony information; and d) visually/audibly displaying the computed sensibility status information and harmonic disharmony index.

[Claim 2] The method as claimed in claim 1, wherein step b) comprises the steps of: bl) identifying and separating an externally input voice signal into a noise signal, a music signal, and a speaker voice signal; and b2) comparing the musical element information stored in the mapping table based on the separated noise signal, music signal, and speaker voice signal, detecting information that is consistent for more than an established rate, and giving a weight to harmony information detected based on the consistency degree and frequency of the detected information.

[Claim 3]

The method as claimed in claim 2, wherein step b2) comprises the steps of: detecting each radical of the separated noise signal, the music signal, and the speaker voice signal according to time, comparing pitch information of at least two of the detected adjacent radicals with chord codes stored in the mapping table, and detecting chord information that is consistent for more than a predetermined rate; giving a weight to harmony information detected based on the consistency degree and frequency of the detected chord information within an established temporal section, and detecting the harmonic disharmony information that is not consistent with the chord information; comparing beat per minute values stored in the mapping table based on temporal variation of each beat and radical of the separated noise signal, music signal, and speaker voice signal, and detecting beat per minute information that is consistent for more than the predetermined rate; giving a weight to beat per minute information detected based on the consistency degree and frequency of the detected beat per minute information within an established temporal section; detecting each volume of the separated noise signal, music signal, and speaker voice signal in real-time, comparing volume values stored in the mapping table with the detected volumes of the separated noise signal, music signal, and speaker voice signal, and detecting volume information that is consistent for more than the predetermined rate; giving a weight to volume information detected based on the consistency degree and frequency of the detected volume information within the established temporal section; detecting each overtone structure and register range values of the separated noise signal, music signal, and speaker voice signal in real-time, comparing overtone structure and register range values stored in the mapping table with the detected overtone structure and register range values, and detecting overtone structure and register range information that is consistent for more than the predetermined rate; and giving a weight to overtone structure and register range information detected based on the consistency degree and frequency of the detected overtone structure and register range information within the established temporal section.

[Claim 4]

The method as claimed in claim 3, further comprising the steps of: if the established temporal section has passed, detecting period information from the variation of each chord, beat per minute, volume, and overtone structure and register range information; and renewing mapping information stored in the mapping table based on the speaker and period information.

[Claim 5]

The method as claimed in claim 4, further comprising the steps of: if there is no speaker voice signal, predicting the speaker's sensibility status based on the detected period information; and selectively providing the speaker with at least one of established noise, automatic noise, established music, and automatic music including the musical element information corresponding to the speaker's sensibility status based on the predicted speaker's sensibility status.

[Claim 6]

The method as claimed in claim 1, wherein step c) comprises the steps of: detecting the sensibility status information corresponding to the detected musical element information from the mapping table based on the musical element information; summing weights of the detected sensibility status information excluding the sensibility status in a section opposite to the sensibility status corresponding to a maximum weight and computing the sensibility status information; and calculating the harmonic disharmony index at a ratio of a time of occurrence of disharmony of the detected harmonic disharmony information with regard to the total time required to input the voice signals.

[Claim 7]

The method as claimed in claim 1, wherein step d) comprises the steps of: reading noise information and music information including the computed sensibility status information from the mapping table; providing the speaker with established noise and established music corresponding to the read noise information and music information in real-time; and playing automatic noise and automatic music including the computed sensibility status information and the harmonic disharmony information based on the computed sensibility status information and the harmonic disharmony information, and providing the speaker with the automatic noise and automatic music.

[Claim 8]

The method as claimed in claim 1, further comprising the step of renewing the visually/audibly displayed sensibility status information by the speaker.

[Claim 9]

The method as claimed in any one of claims 1, 2, 5, and 6, wherein the musical element information includes at least one of the chord information, the beat per minute information, the volume information, and the overtone structure and register range information.