CN103745228B - Dynamic gesture identification method on basis of Frechet distance - Google Patents

Abstract

The invention discloses a dynamic gesture identification method on the basis of a Frechet distance, which at least comprises the following steps of: acquiring gesture position information of a dynamic gesture fragment to be identified in an input video; carrying out matching on an acquired gesture state variation characteristic sequence and a characteristic sequence in a preset model according to the Frechet distance; and according to Frechet distance matching information, acquiring and outputting a similarity result. According to the invention, the extracted characteristic sequence and the pre-obtained model are subjected to similarity measurement in a certain mode and according to the similarity degree, a corresponding classification of the gesture to be identified is determined; and by utilizing the characteristic that the Frechet distance keeps stretching transformation of a time sequence curve along the time dimension unchanged, the dynamic gesture identification method can be well suitable for the condition of nonuniform distribution of the change speed of the dynamic gesture along the time dimension.

Description

Dynamic gesture identification method based on Fr é chet distance

Technical field

The present invention relates to a kind of dynamic gesture identification method based on Fr é chet distance.

Background technology

Gesture is that people are daily to express heart activity, to carry out the important method of communication exchange with other people.We are usual According to gesture situation of change on time dimension, gesture is divided into static gesture and dynamic gesture.

Static gesture refers to the most relatively-stationary finger, the position of palm, constitutes towards features such as, attitudes The particular space state of hands.Specific static gesture can represent with specified point corresponding in feature space.Phase therewith Corresponding is then dynamic gesture, and it is made up of the motion of continually varying hands in a period of time region, attitude sequence.For dynamic For state gesture, if we choose suitable feature, the gesture information of particular point in time is described, then over time T, t=1,2 ..., the consecutive variations of T, we have just obtained one section of characteristic sequence (F that this dynamic gesture is corresponding₁,F₂,..., F_T).If each element in characteristic sequence is made in feature space by successively, then dynamic gesture always can use feature In space, a corresponding curve describes.

And after dynamic gesture describes out, problem below is gesture identification.It is said that in general, gesture identification is basic Task is exactly the feature corresponding to gesture to be matched with existing model, thus by the characteristic parameter space corresponding to gesture Curve or point are divided into the process of different sets or classification.

At present, the flow process framework of gesture recognition system is typically by three parts such as gesture modeling, gesture analysis and gesture couplings Constitute.

Gesture modeling is typically for given gesture.Under normal circumstances, the needs of choosing of gesture model combine actual answering Determine by background.As for dynamic gesture, choosing of gesture model needs to reflect its spy on time, Spatial Dimension Levy, the motion of gesture is considered as the process being front and back closely connected simultaneously.

Once gesture model is decided, and we will calculate the relevant feature parameters of gesture by gesture analysis process, this A little features actually represent for certain gestures attitude and certain description of movement locus.The process of gesture analysis is complex, It is typically the comprehensive of the processes such as gesture location, feature selection, feature extraction, model parameter estimation.By this serial procedures, Relevant to gesture is partially separated out from image or video by we, and according to the location to gesture, from relevant range or According to the relation of frame before and after in video, extract characteristic in order to the observer state of gesture is carried out suitable description, simultaneously root According to training sample feature, model parameter is estimated.The implementation method of above steps needs also exist for specifically should according to problem It is designed by background.

The basic task of gesture coupling is then feature to be matched with model, thus the characteristic parameter corresponding to gesture is empty Curve between or point are divided into the process of different subset.The most conventional thinking is with the model being previously obtained by the feature of extraction Carry out some form of similarity measurement, determine, according to similarity degree, the classification that gesture to be identified is corresponding.

Summary of the invention

For the problems referred to above, it is an object of the invention to provide a kind of dynamic hand gesture recognition side based on Fr é chet distance Method.

For reaching above-mentioned purpose, a kind of dynamic gesture identification method based on Fr é chet distance of the present invention, at least wraps Include following steps:

Obtain the hand gesture location information of dynamic gesture fragment to be identified in input video；

Obtain the gesture state variation characteristic sequence of frame the most front and back at described hand gesture location；

The gesture state variation characteristic sequence of acquisition is carried out according to Fr é chet distance with the characteristic sequence in preset model Coupling；

Obtain correlation result according to Fr é chet distance match information and export.

Preferably, the concrete steps of the hand gesture location information of dynamic gesture fragment to be identified in described acquisition input video For:

According to the rgb value of pixel in video, obtain the probit that in any one two field picture of video, area of skin color occurs；

Judge to obtain all area of skin color of distribution in each two field picture of video according to described probit；

Obtain each area of skin color light flow valuve of two field picture before and after continuously；

The light flow valuve of all area of skin color according to distribution, obtains the region that average light flow valuve is maximum, i.e. hand gesture location district Territory.

Preferably, at the described hand gesture location of described acquisition, the most front and back the gesture state variation characteristic sequence of frame includes motion Trail change characteristic sequence and attitudes vibration characteristic sequence.

Preferably, the coupling step of described movement locus variation characteristic sequence is:

The average light stream of any one frame and former frame that arrange hand gesture location region inner video image is direction vector, it may be assumed that F =(x, y), wherein x, y represent average light stream component in horizontal, longitudinal direction respectively；

Obtaining the motion feature sequence corresponding to each frame of inputted video image is (F₁,F₂,...,F_T)；

Obtaining the motion feature sequence arranged in model is (M₁,M₂,...,M_T);

Choose motion feature sequence (F₁,F₂,...,F_TAny one section of sequence fragment f=(F in)_i,F_i+1,...,F_j), Arbitrarily vector (x in this fragment₁,y₁) and motion feature sequence (M₁,M₂,...,M_TArbitrarily vector (x in)₂,y₂) away from From, it is:

$d_1((x_1,y_1),(x_2,y_2))=1-\frac{x_1\·x_2+y_1\·y_2}{\sqrt{x_1^2+y_1^2}\·\sqrt{x_2^2+y_2^2}};$

From the multiple d obtained₁((x₁,y₁),(x₂,y₂)) maximizing, ask for the lower bound of maximum, just can get f= (F_i,F_i+1,...,F_j) and (M₁,M₂,...,M_T) Fr é chet distance δ_F；

One threshold epsilon is set₁, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₁, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is model coupling Gesture；

Otherwise, it is not.

Preferably, the coupling step of described attitudes vibration characteristic sequence is:

Arrange during in any one frame of hand gesture location region inner video image, finger fingertip and middle finger refer to the corresponding finger of root The coordinate of heart position is attitude vectors, is Z, and wherein Z is the characteristic vector of 12 dimensions；

Obtaining the posture feature sequence corresponding to each frame of inputted video image is (Z₁,Z₂,...,Z_T)；

Obtaining the posture feature sequence arranged in model is (N₁,N₂,...,N_T);

Choose attitude motion characteristic sequence (Z₁,Z₂,...,Z_TAny one section of sequence fragment f=(Z in)_i,Z_i+1,..., Z_j), obtain any vector Z in this fragment and attitude motion characteristic sequence (N₁,N₂,...,N_TArbitrarily vector N in) away from From, it is:

d₂(Z,N)=||Z-N||₂；

From the multiple d obtained₂(Z, N) maximizing, asks for the lower bound of maximum, just can get f=(Z_i,Z_i+1,..., Z_j) and (N₁,N₂,...,N_T) Fr é chet distance δ_F；

One threshold epsilon is set₂, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₂, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is model coupling Gesture；

Otherwise, it is not.

The invention have the benefit that

The present invention, by the characteristic sequence of extraction is carried out some form of similarity measurement with the model being previously obtained, depends on Determine, according to similarity degree, the classification that gesture to be identified is corresponding, and utilize Fr é chet distance to have time-serial position along the time Dimension stretching keeps constant characteristic, it is possible to be well adapted for dynamic gesture pace of change along time dimension skewness Situation.

Accompanying drawing explanation

Fig. 1 is the block schematic illustration of dynamic gesture identification method based on Fr é chet distance described in the embodiment of the present invention；

Fig. 2 is embodiment of the present invention certain gestures orbiting motion schematic diagram；

Fig. 3 is embodiment of the present invention certain gestures attitudes vibration procedure chart；

Fig. 4 is that embodiment of the present invention gesture posture feature chooses schematic diagram；

Fig. 5 is the block schematic illustration that embodiment of the present invention posture feature is chosen.

Detailed description of the invention

Below in conjunction with Figure of description, the present invention will be further described.

As it is shown in figure 1, a kind of dynamic gesture identification method based on Fr é chet distance described in the embodiment of the present invention, at least Comprise the following steps:

Obtain the hand gesture location information of dynamic gesture fragment to be identified in input video；

Obtain the gesture state variation characteristic sequence of frame the most front and back at described hand gesture location；

Characteristic sequence in the gesture state variation characteristic sequence of acquisition and preset model is carried out according to Fr é chet distance Join；

Obtain correlation result according to Fr é chet distance match information and export.

Wherein it is desired to explanation: definition Fr é chet distance is: any one belongs toThe parametric curve in space can To be expressed as Continuous MappingsWherein a <b.Then for curveWith Then Fr é chet distance is

${\mathrm{\&delta;}}_F(f,g):=\underset{\underset{\mathrm{\&beta;}:\left[\mathrm{0,1}\right]\&RightArrow;[b,b^{\&prime;}]}{\mathrm{\&alpha;}:\left[\mathrm{0,1}\right]\&RightArrow;[a,a^{\&prime;}]}}{\inf }\underset{t\&Element;\left[\mathrm{0,1}\right]}{\max }d\left(f(\mathrm{\&alpha;}\left(t\right)\right),g\left(\mathrm{\&beta;}\left(t\right)\right))$

In this formula, α, β are the continuous monotonically increasing function being defined on [0,1], and meet α (0)=a, α (1)=a', β (0)=b, β (0)=b'.

Said method step further explanation is illustrated:

First have to determine the place, position of gesture in video image, the hand gesture location of image to be positioned.It determines Concretely comprising the following steps of hand gesture location information:

According to the rgb value of pixel in video, obtain the probit that in any one two field picture of video, area of skin color occurs；

Judge to obtain all area of skin color of distribution in each two field picture of video according to described probit；

Obtain each area of skin color light flow valuve of two field picture before and after continuously；

The light flow valuve of all area of skin color according to distribution, obtains the region that average light flow valuve is maximum, i.e. hand gesture location district Territory.

The specific explanations of above-mentioned steps is such that in the identification of gesture path, and specific gesture implication is made by hands Show for overall movement locus.For each two field picture, color and kinematic constraint can be passed through to the hands of motion in video Position.

On the one hand, in video, each two field picture is made up of R, G, B triple channel, needs by colour of skin district in some pictures The rgb value of territory and non-area of skin color is added up, obtain one big little be 256 × 256 × 256 form, each item in form Represent the probability that the rgb value of its correspondence occurs with colour of skin form.So, each pixel for each two field picture in video must Arrive its approximation probability as skin, and then can substantially estimate region the dividing in each two field picture of video of the approximation colour of skin Cloth.

On the other hand, owing in expressing in gesture, hands is kept in motion all the time as entirety, a certain in video Two field picture, need to choose the region that average light stream is maximum, i.e. by itself and the light stream of previous frame image in the region of the approximation colour of skin Obtain the location to motion hands.

Behind hand gesture location location, characteristic sequence to be extracted.Before and after at the described hand gesture location of described acquisition continuously The gesture state variation characteristic sequence of frame includes movement locus variation characteristic sequence and attitudes vibration characteristic sequence.Accordingly, it would be desirable to Respectively the feature in terms of movement locus aspect and attitude is extracted.

The coupling step of described movement locus variation characteristic sequence is:

The average light stream of any one frame and former frame that arrange hand gesture location region inner video image is direction vector, it may be assumed that F =(x, y), wherein x, y represent average light stream component in horizontal, longitudinal direction respectively；

Obtaining the motion feature sequence corresponding to each frame of inputted video image is (F₁,F₂,...,F_T)；

Obtaining the motion feature sequence arranged in model is (M₁,M₂,...,M_T);

Choose motion feature sequence (F₁,F₂,...,F_TAny one section of sequence fragment f=(F in)_i,F_i+1,...,F_j), Arbitrarily vector (x in this fragment₁,y₁) and motion feature sequence (M₁,M₂,...,M_TArbitrarily vector (x in)₂,y₂) away from From, it is:

$d_1((x_1,y_1),(x_2,y_2))=1-\frac{x_1\&CenterDot;x_2+y_1\&CenterDot;y_2}{\sqrt{x_1^2+y_1^2}\&CenterDot;\sqrt{x_2^2+y_2^2}};$

From the multiple d obtained₁((x₁,y₁),(x₂,y₂)) maximizing, ask for the lower bound of maximum, just can get f= (F_i,F_i+1,...,F_j) and (M₁,M₂,...,M_T) Fr é chet distance δ_F；

One threshold epsilon is set₁, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₁, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is model coupling Gesture；

Otherwise, it is not.

Specific explanations is:

1, model is set up

Certain gestures track is to be combined in a fixed order by some most basic strokes, and therefore, we choose Specifically by direction vector, (x, sequence y) constituted is as the model representing gesture path.As in figure 2 it is shown, in figure each point is successively Representative model each element of correspondence seasonal effect in time series, (x y) is marked the direction vector corresponding to each element in the drawings by arrow.

2, feature selection and extraction

The sequence that gesture path is made up of specific direction vector represents, and the direction of motion of gesture can be by light stream The calculating of (optical flow) is asked for.According to the location estimation to motion hands, can be previous with adjacent by this frame in video (x, y), wherein x, y represent that average light stream is transversely, longitudinally to the average light stream F=of the image calculating gesture region of frame respectively Component, and as this two field picture character pair.

So for each frame (V in video₁,V₂,...,V_T), just obtain corresponding characteristic sequence (F₁,F₂,..., F_T)。

3, Fr é chet distance coupling

Choose motion feature sequence (F₁,F₂,...,F_TAny one section of sequence fragment f=(F in)_i,F_i+1,...,F_j), Arbitrarily vector (x in this fragment₁,y₁) and motion feature sequence (M₁,M₂,...,M_TArbitrarily vector (x in)₂,y₂) away from From, it is:

$d_1((x_1,y_1),(x_2,y_2))=1-\frac{x_1\&CenterDot;x_2+y_1\&CenterDot;y_2}{\sqrt{x_1^2+y_1^2}\&CenterDot;\sqrt{x_2^2+y_2^2}};$

From the multiple d obtained₁((x₁,y₁),(x₂,y₂)) maximizing, ask for the lower bound of maximum, just can get f= (F_i,F_i+1,...,F_j) and (M₁,M₂,...,M_T) Fr é chet distance δ_F；

One threshold epsilon is set₁, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₁, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is model coupling Gesture；

Otherwise, it is not.

The coupling step of described attitudes vibration characteristic sequence is:

Arrange during in any one frame of hand gesture location region inner video image, finger fingertip and middle finger refer to the corresponding finger of root The coordinate of heart position is attitude vectors, is Z, and wherein Z is the characteristic vector of 12 dimensions；

Obtaining the posture feature sequence corresponding to each frame of inputted video image is (Z₁,Z₂,...,Z_T)；

Obtaining the posture feature sequence arranged in model is (N₁,N₂,...,N_T);

Choose attitude motion characteristic sequence (Z₁,Z₂,...,Z_TAny one section of sequence fragment f=(Z in)_i,Z_i+1,..., Z_j), obtain any vector Z in this fragment and attitude motion characteristic sequence (N₁,N₂,...,N_TArbitrarily vector N in) away from From, it is:

d₂(Z,N)=||Z-N||₂；

From the multiple d obtained₂(Z, N) maximizing, asks for the lower bound of maximum, just can get f=(Z_i,Z_i+1,..., Z_j) and (N₁,N₂,...,N_T) Fr é chet distance δ_F；

One threshold epsilon is set₂, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₂, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is model coupling Gesture；

Otherwise, it is not.

Specific explanations is:

1, model is set up

For specific gesture attitudes vibration process, choose its representative gesture stationary posture sequence as in order to The model of gesture attitudes vibration process is described.Each element in attitude sequence is taken from the stationary posture data base of hands, they with Feature one_to_one corresponding cited below.It is illustrated in figure 3 attitudes vibration figure.

2, character selection and abstraction

Specific gesture attitude by hands towards, finger with palm relative to information decisions such as positions.Therefore each finger is chosen Finger tip and middle finger refer to that the root joint position coordinates relative to wrist center is as the feature describing certain gestures attitude.As shown in Figure 4.

It is illustrated in figure 5 the block schematic illustration that posture feature is chosen, actual when carrying out feature assessment, need to be by this two field picture Characteristics of image retrieve in the stationary posture data base of hands, obtain the stationary posture model approximated the most with it, as It is in the estimation of this moment attitude, and each finger fingertip and the middle finger of model are referred to the root joint horizontal stroke relative to wrist center To, lengthwise position coordinate information, totally 12 dimension real numbers are as to this moment gesture attitude real features.

3, Fr é chet distance coupling

Choose attitude motion characteristic sequence (Z₁,Z₂,...,Z_TAny one section of sequence fragment f=(Z in)_i,Z_i+1,..., Z_j), obtain any vector Z in this fragment and attitude motion characteristic sequence (N₁,N₂,...,N_TArbitrarily vector N in) away from From, it is:

d₂(Z,N)=||Z-N||₂；

From the multiple d obtained₂(Z, N) maximizing, asks for the lower bound of maximum, just can get f=(Z_i,Z_i+1,..., Z_j) and (N₁,N₂,...,N_T) Fr é chet distance δ_F；

One threshold epsilon is set₂, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₂, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is model coupling Gesture；

Otherwise, it is not.

Above, only presently preferred embodiments of the present invention, but protection scope of the present invention is not limited thereto, any it is familiar with basis Those skilled in the art in the technical scope that the invention discloses, the change that can readily occur in or replacement, all should contain Within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain that claim is defined.

Claims (5)

1. a dynamic gesture identification method based on Fr é chet distance, it is characterised in that at least comprise the following steps:

Obtain the hand gesture location information of dynamic gesture fragment to be identified in input video；

Obtain the gesture state variation characteristic sequence of frame the most front and back at described hand gesture location；

Characteristic sequence in the gesture state variation characteristic sequence of acquisition and preset model is carried out according to Fr é chet distance Join；

Obtain correlation result according to Fr é chet distance match information and export.

Dynamic gesture identification method based on Fr é chet distance the most according to claim 1, it is characterised in that described in obtain Take the concretely comprising the following steps of hand gesture location information of dynamic gesture fragment to be identified in input video:

According to the rgb value of pixel in video, obtain the probit that in any one two field picture of video, area of skin color occurs；

Judge to obtain all area of skin color of distribution in each two field picture of video according to described probit；

Obtain each area of skin color light flow valuve of two field picture before and after continuously；

The light flow valuve of all area of skin color according to distribution, obtains the region that average light flow valuve is maximum, i.e. hand gesture location region.

Dynamic gesture identification method based on Fr é chet distance the most according to claim 1, it is characterised in that described in obtain The gesture state variation characteristic sequence taking frame the most front and back at described hand gesture location includes movement locus variation characteristic sequence and appearance State variation characteristic sequence.

Dynamic gesture identification method based on Fr é chet distance the most according to claim 3, it is characterised in that described fortune The coupling step of dynamic trail change characteristic sequence is:

The average light stream of any one frame and former frame that arrange hand gesture location region inner video image is direction vector, it may be assumed that F= (x, y), wherein x, y represent average light stream component in horizontal, longitudinal direction respectively；

Obtaining the motion feature sequence corresponding to each frame of inputted video image is (F₁,F₂,…,F_T)；

Obtaining the motion feature sequence arranged in model is (M₁,M₂,…,M_T)；

Choose motion feature sequence (F₁,F₂,…,F_TAny one section of sequence fragment f=(F in)_i,F_i+1,…,F_j), obtain this sheet Arbitrarily vector (x in section₁,y₁) and motion feature sequence (M₁,M₂,…,M_TArbitrarily vector (x in)₂,y₂) distance, be:

$d_1\left(\right(x_1,y_1),(x_2,y_2\left)\right)=1-\frac{x_1\&CenterDot;x_2+y_1\&CenterDot;y_2}{\sqrt{x_1^2+y_1^2}\&CenterDot;\sqrt{x_2^2+y_2^2}};$

From the multiple d obtained₁((x₁,y₁),(x₂,y₂)) maximizing, ask for the lower bound of maximum, just can get f=(F_i, F_i+1,…,F_j) and (M₁,M₂,…,M_T) Fr é chet distance δ_F；

One threshold epsilon is set₁, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₁, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is the hands of model coupling Gesture；

Otherwise, if d₁>ε₁, then judge that in video to be identified, i-th frame is not model to the corresponding gesture motion shown in jth frame The gesture joined.

Dynamic gesture identification method based on Fr é chet distance the most according to claim 3, it is characterised in that described appearance The coupling step of state variation characteristic sequence is:

Finger fingertip and middle finger in any one frame of hand gesture location region inner video image are set and refer to root corresponding finger centre position The coordinate put is attitude vectors, is Z, and wherein Z is the characteristic vector of 12 dimensions；

Obtaining the posture feature sequence corresponding to each frame of inputted video image is (Z₁,Z₂,…,Z_T)；

Obtaining the posture feature sequence arranged in model is (N₁,N₂,…,N_T)；

Choose attitude motion characteristic sequence (Z₁,Z₂,…,Z_TAny one section of sequence fragment f=(Z in)_i,Z_i+1,…,Z_j), obtain Any vector Z in this fragment and attitude motion characteristic sequence (N₁,N₂,…,N_TThe distance of the arbitrarily vector N in), is:

d₂(Z, N)=| | Z-N | |₂；

From the multiple d obtained₂(Z, N) maximizing, asks for the lower bound of maximum, just can get f=(Z_i,Z_i+1,…,Z_j) and (N₁,N₂,…,N_T) Fr é chet distance δ_F；

One threshold epsilon is set₂, it is judged that this threshold value and δ_FSize:

If δ_F≤ε₂, then judge that in video to be identified, i-th frame to the corresponding gesture motion shown in jth frame is the hands of model coupling Gesture；

Otherwise, if d₂>ε₂, then judge that in video to be identified, i-th frame is not model to the corresponding gesture motion shown in jth frame The gesture joined.