EP1197952A1 - Coding method of the prosody for a very low bit rate speech encoder - Google Patents

Abstract

The speech coding decoding system has a step of learning to identify speech signal representatives and a coding step segmenting the speech signals, and determining the best associated representation. There is a step of coding/decoding of one parameter from the recognised information segment set which is the best representation of energy or pitch and/or closeness and/ or segment length.

Description

The present invention relates to a method for coding the speech at very low speed and the associated system. It applies in particular for speech coding-decoding systems by unit indexing of variable size.

The speech coding method implemented at low bit rate, by example of the order of 2400 bits / s, is generally that of the vocoder using a fully parametric model of the speech signal. The parameters used relate to voicing which describes the character periodic or random signal, the fundamental frequency of voiced sounds still known by the English term "PITCH", the evolution temporal energy, as well as the spectral envelope of the signal generally modeled by an LPC filter (abbreviation for Anglo-Saxon Linear Predictive Coding).

These different parameters are estimated periodically over the speech signal, typically every 10 to 30 ms. They are developed in level of an analysis device and are generally transmitted remotely in direction of a synthesis device reproducing the speech signal from of the quantized value of the model parameters.

So far, the lowest standard rate for a coder speech using this technique is 800 bits / s. This encoder, standardized in 1994 is described by the NATO standard STANAG 4479 and in the article entitled "NATO STANAG 4479: A standard for an 800 bps vocoder and channel coding in HF-ECCM system ”, IEEE Int. Conf. on ASSP, Detroit, pp 480-483, May 1995 whose authors are Mouy, B., De La Noue, P., and Goudezeune, G. he is based on a frame by frame analysis technique (22.5 ms) of the type LPC 10 and makes maximum use of the temporal redundancy of the signal speech by grouping the frames 3 by 3 before encoding the parameters.

Although intelligible, the speech reproduced by these techniques of coding is of fairly poor quality and is no longer acceptable from when the bitrate is less than 600 bits / s.

One way to reduce the speed is to use vocoders phonetic type segmentals with segments of variable duration which combine principles of speech recognition and synthesis.

The encoding procedure essentially uses a system of automatic speech recognition in continuous flow, which segments and "Labels" the speech signal according to a number of speech units of size variable. These phonetic units are coded by indexing in a small dictionary. Decoding is based on the principle of speech synthesis by concatenation from the index of phonetic units and the prosody. The term "prosody" mainly includes parameters following: signal energy, pitch, voicing information and possibly the time rhythm.

However, the development of phonetic coders requires significant knowledge of phonetics and liguistics, as well as phonetic transcription phase of a learning database which is expensive and which can be the source of errors. In addition, the coders phonetics hardly adapt to a new language or a new speaker.

Another technique, described for example in the thesis of J. Cernocky, entitled "Speech Processing Using Automatically Derived Segmental Units: Applications to very Low Rate Coding and Speaker Verification ”of the University Paris XI Orsay, December 1998 allows to work around problems with phonetic transcription of the database learning data by determining the speech units so automatic and regardless of language.

The operation of this type of encoder is broken down mainly in two stages: a learning stage and a coding-decoding described in figure 1.

During the learning step (FIG. 1), an automatic procedure determines for example after a parametric analysis 1 and a segmentation step 2, a set of 64 classes of acoustic units designated "UA". Each of these classes of acoustic units is associated with a statistical model 3, of the Markov model type (HMM abbreviation Anglo-Saxon de Hidden Markov Model), as well as a small number of units representing a class, designated under the term “representatives” 4. In the current system, representatives are simply the 8 longest units belonging to the same acoustic class. They can also be determined as being the N units most representative of the acoustic unit. During the coding of a speech signal after a parametric analysis step 5 making it possible in particular to obtain the spectral parameters, the energies, the pitch, a recognition procedure (6, 7), using an algorithm of Viterbi, determines the succession of acoustic units of the speech signal and identifies the "best representative" to use for speech synthesis. This choice is made for example by using a spectral distance criterion, such as the DTW algorithm (abbreviation for Dynamic Time Warping).
The number of the acoustic class, the index of this representative unit, the length of the segment, the content of DTW and the prosodic information resulting from the parametric analysis are transmitted to the decoder. Speech synthesis is done by concatenation of the best representatives, possibly using a parametric LPC type synthesizer.

To concatenate the representatives during speech decoding, we use, for example, a parametric analysis / synthesis process of the speech. This parametric process notably allows modifications of prosody such as time evolution, fundamental frequency or pitch, compared to a simple concatenation of waveforms.

The parametric speech model used by the process analysis / synthesis can be binary excitation voiced / unvoiced type LPC 10 as described in the document entitled "The government standard linear predictive coding algorithm: LPC-10 ”by T.Tremain published in the Speech Technology, vol.1, n ° 2, pp 40-49.

This technique makes it possible to code the spectral envelope of the signal in around 185 bits / s for a single-speaker system, for an average about 21 segments per second.

• le terme « représentant » correspond à l'un des segments de la base d'apprentissage qui a été jugé représentatif d'une des classes d'unités acoustique,
• l'expression « segment reconnu » correspond à un segment de la parole qui a été identifié comme appartenant à l'une des classes acoustiques, par le codeur,
• l'expression « meilleur représentant » désigne le représentant déterminé au niveau du codage qui représente le mieux le segment reconnu.

In the following description, the following terms have the following meanings:

• the term “representative” corresponds to one of the segments of the learning base which has been deemed representative of one of the classes of acoustic units,
• the expression “recognized segment” corresponds to a segment of speech which has been identified as belonging to one of the acoustic classes, by the coder,
• the expression “best representative” designates the representative determined at the coding level which best represents the recognized segment.

The object of the present invention relates to a coding method, prosody decoding for a very low bit rate speech coder using including the best representatives.

It also relates to data compression.

The invention relates to a speech coding-decoding method. using a very low bit rate coder including a learning step allowing to identify “representatives” of the speech signal and a step coding to segment the speech signal and determine the "best representative ”associated with each recognized segment. It is characterized in that that it includes at least one coding-decoding step of one of the parameters of at least the prosody of the recognized segments, such as the energy and / or the pitch and / or the voicing and / or the length of the segments, in using prosody information from the "best representatives".

The prosody information of the representatives used is by example the energy contour or the voicing or the length of the segments or the pitch.

The step of coding the length of the recognized segments consists for example in coding the difference in length between the length of a recognized segment and the length of the "best representative" multiplied by a given factor.

According to one embodiment, it includes a step of coding the time alignment of the best representatives using the path of DTW and looking for the nearest neighbor in a shape table.

The energy coding step may include a step of determining for each start of a “recognized segment” the difference ΔE (j) between the energy value E _rd (j) of the “best representative” and the value d energy E _sd (j) from the start of the “recognized segment” and the decoding step include, for each recognized segment, a first step consisting in translating the energy contour of the best representative by a quantity ΔE (j) to make coincide the first energy E _rd (j) of the "best representative" with the first energy E _sd (j + 1) of the recognized segment of index j + 1.

The voicing coding step comprises for example a step of determining the existing differences ΔT _k for each end of a voicing area of index k between the voicing curve of the recognized segments and that of the best representatives and the step of decoding comprises for example for each end of a voicing area of index k a step of correcting the time position of this end with a corresponding value ΔT _k and / or a step of deleting or inserting a transition .

The method also relates to a coding-decoding system for the speech comprising at least one memory for storing a dictionary comprising a set of representatives of the speech signal, a microprocessor suitable for determining the recognized segments, for reconstruct speech from the "best representatives" and to put implementing the process steps according to one of the aforementioned characteristics.

The representative dictionary is for example common to coder and decoder of the coding-decoding system.

The method and the system according to the invention can be used for speech coding and decoding at bit rates below 800 bits / s and preferably less than 400 bits / s.

The coding-decoding method and system according to the invention offer in particular the advantage of coding prosody at very low speed and of thus providing a complete encoder in this field of application.

• la figure 1 représente un schéma d'apprentissage, de codage et de décodage de la parole selon l'art antérieur,
• les figures 2 et 3 décrivent des exemples de codage de la longueur des segments reconnus,
• la figure 4 schématise un modèle d'alignement temporel des « meilleurs représentants »,
• les figures 5 et 6 montrent des courbes des énergies du signal à coder et des représentants alignés, ainsi que les contours des énergies initial et décodé obtenus en mettant en oeuvre le procédé selon l'invention,
• la figure 7 schématise le codage du voisement du signal de parole, et
• la figure 8 est un exemple de codage du pitch.

Other characteristics and advantages will appear on reading the detailed description of an embodiment taken by way of nonlimiting example and illustrated by the appended drawings where:

• FIG. 1 represents a diagram of learning, coding and decoding of speech according to the prior art,
• FIGS. 2 and 3 describe examples of coding the length of the recognized segments,
• FIG. 4 schematizes a model of temporal alignment of the "best representatives",
• FIGS. 5 and 6 show curves of the energies of the signal to be coded and of the aligned representatives, as well as the contours of the initial and decoded energies obtained by implementing the method according to the invention,
• FIG. 7 diagrams the coding of the voicing of the speech signal, and
• Figure 8 is an example of pitch coding.

The coding principle according to the invention is based on the use of "Best representatives", including their prosody information, for coding and / or decoding at least one of the prosody parameters of a signal speech, for example pitch, signal energy, voicing, length recognized segments.

To compress prosody at very low speed, the principle work uses encoder segmentation as well as information prosodic "best representatives".

The following description given by way of illustration and in no way limitative describes a method for coding prosody in a coding-decoding device low-speed speech that includes a dictionary obtained from automatically, for example, during learning as described in figure 1.

• plusieurs classes d'unités acoustiques UA, chaque classe étant déterminée à partir d'un modèle statistique,
• pour chaque classe d'unités acoustiques, un ensemble de représentants.

The dictionary includes the following information:

• several UA acoustic unit classes, each class being determined from a statistical model,
• for each class of acoustic units, a set of representatives.

This dictionary is known to the coder and the decoder. he corresponds for example to one or more languages and to one or more speakers.

The coding-decoding system comprises for example a memory to store the dictionary, a microprocessor suitable for determine the recognized segments, for the implementation of the different steps of the method according to the invention and for reconstructing speech from best representatives.

The method according to the invention implements at least one of the steps following: the coding of the length of the segments, the coding of the time alignment of the “best representatives”, the coding and / or the energy decoding, encoding and / or decoding information from voicing and / or encoding and / or decoding of pitch and / or decoding of the length of the segments and the time alignment.

Segment length coding

The coding system determines on average a number Ns of segments per second, for example 21 segments. The size of these segments varies depending on the class of UA acoustic units. It appears that for the majority of UA, the number of segments decreases according to a relation 1 / x ^2.6 , where x is the length of the segment.

An alternative embodiment of the method according to the invention consists in code the difference in variable length between the "recognized segment" and the length of the “best representative” according to a diagram described in Figure 2.

This diagram in the left column shows the length of the code word to use and in the right column the length difference between the length of the segment recognized by the coder for the speech signal and that of the best representative.

According to another embodiment given in FIG. 3, the coding of the absolute length of a recognized segment is carried out using a code of variable length similar to that of Huffman known to man of the profession, which makes it possible to obtain a bit rate of around 55 bits / s.

Using long code words to code lengths recognized large segments, in particular, retains the value of flow in a limited variation range. Indeed, these long segments reduce the number of segments recognized per second and the number of lengths to code.

In summary, we code for example with a length code variable the difference between the length of the recognized segment and the length of the best representative multiplied by a certain factor, this factor can be between 0 (absolute coding) and 1 (difference coding).

Coding of the best representatives' time alignment

The time alignment is for example carried out by following the DTW path (Anglo-Saxon abbreviation for Dynamic Time Warping) which was determined when looking for the "best representative" for code the "recognized segment".

FIG. 4 represents the path (C) of the DTW corresponding to the time contour which minimizes the distortion between the parameter to be coded (axis abscissas), for example the vector of “cepstral” coefficients, and the "Best representative" (ordinate axis). This approach is described in the book entitled “Word processing”, for author René Box and Murat Kunt published by Presses Polytechnique Romandes éditions 1987.

The coding of the alignment of the "best representatives" is performed by finding the nearest neighbor in a table containing type forms. The choice of these standard forms is made for example by a statistical approach, such as learning on a database of speech or by an algebraic approach for example the description by configurable mathematical equations, these different methods being known to those skilled in the art.

According to another approach, valid in the case where the segments of small size are in significant proportion, the process performs a alignment of segments along the diagonal rather than the exact path of DTW. The flow is then zero.

Energy coding and decoding

When we classify and analyze the base segments of speech data belonging to each of the classes of acoustic units, we can see that there is a certain consistency in the form of contours of energies. In addition, there are similarities between energy contours of the best representatives aligned by DTW and contours of the energy of the signal to be coded.

The energy coding is described below in relation to FIGS. 5 and 6, where the ordinate axis corresponds to the energy of the speech signal at code expressed in dB and the abscissa axis at time expressed in frames.

FIG. 5 represents the curve (III) gathering the energy contours of the best aligned representatives and the curve (IV) of the energy contours of the recognized segments separated by * in the figure. A recognized segment of index j is delimited by two points of respective coordinates [E _sd (j); T _sd (j)] and [E _sf (j); T _sf (j)] where E _sd (j) is the start of segment energy and E _sf (j) the end of segment energy, for the instants T _df and T _sf corresponding. The references E _rd (j) and E _rf (j) are used for the energy values of the beginning and the end of a "best representative" and the reference ΔE (j) corresponds to the translation determined for a recognized segment index j.

Energy coding

The method comprises a first step of determining the translation to be carried out.

For this, the difference ΔE (j) existing between the energy value E _rd (j) of the best representative (curve III) and the energy value E _sd at the start of the segment is determined for each start of the “recognized segment”. recognized (curve IV). We obtain a set of values ΔE (j) that we quantify for example uniformly so as to know the translation to be applied during decoding. The quantification is carried out for example using methods known to those skilled in the art.

Speech signal energy decoding

The method notably consists in using the energy contours of the best representatives (curve III) to reconstruct the contours energy of the signal to be coded (curve IV).

For each recognized segment, a first step consists in translating the energy contour of the best representative to make it coincide with the first energy E _rd (j) by applying to it the translation ΔE (j), defined in the coding step by example, to determine the value E _sd (j). After this first translation step, the method comprises a step of modifying the slope of the energy contour of the best representative in order to link the last energy value E _rd (j) of the "best representative" to the first energy E _sd (j + 1) of the next segment of index j + 1.

FIG. 6 represents the curves (VI) and (VII) corresponding respectively to the original energy contour of the speech signal to be coded and of the energy contour decoded after implementation of the steps described previously.

For example, coding the start energies of each segment on 4 bits provides for segmental energy coding a bit rate of around 80 bits / s.

Voice information coding

FIG. 7 represents the temporal evolution of information of binary voicing of four successive segments 35, 36, 37 for the signal to code curve (VII) and for the best representatives (curve VIII) after time alignment by DTW.

Voice information coding

During coding, the method performs a coding step of the voicing information, for example by browsing the temporal evolution of the voicing information of the recognized segments and that of the best aligned representatives (curve VIII) and by coding the differences existing ΔT _k between these two curves. These differences ΔT _k can be: an advance a of the frame, a delay b of the frame, the absence and / or the presence of a reference transition c (k corresponds to the index of an end of a zone of voicing).

For this, it is possible to use a variable length code an example of which is given in table I below, to code the correction to be made to each of the voicing transitions for each recognized segments. All segments with no transition voicing, it is possible to reduce the flow associated with voicing by not coding that the existing voicing transitions in the voicing at code and in the best representatives.

According to this method, the voicing information is coded on approximately 22 bits per second. Example of coding table for voicing transitions: Coded Interpretation 000 Transition to be deleted 001 1 frame shift to Right 010 Offset 1 frame to the Left 011 Offset 2 frames to the Right 100 Offset 2 frames to the Left 101 Insert a transition (a code specifying the location of the transition follows this one) 110 No lag 111 Displacement greater than 3 frames (another code follows this one)

• le taux de voisement en sous-bande, l'analyse de cette information fait appel à une méthode décrite par exemple dans le document suivant : "Multiband Excitation Vocoders", ayant pour auteurs D.W. Griffin and J.S. Lim, IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. 36, no. 8, pp. 1223-1235, 1988 ;
• la fréquence de transition entre une bande basse voisée et une bande haute non-voisée, le codage utilise une méthode telle que décrite dans le document ayant pour auteurs C. Laflamme, R. Salami, R. Matmti, and J-P. Adoul, intitulé "Harmonic Stochastic Excitation (HSX) speech coding below 4 kbits/s", IEEE International Conférence on Acoustics, Speech, and Signal Processing, Atlanta, May 1996, pp. 204-207.

For mixed voicing information such as:

• the voicing rate in sub-band, the analysis of this information uses a method described for example in the following document: "Multiband Excitation Vocoders", having as authors DW Griffin and JS Lim, IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. 36, no. 8, pp. 1223-1235, 1988;
• the transition frequency between a voiced low band and a non-voiced high band, the coding uses a method as described in the document having as authors C. Laflamme, R. Salami, R. Matmti, and JP. Adoul, entitled "Harmonic Stochastic Excitation (HSX) speech coding below 4 kbits / s", IEEE International Conférence on Acoustics, Speech, and Signal Processing, Atlanta, May 1996, pp. 204-207.

In these two cases, the coding of the voicing information comprises also the coding of the variation in the voicing proportion.

Decoding of voicing information

The decoder has the voicing information of the "Best aligned representatives" obtained at the coder level.

The correction is made, for example, as follows:

Each time the end of a voicing zone is detected on the best representatives chosen for the synthesis, the process provides additional information to the decoder which is the correction to be made to this end. The correction can be an advance a or a delay b to bring to this end. This time difference is for example expressed in number of frames in order to obtain the exact position of the end of voicing of the original speech signal. The correction can also take the form of a deletion or insertion of a transition.

Pitch coding

Experience shows that, on speech recordings, the number of voiced areas obtained per second is on average around of 3 or 4. To faithfully account for pitch variations, a way of proceeding consists in transmitting several pitch values by voiced area. In order to limit the speed, instead of transmitting the entire succession of pitch values on a voiced area, the contour of the pitch is approximated by a succession of linear segments.

Pitch coding

• le procédé considère uniquement les valeurs du pitch au début des segments reconnus. Partant de la droite Di joignant les valeurs du pitch aux deux extrémités de la zone voisée, le procédé recherche le début de segment dont la valeur de pitch est la plus éloignée de cette droite, ce qui correspond à une distance d_max. Il compare cette valeur d_max à une valeur seuil d_seuil. Si la distance d_max est supérieure à d_seuil, le procédé décompose la droite initiale Di en deux droites D_i1 et D_i2, en prenant le début du segment trouvé comme nouvelle valeur de pitch à transmettre. Cette opération est réitérée sur ces deux nouvelles zones voisée délimitées par les droites D_i1 et D_i2 jusqu'à ce que la distance d_max trouvée soit inférieure à la distance d_seuil.

For each voiced area of the speech signal, the method includes a step of searching for the values of the pitch to be transmitted. The pitch values at the beginning and at the end of the voiced area are systematically transmitted. The other values to be transmitted are determined as follows:

• the method considers only the values of the pitch at the start of the recognized segments. Starting from the line Di joining the pitch values at the two ends of the voiced area, the method searches for the start of the segment whose pitch value is the furthest from this line, which corresponds to a distance d _max . It compares this d _max value with a threshold d _threshold value. If the distance d _max is greater than d _threshold , the method decomposes the initial line Di into two lines D _i1 and D _i2 , taking the start of the segment found as the new pitch value to be transmitted. This operation is repeated on these two new voiced zones delimited by the lines D _i1 and D _i2 until the distance d _max found is less than the distance d _threshold .

To code the pitch values thus determined, the process uses for example a predictive scalar quantizer on for example 5 bits applied to the logarithm of the pitch.

Prediction is for example the first pitch value of the best representative corresponding to the position of the pitch to be decoded, multiplied by a prediction factor for example between 0 and 1.

According to another way of proceeding, the prediction can be the minimum value of the speech recording to be coded. In this case, this value can be transmitted to the decoder by scalar quantization on by 8-bit example.

The values of the pitches to be transmitted having been determined and coded, the method comprises a step where the temporal spacing is specified, for example in number of frames, between each of these values of pitch. A variable length code allows for example to code these 2-bit spacing on average.

This procedure allows a flow of approximately 65 / bits per second for a maximum distance over the pitch period of 7 samples.

Pitch decoding

The decoding step firstly includes a step of decoding the time spacing between the different pitch values transmitted in order to recover the pitch update times, as well as the pitch value for each of these moments. The value of the pitch for each frames of the voiced area is reconstructed for example by interpolation linear between the transmitted values.

Claims (11)

Speech coding-decoding method using a very low bit rate coder comprising a learning step for identifying “representatives” of the speech signal and a coding step for segmenting the speech signal and determining the “best representative »Associated with each recognized segment characterized in that it comprises at least one coding-decoding step of at least one of the parameters of the prosody of the recognized segments, such as energy and / or pitch and / or voicing and / or the length of the segments, using prosody information from the "best representatives". Method according to claim 1 characterized in that the prosody information of the representatives used is the energy contour or the voicing or the length of the segments or the pitch. Method according to Claim 1, characterized in that it includes a step of coding the length of the recognized segments consisting in coding the difference in length between the length of a recognized segment and the length of the "best representative" multiplied by a given factor . Method according to Claim 1, characterized in that it includes a step of coding the time alignment of the best representatives using the DTW path and searching for the closest neighbor in a shape table. Method according to one of Claims 1 to 4, characterized in that the energy coding step comprises a step of determining for each start of a "recognized segment" the difference ΔE (j) between the energy value E _rd (j) of the “best representative” and the energy value E _sd (j) of the start of the “recognized segment”. Method according to claim 5 characterized in that the energy decoding step comprises for each recognized segment, a first step consisting in translating the energy contour of the best representative by a quantity ΔE (j) to make the first energy E _rd (j) of the "best representative" with the first energy E _sd (j + 1) of the recognized segment of index j + 1. Method according to one of Claims 1 to 4, characterized in that the voicing coding step comprises a step of determining the existing differences ΔT _k for each end of a voicing area of index k between the voicing curve of the recognized segments and that of the best representatives. Method according to Claim 7, characterized in that the decoding step comprises, for each end of a voicing zone of index k, a step of correcting the time position of this end with a corresponding value ΔT _k and / or a step of deleting or inserting a transition. Speech coding-decoding system comprising at least one memory for storing a dictionary comprising a set of representatives of the speech signal, a microprocessor suitable for determine the recognized segments, to reconstruct speech from "Best representatives" and to implement the process steps according to one of claims 1 to 8. System according to Claim 9, characterized in that the dictionary of representatives is common to the coder and to the decoder of the coding-decoding system. Use of the method according to one of claims 1 to 8 or of system according to one of claims 9 and 10 for coding-decoding the speech for bit rates lower than 800 bits / s and preferably lower than 400 bits / s.

Images