WO1998022936A1 - Subword-based speaker verification using multiple classifier fusion, with channel, fusion, model, and threshold adaptation - Google Patents

Subword-based speaker verification using multiple classifier fusion, with channel, fusion, model, and threshold adaptation Download PDF

Info

Publication number
WO1998022936A1
WO1998022936A1 PCT/US1997/021261 US9721261W WO9822936A1 WO 1998022936 A1 WO1998022936 A1 WO 1998022936A1 US 9721261 W US9721261 W US 9721261W WO 9822936 A1 WO9822936 A1 WO 9822936A1
Authority
WO
WIPO (PCT)
Prior art keywords
speech
enrollment
channel
speaker verification
classifier
Prior art date
Application number
PCT/US1997/021261
Other languages
French (fr)
Inventor
Manish Sharma
Xiaoyu Zhang
Richard J. Mammone
Original Assignee
T-Netix, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by T-Netix, Inc. filed Critical T-Netix, Inc.
Priority to EP97950642A priority Critical patent/EP0954854A4/en
Priority to AU53594/98A priority patent/AU5359498A/en
Publication of WO1998022936A1 publication Critical patent/WO1998022936A1/en

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00Speaker identification or verification
    • G10L17/06Decision making techniques; Pattern matching strategies
    • G10L17/10Multimodal systems, i.e. based on the integration of multiple recognition engines or fusion of expert systems
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/04Segmentation; Word boundary detection
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/06Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/065Adaptation
    • G10L15/07Adaptation to the speaker
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00Speaker identification or verification
    • G10L17/04Training, enrolment or model building
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/08Speech classification or search
    • G10L15/10Speech classification or search using distance or distortion measures between unknown speech and reference templates
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/08Speech classification or search
    • G10L15/16Speech classification or search using artificial neural networks
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/08Speech classification or search
    • G10L15/18Speech classification or search using natural language modelling
    • G10L15/1815Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Definitions

  • the invention directed to an automatic speaker verification (ASV) system and method useful for storing and processing voice signals to automatically ascertain the identity of an individual.
  • ASV automatic speaker verification
  • the invention relates to the fields of digital speech processing and speaker recognition.
  • Another systems of the prior art includes placing a photograph of an authorized user on the credit card. At the time of the transaction, the merchant compares the photograph on the card with the face of the person presenting the card. If there appears to be a match, the transaction is allowed to proceed.
  • these prior art methods have serious drawbacks. These systems are manual and consequently prone to human error. Signatures are relatively easy to forge and differences between signatures and photographs may go unnoticed by inattentive merchants. Further, these systems cannot be used with credit card transactions which do not occur in person, for example, transactions which occur via telephone.
  • Voice verification systems sometimes known as automatic speaker verification (ASV) systems, attempt to cure the deficiencies of these prior art methods. These systems attempt to match the voice of the person whose identity is undergoing verification with a known voice.
  • ASV automatic speaker verification
  • One type of voice recognition system is a text-dependent automatic speaker verification system.
  • the text-dependent ASV system requires that the user speak a specific password or phrase (the "password"). This password is determined by the system or by the user during enrollment.
  • the password is constrained to be within a fixed vocabulary, such as a limited number of numerical digits.
  • the limited number of password phrases gives an imposter a higher probability of discovering a person's password, reducing the reliability of the system.
  • Other text-independent ASV systems of the prior art utilize a user- selectable password. In such systems, the user enjoys the freedom to make-up his/her own password with no constraints on vocabulary words or language.
  • the disadvantage of these types of systems is that they increase the processing requirement of the system because it is much more technically challenging to model and verify a voice pattern of an unknown transcript (i.e. a highly variable context).
  • Modeling of speech has been done at the phrase, word, and subword level.
  • HMM Hidden Markov Models
  • ANN Artificial Neural Network
  • noise and background speech/music may be detected and considered as part of the password.
  • Other problems with transmission or communications systems is that channel-specific distortion occurs over channels, such as transducers, telephone lines and telephone equipment which connect users to the system. Further, ASV systems using modeling need to adapt to changes in the user and to prior successful and unsuccessful attempts at verification.
  • What is needed is a user-selectable ASV system in which accuracy is improved over prior ASV systems. What is needed is a word or phrase detector which can identify key portions of spoken password phrases over background noise.
  • the voice print system of the present invention builds and improves upon existing ASV systems.
  • the voice print system of the present invention is a subword-based, text-dependent automatic speaker verification system that embodies the capability of user-selectable passwords with no constraints on the choice of vocabulary words or the language.
  • Automatic blind speech segmentation allows speech to be segmented into subword units without any linguistic knowledge of the password.
  • Subword modeling is performed using a discriminant training- based classifier, namely a Neural Tree Network (NTN).
  • NTN Neural Tree Network
  • the present NTN is a hierarchical classifier that combines the properties of decision trees and feed-forward neural networks.
  • the system also takes advantage of such concepts as multiple classifier fusion and data resampling to successfully boost performance.
  • Channel adaptation removes the nonuniform effects of different environments which lead to varying channel characteristics, such as distortion.
  • Channel adaptation is able to remove the characteristics of the test channel and /or enrollment channel to increase accuracy.
  • Fusion adaptation is used to dynamically change the weight accorded to the individual classifier models, which increases the flexibility of the system. Threshold adaptation dynamically alters the threshold necessary to achieve successful verification. Threshold adaptation is useful to incrementally change false-negative results. Model adaptation gives the system the capability to retrain the classifier models upon the occurrence of subsequent successful verifications.
  • the voice print system can be employed for user validation for telephone services such as cellular phone services and bill-to-third-party phone services. It can also be used for account validation for information system access.
  • All ASV systems include at least two components, an enrollment component and a testing component.
  • the enrollment component is used to store information concerning a user's voice. This information is then compared to the voice undergoing verification (testing) by the test component.
  • the system of the present invention includes inventive enrollment and testing components, as well as a third, "bootstrap" component.
  • the bootstrap component is used to generate data which assists the enrollment component to model the user's voice.
  • An enrollment component is used to characterize a known user's voice and store the characteristics in a database, so that this information is available for future comparisons.
  • the system of the present invention utilizes an improved enrollment process.
  • the user speaks the password, which is sampled by the system.
  • Digital to analog conversion (if necessary) is conducted to obtain digital speech samples.
  • Preprocessing is performed to remove unwanted silence and noise from the voice sample, and to indicate portions of the voice sample which correspond to the user's voice.
  • the transmission channel carrying the user's enrollment voice signal is examined.
  • the characteristics of the enrollment channel are estimated and stored in a database.
  • the database may be indexed by identification information, such as by the user's name, credit card number, account identifier, etc.
  • Feature extraction is then performed to extract features of the user's voice, such as pitch, spectral frequencies, intonations, etc.
  • Feature extraction may also focus, or capture, desired segments of the voice sample and reject other unwanted segments.
  • the feature extraction process generates a number of vectors relating to features of the voice segment.
  • a key word/ key phrase reference template may be generated and stored in a voice print database.
  • the reference template is used during testing to locate the spoken password from extraneous speech or noise.
  • segmentation of the voice segment occurs. Segmentation preferably occurs via automatic blind speech segmentation techniques. Alternatively, segmentation may be performed by older manual or semiautomatic techniques. Segmentation divides the voice sample into a number of subwords. The subwords are used in a modeling process. In recent years, several subword-based speaker verification systems have been proposed.
  • the present invention uses subword modeling and may use any of the known techniques, but preferably uses a discriminant training based classifier.
  • the discriminant training based classifier is called a Neural Tree Network (NTN).
  • the NTN is a hierarchical classifier that combines the properties of decision trees and feed-forward Neural Networks.
  • the system also utilizes the principles of multiple classifier fusion and data resampling.
  • a multiple classifier system is a powerful solution for robust pattern classification because it allows for simultaneous use of arbitrary feature descriptors and classification procedures.
  • the additional classifier used herein is the Gaussian Mixture Model (GMM) classifier.
  • GMM Gaussian Mixture Model
  • Data resampling artificially expands the size of the sample pool and therefore improves the generalizations of the classifiers.
  • One of the embodiments of the classifier fusion and data resampling scheme is a "leave-one-out" data resampling method.
  • a fusion function which is set at a default value and stored in the database, is used to weigh the individual scored classifiers, and to set a threshold value.
  • the threshold value is stored in the database for use in the verification process.
  • the threshold is used when a user is undergoing verification (or testing by the test component). A user is verified as the known user when the threshold is reached or exceeded.
  • the test component is the component which performs the verification. During testing or verification, the system first accepts "test speech" and index information from a user claiming to be the person identified by the index information. Voice data indexed in the database is retrieved and used to process the test speech sample.
  • the user speaks the password into the system.
  • This "test speech" password undergoes preprocessing, as previously described, with respect to the enrollment component.
  • the next step is to perform channel adaptation.
  • Channel adaptation in a preferred embodiment, is performed by removing from the test sample the characteristics of the channel from which the test sample was received. Next, the characteristics of the enrollment channel which were stored by the enrollment component are recalled. The test sample is filtered through the recalled enrollment channel. This type of channel adaptation removes the characteristics from the test channel and supplies the characteristics from the enrollment channel to the test speech so that the test speech matches the transmission channel of the originally enrolled speech.
  • feature extraction is performed on the test sample. This occurs as previously described with respect to the enrollment component. After feature extraction, it is desired to locate, or "spot" the phrases in the test speech and simultaneously avoid areas of background noise.
  • the performance of ASV systems can be significantly degraded by background noise and sounds, such as speech and music, that can lead and/or trail the user's actual spoken password. This is because small differences between the speech and the high volume noise /sounds may lead the preprocessing algorithm to incorrectly treat the background noise and sounds as part of the password. Accordingly, a sample of password including the noise and background sounds will not be recognized.
  • the invention uses a key word/ key phrase spotter to identify the password phrase. After key word/ key phrase spotting, automatic speech segmentation occurs.
  • the automatic speech segmentation is not "blind” segmentation (although "blind” segmentation could be used), but is "force” alignment segmentation. This force segmentation uses the segments previously obtained by the blind segmentation performed in the enrollment component.
  • the test speech is therefore segmented using the segmentation information previously stored.
  • the "force" segmentation results in the identification of subword borders.
  • the subwords undergo multiple classifier fusion.
  • the multiple classifiers of the enrollment component are used to "score" the subword data, and the scores are the fused, or combined.
  • the result of the fusion is a "final score.”
  • the final score is compared to the stored threshold. If the final score exceeds or equals the threshold, the test sample is verified as the user's. If the final score is less than the threshold, the test sample is declared not to be the user's.
  • the final score and date of verification, as well as other related details, may be stored in the database as well.
  • the invention also used a number of adaptation techniques, in addition to channel adaptation. These techniques include fusion adaption, threshold adaption and model adaption.
  • Fusion adaptation modifies the fusion function for n classifiers
  • the fusion function provides more weight to some classifiers than to others. Fusion adaptation dynamically reallocates the weight between the classifiers, preferably by changing a fusion constant, ⁇ .
  • Threshold adaptation dynamically modifies the stored threshold value over time.
  • the initial threshold is determined during enrollment using voice samples. By further using information on the success of recent verification attempts, the decision threshold can be better estimated.
  • Model adaptation changes the models learned during the enrollment component dynamically over time, to track aging of the user's voice. For example, every time a user is successfully verified, the test data may be considered as enrollment data, and the classifier trained and modeled using the steps following automatic blind segmentation (in the enrollment component). Model adaptation effectively increases the number of enrollment samples and improves the accuracy of the system.
  • Bootstrapping is used to generate a pool of speech data representative of the speech of nonspeakers, or "antispeakers.” This data is used during enrollment to train the discriminant training-based classifiers. Bootstrapping involves obtaining voice samples from antispeakers, preprocessing the voice samples (as in the enrollment phase), and inverse channel filtering the preprocessed voice samples. Inverse channel filtering removes the characteristics of the channel on which the antispeaker voice sample is obtained. After inverse channel filtering, feature generation and automatic blind voice segmentation occur, as in the enrollment component. The segments and feature vectors are stored in an antispeaker database for use by the enrollment component.
  • Figure IA is a diagram of a enrollment component of the present invention.
  • Figure IB shows pseudo-code for creating a filter to perform the channel estimation shown in Figure IA.
  • Figure 1C shows pseudo-code for inverting the filter of Figure IB.
  • Figure 2 is a diagram of a testing component of the present invention.
  • Figures 3A and 3B are flow diagrams of a channel adaptation module, shown in Figure 2, of the present invention.
  • FIG 4 is a flow diagram of a key word/ key phrase spotter, shown in Figure 2, of the present invention.
  • Figure 5 is a diagram of an utterance representation in the feature vector domain.
  • Figure 6 is a diagram of dynamic time warping used to obtain a reference template in the key word/ key phrase spotter of Figure 4.
  • Figure 7 is a diagram of dynamic time warping distortion, used to match the reference template of Figure 6 to test speech.
  • Figure 8 is a flow diagram of a fusion adaptation module shown in
  • Figure 9 is a flow diagram of a threshold adaptation module shown in Figure 2.
  • Figure 10 is a diagram of a model adaptation module used in the system of Figures 1 and 2.
  • Figure 11 is a diagram of a bootstrapping component, used to generate antispeaker data in the system of Figure IA.
  • the system of the present invention includes an enrollment component, a testing component, and a bootstrap component.
  • the enrollment component uses antispeaker data to generate and store information concerning a user's voice. The information concerning the user's voice is compared to the voice undergoing verification (testing) by the test component.
  • the bootstrap component is used to provide initial antispeaker data for use by the enrollment component, such that the enrollment component may properly perform its function of generating data concerning the user's voice.
  • the enrollment component is used to store information (using supervised learning) about a known user's voice into a voice print database, so that this information is available for future comparisons.
  • the enrollment component also stores information concerning the channel on which the user provides the speech, the "enrollment channel" into the voice print database.
  • Figure IA shows the enrollment component 10.
  • the first step 20 is to obtain enrollment speech (the password) and to obtain 26 an index , such as the user's name or credit card number.
  • the enrollment speech may be obtained via a receiver, telephone or other sources, and be received from any transmission media, digital or analog, including terrestrial links, land lines, satellite, microwave, etc... More than one sample of enrollment speech should be supplied, each of which is used to generate multiple data sets.
  • the enrollment speech is then analog-to-digital converted 25, if necessary.
  • Analog-to-digital conversion can be performed with standard telephony boards such as those manufactured by Dialogic.
  • a speech encoding method such as ITU G711 standard ⁇ and A law can be used to encode the speech samples.
  • a sampling rate of 8000 Hz is used.
  • the speech may be obtained in digital format, such as from an ISDN transmission. In such a case, a telephony board is used to handle Telco signaling protocol.
  • the computer processing unit for the speaker verification system is an Intel Pentium platform general purpose computer processing unit (CPU) of at least 100 MHz having about 10MB associated RAM memory and a hard or fixed drive as storage.
  • CPU Intel Pentium platform general purpose computer processing unit
  • an additional embodiment could be the Dialogic Antares card.
  • Preprocessing 30 may include one or more of the following techniques, as follows:
  • Silence removal based on an energy histogram.
  • a histogram of frame energies is generated.
  • a threshold energy value is determined based on the assumption that the biggest peak in the histogram at the lower energy region shall correspond to the background silence frame energies. This threshold energy value is used to perform speech versus silence discrimination.
  • the preprocessing is preferably conducted using hamming windowed analysis frames, with 30 millisecond analysis frames and 10 millisecond shifts between adjacent frames. Following preprocessing 30, channel estimation 40 is performed.
  • the voice print database 115 may be RAM, ROM, EPROM, EEPROM, hard disk, CD ROM, writable CD ROM, minidisk, a file server, or other storage device.
  • the distortion present on the channel is considered.
  • a speech signal with frequency spectrum S( ⁇ ) is distorted by a transmission
  • ⁇ c(n) c(n) + x(n)
  • cepstrum is defined as the inverse Fourier transform of the logarithm of short-time spectral magnitude.
  • Time invariant convolution distortion X( ⁇ ) can be eliminated by Cepstral Mean Subtraction (CMS) or Cepstral Mean Normalization (CMN), which is averaging in the cepstral domain, and subtracting the average component.
  • CMS Cepstral Mean Subtraction
  • CNN Cepstral Mean Normalization
  • CMS may be conducted on the cepstral features obtained for the voice signal to remove the distortion of the channel.
  • the cepstral mean may include information other than the estimate of the time-invariant convolutional distortion, such as coarse spectral distribution of the speech itself.
  • Pole filtering attempts to decouple the speech information from the channel information in the cepstral mean. Since cepstrum is the weighted combination of LP poles or spectral components, the effect of individual components on the cepstral mean was examined. It was found that broad band-width components exhibited smoother frequency characteristics corresponding to the "roll-off" of channel distortion, assuming that narrow band-width components in the inverse filter were influenced more by speech characteristics.
  • the narrow band-width LP poles were selectively deflated by broadening their bandwidth and keeping their frequency the same. Therefore, for every frame of speech, the pole filtered cepstral coefficients (PFCC) are computed along with LP-derived cepstral coefficients (LPCC). To achieve cepstral mean subtraction, the mean of the PFCC is subtracted from the LPCC, instead of the regular LPCC mean. This procedure is called pole filtered cepstral mean subtraction (PF-CMS).
  • PF-CMS pole filtered cepstral mean subtraction
  • the procedure outlined in the flow chart of Figure IB is followed.
  • the first block of pseudo-code 42 sets the pole bandwidth threshold.
  • z and Z ⁇ are obtained and LPCC and PFCC are evaluated 44.
  • the PFCC mean may be used to create an LPC filter.
  • FIG. 1C An inverse of this filter may be generated as shown in Figure 1C.
  • the PFCC mean is converted from cepstral to the LPC filter coefficient domain 52.
  • the LPC filter may be inverted 54, and speech passed through the inverted filter 56.
  • the preprocessed speech during enrollment may be inverse-filtered by inverting the filter of Figure 1C, filter (as described below with respect to Figure 3B). While inverse filtering will theoretically remove the enrollment channel distortion, it is preferred to inverse filter the test speech (on the testing channel) and then feed the test speech through the saved enrollment filter, as described below with reference to Figure 3A.
  • feature extraction 50 is performed on the processed speech. Feature extraction may occur after (as shown) or simultaneously with the step of channel estimation 40 (in parallel computing embodiments). Spectral features are represented by speech feature vectors determined within each frame of the processed speech signal. In feature extraction 50, spectral feature vectors can be obtained with conventional methods such as linear predictive (LP) analysis to determine LP cepstral coefficients, Fourier Transform Analysis and filter bank analysis.
  • LP linear predictive
  • One method of feature extraction 50 is disclosed in U.S. Patent 5,522,012, entitled “Speaker Identification and Verification System,” issued on May 28, 1996 and incorporated herein by reference.
  • the result of feature extraction 50 is that vectors representing a template of the password are generated. This may occur as described with respect to Figure 6.
  • This template is stored 60 in the voice print database 115. Following storage of the template 60, the speech is segmented into sub-words for further processing.
  • subword generation 70 is automatic blind speech segmentation, or "Blind Clustering" such as disclosed in U.S. patent application Serial No. 08/827,562 entitled “Blind Clustering of Data With Application to Speech Processing Systems", filed on April 1, 1997, and its corresponding U.S. provisional application no. 60/014,537 entitled
  • a first alternative is the traditional approach, where segmentation and labelling of speech data is performed manually by a trained phonetician using listening and visual cues.
  • a second alternative to subword generation 70 is automatic hierarchical speech segmentation, which involves a multi-level, fine-to- course segmentation. This segmentation can be displayed in a tree-like fashion called dendogram. The initial segmentation is a fine level with the limiting case being a vector equal to one segment. Thereafter, a segment is chosen to be merged with either its left or right neighbor using a similarity measure. This process is repeated until the entire utterance is described by a single segment.
  • a third alternative to subword generation 70 is automatic non- hierarchical speech segmentation. This segmentation method attempts to locate the optimal segment boundaries by using a knowledge engineering- based rule set or by extremizing a distortion or score metric.
  • each sub-word is then modeled 80, 90, preferably with multiple classifier modules.
  • a first neural tree network (NTN) 80 and a second Gaussian mixture model (GMM) 90 are used.
  • the NTN 80 provides a discriminative-based model and the GMM 90 provides one that is based on a statistical measure.
  • a leave-one-out method data resampling scheme 100 is used. Data resampling 100 is performed by creating multiple subsets of the training data, each of which is created by leaving one data sample out at a time. The subsets of the training data are then used to train multiple models of each of the classifiers, which are stored in the voice print database 115.
  • Figure IA shows N models for the NTN classifier 80 and N models for the GMM classifier 90.
  • a enrollment sample such as the 1st sample, is left out of the classifier.
  • the antispeaker database 110 may be RAM, ROM, EPROM, EEPROM, hard disk, CD ROM, a file server, or other storage device.
  • the subword data from the speaker being trained is labeled as enrollment speaker data. Because there is a no linguistic labelling information in the antispeaker database 110, the entire database 110 is searched for the closet subword data from other speakers. This data is labeled the anti-speaker data. The mean vector and covariance matrix of the subword segments obtained from subword generation are used to find the "close" subwords.
  • An anti-speaker module 120 searches the antispeaker database 110 to find the "close” subwords of antispeaker data, which are used in the NTN model 20 . Preferably, 20 "close" subwords are identified.
  • the anti-speaker data in the antispeaker database 110 is either manually created, or created using a "bootstrapping" component, described below with reference to Figure 11.
  • the classifier models 80, 90 are trained by comparing antispeaker data with N-l samples of enrollment speech. Both modules 80, 90 can determine a score for each spectral feature vector of a subword segment. The individual scores of the NTN 80 and GMM 90 modules can be combined, or "fused" by a classifier fusion module 130 to obtain a composite score for the subword. Since these two modeling approaches tend to have errors that are uncorrelated, it has been found that performance improvements can be obtained by fusing the model outputs
  • results of the neural tree network 80 and the Gaussian mixture model 90 are fused 130 using a linear opinion pool, as described below.
  • a log opinion pool or a "voting" mechanism wherein hard decisions from both the NTN and
  • GMM are considered in the voting process.
  • the NTN is a hierarchical classifier that uses a tree architecture to implement a sequential linear decision strategy.
  • the training data for a NTN consists of data from a target speaker, labeled as one, along with data from other speakers (antispeaker data) that are labeled as zero.
  • the NTN learns to distinguish regions of feature space that belong to the target speaker from those that are more likely to belong to an impostor. These regions of feature space correspond to leaves in the NTN that contain probabilities. These probabilities represent the likelihood of the target speaker having generated data that falls within that region of feature space.
  • NTN modeling 220 is performed using the following forward pruning criteria: (a) maximum depth of four, (b) pruned nodes containing less than 10% of data vectors at the root.
  • the NTN scores for individual feature vectors are accumulated across subwords by an NTN scoring algorithm 145.
  • a Gaussian mixture model (GMM) 90 is also used to model each of the subwords.
  • GMM a region of feature space for a target speaker is represented by a set of multivariate Gaussian distributions.
  • the mean vector and covariance matrix of the subword segments are obtained as a by-product of subword generation using automatic blind speech segmentation and are saved as part of the GMM module, as described in U.S. patent application no. 08/827, 562 entitled "Blind Clustering of Data With Application to
  • G p(x) ⁇ PC ⁇ pCx/ ⁇ Cf*).
  • G mixture components are defined by a mixture weight
  • a scoring algorithm 145, 150 is used for each of the NTN and GMM models.
  • the output score (estimated a-posteriori probabilities) of the subword models is combined across all the subwords of the password phrase, so as to yield a composite score for the test utterance.
  • the scoring algorithm 145, 150 for combining the score the subword models 80, 90 can be based on either of the following schemes:
  • Transitional (or durational) probabilities between the subwords can also be used while computing the composite score for the password phrase.
  • the preferred embodiment is (b) subword-average scoring. The result of scoring provides a GMM score and an NTN score, which must then be combined.
  • a classifier fusion module 130 using the linear opinion pool method combines the NTN score and the GMM score.
  • Use of the linear opinion pool is referred to as a data fusion function, because the data from each classifier is "fused," or combined.
  • the data fusion function for n classifiers, S( ⁇ ), is governed by the following linear opinion pool equation: n
  • variable ⁇ is set as a constant (although it may be dynamically adapted as discussed below), and functions to provide more influence on one classifier method as opposed to the other. For example, if the NTN method 80 was found to be more accurate, the first classifier Si would be more important, and would be made greater than 0.5, or its previous value. Preferably, ⁇ is only incremented or decremented by a small amount, £.
  • a threshold value 140 is output and stored in the voice print database 115. The threshold value output 140 is compared to a "final score" in the testing component to determine whether a test user's voice has so closely matched the model that it can be said that the two voices are from the same person.
  • FIG 2 shows a general outline of the testing component 150, which has many features similar to those described with respect to the enrollment component 10 of Figure IA.
  • the testing component 150 is used to determine whether test speech received from a user sufficiently matches identified stored speech characteristics so as to validate that the user is in fact the person whose speech was stored.
  • the test speech and index information 160 is supplied to the test component.
  • the index information is used to recall subword /segmentation information and the threshold value 140 from the voice print database 115.
  • the index information may be any non voice data which identifies the user, such as the user's name, credit card number, etc...
  • Preprocessing 170 may be performed as previously described in the enrollment component 10 ( Figure IA).
  • the same preprocessing 30,170 is conducted on the test speech as was performed during enrollment.
  • channel adaptation 180 adapts the system to the particular enrollment channel and test channel.
  • Channel adaptation 180 includes processing under both the enrollment component 10 and the test component 150.
  • Figures 3A and 3B show alternatives of channel adaptation 180.
  • the enrollment channel is estimated 40 during the enrollment component 10, also shown in Figures 3A and 3B at 300.
  • the enrollment channel estimate is also stored 310 in the voice print database 115 during the enrollment component.
  • the enrollment channel may be estimated and stored using the procedures previously discussed with respect to
  • the test channel is estimated 320 during the testing component.
  • the test channel may be estimated by generating a filter using the procedures previously discussed with respect to Figure IB.
  • the test speech is inverse filtered through the test channel 330.
  • the test speech is passed through the inverse filter of the test channel using the procedure of Figure 1C. This process removes the distortion of the test channel from the test speech.
  • the distortion of the enrollment channel is added to the test speech by filtering the test speech through the enrollment channel. To perform this, the saved enrollment filter is recalled 340 and the test speech is filtered through the enrollment filter 350.
  • the procedure of Figure 3A stores the enrollment data with the enrollment channel distortion during the enrollment component, and then removes the distortion of the test channel and adds the distortion of the original enrollment channel during testing.
  • the enrollment channel is estimated 300 during the enrollment component.
  • the enrollment speech is filtered through an inverse of the enrollment channel filter 360.
  • the enrollment speech is inverse filtered using the techniques previously discussed.
  • the test channel is estimated 370, and an inverse filter constructed using the techniques previously described.
  • the test speech is then filtered through the inverse filter 380.
  • channel adaptation 180 can be used to validate the user since the channel print carries the characteristics of the particular cellular handset of which the speaker is an authorized user, and therefore creates an association between the voice print and the phone channel print.
  • the combination of voice print and phone print ensures that a particular cellular handset can only be used by its registered subscriber.
  • bill-to— third party phone services where the users are allowed to have access to the service from various locations, an authorized user's request for service may be denied due to the phone print mismatch.
  • Channel adaptation 180 provides a solution to this problem. It first removes the phone and channel print of the test environment from the test speech by performing an inverse filtering of the channel. Thereafter, channel adaptation can add the phone and channel print of the training environment to the speech so that it looks as if the verification speech is recorded through the training channel.
  • Channel adaptation 180 in this manner can still be advantageous in cellular fraud control when the channel mismatch is primarily due to variations in the cellular network rather the phone set.
  • the channels can be estimated using techniques such as pole-filtered cepstrum, as described in Figure IB, LP derived cepstrum mean, fast Fourier transform (FFT)- derived cepstrum mean as well as FFT based periodogram of the speech signal.
  • Pole-filtered cepstrum as shown in Figure IB, is the preferred method.
  • feature extraction 190 is performed after preprocessing. Feature extraction 190 may occur immediately after channel adaption 180, or may occur simultaneously with channel adaption 180 (in a multiple processor embodiment). Feature extraction is conducted using any of the techniques previously described with respect to Figure 2, and is preferably conducted using the same techniques performed during enrollment.
  • key word /key phrase spotting 200 is performed.
  • the key- word /key-phrase spotter 200 separates the password (test speech) from the leading and trailing background noise and sounds using a template of the password generated from the enrollment speech.
  • the template is generated by determining the common part in the repeated utterances of the password during the training. During the verification, the template is compared with the entire test utterance to search for the portion of the speech that best matches the password template.
  • the preferred embodiment of the key-word spotter is based on the dynamic time warping (DTW) method.
  • DTW dynamic time warping
  • the feature vector based template of the password is generated by taking an average of the warped version of all the password repetitions.
  • the word template is compared with the entire test utterance. The part of the test utterance that yields the minimum warping distance to the template is regarded as the actual spoken password.
  • the DTW algorithm uses a reference template and test sample.
  • the DTW algorithm attempts to find an optimal path w(.) in the (n,m) plane, such that the path function minimizes the total distance function D.
  • the first step of key word/ key phrase spotting 200 is to select a reference utterance 300.
  • the reference utterance may be one of the utterances collected during enrollment.
  • Figure 5 shows a symbolic representation of any utterance 355 in the feature vector domain.
  • the utterance 355 contains a number of feature vectors 350 and is length
  • the reference utterance 350 could be chosen in a number of ways.
  • a second approach to select the enrollment utterance with median duration is sorted in order of increasing length, for example [ L 2 , L 4 , Li, L 3 ] (using the utterances of Figure 6).
  • L 4 is the reference utterance of because it is the median value.
  • a third approach is to select an utterance with a duration closest to the average duration.
  • a fourth approach is to select an utterance with minimum combined distortion with respect to the other utterances.
  • a distortion matrix D is created: d d 12 d 13 d 14
  • the distortion, d ⁇ is the DTW distortion between utterance i and utterance j.
  • the utterance-wise combined distortion is:
  • the utterance selected is the one with minimum dj.
  • FIG. 6 shows four utterances 370, 380, 390 and 400 of different lengths (L l7 L 2 , L 3 , L 4 ) "warped" onto the reference utterance 350, which produces four "warped” utterances 410, 420, 430 and 440 of length L ref .
  • the four warped utterances 410, 420, 430 and 440 are averaged 320 to form a reference template 450 of length L ref .
  • Figure 7 shows the test utterance 460 of length L test , and the reference template 450 of length L ref being slid across the test utterance 460.
  • the distortion at each point 470 is also shown on the graph of Figure 7.
  • the key word/ key phrase frames 480 are selected 340 from the portion of the test utterance 460 which produces the least DTW distortion 470.
  • the key word/ key phrase has been spotted and processing can now move to subword generation.
  • subword generation 210 occurs. Because segmentation was already performed during the enrollment component, subword generation 210 in the testing component is performed based on the subwords /segment model computed in the enrollment phase 10. As previously described with respect to Figure IA, during the enrollment component 10 GMM modeling 90 was performed. The GMM modeling 90 is used in the test component subword generation 210 to "force align" the test phrase into segments corresponding to the previously formed subwords. Using the subword GMMs as reference models, Viterbi or Dynamic programming (DP) based algorithms are used to locate the optimal boundaries for the subword segments.
  • DP Dynamic programming
  • scoring 240, 250 using the techniques previously described with respect to Figure 2 (i.e. multiple classifiers such as GMM 230 and NTN 220) is performed on the subwords. Scoring using the NTN and GMM classifiers 220, 230 is disclosed in U.S. patent application Ser. No. 08/ , entitled "Model Adaption System
  • NTN scoring algorithm 240 and a GMM scoring algorithm 250 are used, as previously described with respect to Figure IA, to provide a GMM score and a NTN score to the classifier fusion module 260.
  • the classifier fusion module 260 outputs a "final score” 270.
  • the “final score” 270 is then compared 280 to the threshold value 140. If the “final score” 270 is equal to or greater than the threshold value 140 obtained during enrollment, the user is verified. If the “final score” 270 is less than the threshold value 140 then the user is not verified or permitted to complete the transaction requiring verification.
  • the present invention also employs a number of additional adaptations, in addition to channel adaptation 180.
  • the multiple classifier system uses a classifier fusion module 130, 260 incorporating a fusion function to advantageously combine the strength of the individual classifiers and avoid their weakness.
  • the fusion function that is set during the enrollment may not be optimal for the testing in that every single classifier may have its own preferred operating conditions. Therefore, as the operating environment changes, the fusion function changes accordingly in order to achieve the optimal results for fusion. Also, for each user, one classifier may perform better than the other. An adaptable fusion function provides more weight to the better classifier. Fusion adaptation uses predetermined knowledge of the performance of the classifiers to update the fusion function so that the amount of emphasis being put on a particular classifier varies from time to time based on its performance.
  • a fusion adaptation module 290 is connected to the classifier fusion module 280.
  • the fusion adaptation module 290 changes the constant, ⁇ , in the linear pool data fusion function described previously with respect to Figure 2, which is: n
  • two classifiers are used (NTN 80, 220 and GMM 90, 230) and s 1 is the score of the first classifier and s 2 is the score of
  • the fusion adaptation module 290 dynamically changes ⁇ to weigh either the NTN (si) or GMM (s 2 ) classifiers more than the other, depending on which classifier turns out to be more indicative of a true verification.
  • the fusion adaptation module 290 is shown in Figure 8.
  • the first step of fusion adaptation is to determine whether the fusion adaptation criteria are met 500.
  • the fusion adaptation criteria are met in any number of circumstances, which may be dependent on the type of voice verification system being implemented in a particular application. For example, the fusion adaptation criteria may be met in the following cases: after every five (or another predetermined number of) successful verifications, if the scores of the classifiers (i.e.
  • the GMM score and the NTN score differ by more than a predetermined amount, if it is found that the true user was not verified for a predetermined number of attempts (false-negative results), if it is found that an imposter was verified for one or more attempts (false-positive results), or during a time period (i.e. the first week of use by a particular user).
  • a time period i.e. the first week of use by a particular user.
  • the system is not working at its optimal efficiency and needs further adaptation to improve. Because fusion adaptation may effect the amount of false-positive results and the amount of false-negative results, the inclusion criteria may be made dependent on the amount of tolerance which is deemed acceptable for these possibilities.
  • the classifier closest to the threshold is assessed. Specifically, it is determined whether
  • S ⁇ is closer to the threshold value than s 2 510. If Si is closer to the threshold than s 2 , the constant, ⁇ , is increased 520 to provide more weight to s-, . If not, then ⁇ is decreased 530 to provide more weight to s 2 .
  • the amount that ⁇ is increased or decreased depends on the particular application, and may be a constant amount or a variable amount, depending on the amount of error in the system, the amount of tolerance for false-positive results, the amount of tolerance for false-negative results, etc...
  • the modified constant, ⁇ is then stored 540 in the voice print database 115 for use in the testing component.
  • the weighing of the different classifier models may be dynamically changed to adapt the system by changing the fusion constant, ⁇ .
  • Threshold adaptation adapts the threshold value in response to prior final scores.
  • Threshold adaptation module 295 is shown in Figure 2.
  • Figure 9 shows an example of threshold adaptation 600.
  • the threshold adaptation inclusion criteria are assessed 610. If the inclusion criteria are not met, the process ends and no threshold adaptation takes place.
  • the inclusion criteria may vary depending on the particular application, as described previously with respect to Figure 8 (fusion adaptation) and Figure 10 (model adaptation). It is also to be noted that threshold adaptation 600 may affect the amount of false-positive results and the amount of false-negative results. Therefore, the inclusion criteria may be made dependent on the amount of tolerance which is deemed acceptable for these possibilities.
  • Threshold adaptation 600 analyzes one or more prior final scores and adapts the threshold in response to the analysis.
  • one or more previous final scores which may include the present final score, are recalled (if necessary) and analyzed 620.
  • the analysis may be simple or complex.
  • the analysis may be the average or mean of all the successful verifications, or, preferably, the analysis may be the average or mean of one or more unsuccessful verifications in which it is known that false-negative results were obtained.
  • the new threshold is calculated 630 from this analysis. For example, if the average of four unsuccessful verifications in which it is known that false-negative results were obtained is 0.4, then the new threshold may be set to 0.3.
  • the analyzation 620 and calculation 630 of a new threshold may depend on the amount of tolerance which is deemed acceptable for false-negative and false-positive results. For example, if false positive results are somewhat tolerable, then the new threshold may be set to the lowest final score in which it is known that a false-negative result occurred.
  • Model adaptation adapts the classifier models to subsequent successful verifications.
  • Figure 10 shows one example of model adaptation 540.
  • the inclusion criteria for model adaptation are assessed 550. If the inclusion criteria are not met, the process ends and no model adaptation takes place.
  • the inclusion criteria may vary depending on the particular application, as described previously with respect to Figure 8 and fusion adaptation. It is also to be noted that model adaptation 540 may effect the amount of false-positive results and the amount of false- negative results because there is a small chance that the successful test speech is a false-positive. Therefore, the inclusion criteria may be made dependent on the amount of tolerance which is deemed acceptable for these possibilities.
  • Model adaptation 540 uses the test speech as enrollment speech, and retrains the classifier models 80, 90, 220 and 230 with the additional data sample (test speech) in a re-enrollment process that is transparent to the user. Therefore, one of the inclusion criteria is that verification is successful for each test speech used in model adaptation.
  • the number of samples and their corresponding enrollment speech is identified 560, or recalled from the voice print database 115 if necessary.
  • the previously stored enrollment speech, extracted features, and segmentation (subword) information is recalled from the voice print database 115, along with previous successful test speech, and its associated extracted features.
  • the previous four test speech samples in which successful verification occurred may be recalled from the voice print database 115, as well as the four initial training samples of enrollment speech. This doubles the number of training samples from four to eight.
  • a "forget" factor may be built into the system, the forget feature may discard one or more samples. For example, only the latest eight samples may be remembered, or only the initial four enrollment speech samples and the newest four successful test samples. The number of samples, and which samples are used, may depend on the tolerance for false-positive results and false-negative results, since the model adaptation will change these probabilities.
  • Model adaptation 540 operates in conjunction with the classifiers shown in Figures 1 and 2.
  • Model adaptation 540 may also occur as described in copending
  • Model adaptation 540 is useful for adjusting the system to adapt to gradual changes in the user's voice over long periods of time.
  • Fusion adaptation 290 model adaptation 540, and threshold adaptation 600 all may effect the number and probability of obtaining false- negative and false-positive results, so should be used with caution.
  • These adaptive techniques may be used in combination with channel adaptation
  • Model adaptation is more dramatic than threshold adaptation or fusion adaptation, which both provide incremental changes to the system.
  • the voiceprint database 115 may or may not be coresident with the antispeaker database 110.
  • Voice print data stored in the voice print database may include: enrollment channel estimate, classifier models, list of antispeakers selected for training, fusion constant, theshold value, normalized segment durations, and /or other intermediate scores or authorization results used for adaptation.
  • the antispeaker database 110 must be initially be filled with antispeaker data.
  • the initial antispeaker data may be generated via artifical simulation techniques, or can be obtained from a pre-existing database, or the database may be "bootstrapped" with data by the bootstrapping component.
  • Figure 11 shows a bootstrapping component 700.
  • the bootstrapping component 700 first obtains antispeaker speech 710, and then preprocess the speech 720 as previously described with respect to Figure IA.
  • the antispeaker speech may be phrases from any number of speakers who will not be registered in the database as users.
  • the antispeaker speech is inverse-channel filtered 730 to remove the effects of the antispeaker channel as described with respect to Figures 1 and 2.
  • the processed and filtered antispeaker speech then undergoes feature extraction 770.
  • the feature extraction may occur as previously described with respect to Figure IA.
  • the antispeaker speech undergoes subword generation 750, using the techniques previously described with respect to Figure IA.
  • the preferable method of sub-word generation is automatic blind speech segmentation, discussed previously with respect to Figure IA.
  • the sub-words are then registered as antispeaker data 760 in the database.
  • the bootstrapping component initializes the database with antispeaker data which then may be compared to enrollment data in the enrollment component.
  • the present invention provides for an accurate and reliable automatic speaker verification, which uses adaptive techniques to improve performance.
  • a key word/ key phrase spotter 200 and automatic blind speech segmentation improve the usefulness of the system.
  • Adaptation schemes adapt the ASV to changes in success /failures and to changes in the user by using channel adaptation 180, model adaptation 540, fusion adaptation 290, and threshold adaptation 600.

Abstract

The subword-based, text-dependent automatic speaker verification voiceprint system embodies a capability of user-selectable passwords with no constraints on the choice of vocabulary words or the language. Automatic blind speech segmentation allows speech to be segmented into subword units (210) without any linguistic knowledge of the password. Subword modeling is performed using multiple classifiers (240, 250). The system also takes advantage of such concepts as multiple classifier fusion (260) and data resampling to successfully boost the performance. Key word/key phrase spotting (200) is used to optimally locate the password phrase. Numerous adaptation techniques increase the flexibility of the base system, and include: channel adaptation (180), fusion adaptation (290), model adaptation (220, 230) and threshold adaptation (295).

Description

SUBWORD-BASED SPEAKER VERIFICAΗON USING MULTIPLE CLASSIFIER FUSION, WITH CHANNEL, FUSION, MODEL, AND THRESHOLD ADAPTATION
CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority from Provisional Application 60/031,639, filed November 22, 1996, entitled Voice Print System.
BACKGROUND OF THE INVENTION
The invention directed to an automatic speaker verification (ASV) system and method useful for storing and processing voice signals to automatically ascertain the identity of an individual.
1. Field of The Invention. The invention relates to the fields of digital speech processing and speaker recognition.
2. Description of Related Art
In many situations it is desired to verify the identity of a person, such as a consumer. For example, in credit card transactions, it is important to confirm that a consumer presenting a credit card (or credit card number) to a merchant is authorized to use the credit card. Currently, the identity of the consumer is manually verified by the merchant. The back of the credit card contains a signature strip, which the consumer signs upon credit card issuance. The actual signature of the consumer at the time of sale is compared to the signature on the back of the credit card by the merchant. If in the merchant's judgement, the signatures match, the transaction is allowed to proceed.
Another systems of the prior art includes placing a photograph of an authorized user on the credit card. At the time of the transaction, the merchant compares the photograph on the card with the face of the person presenting the card. If there appears to be a match, the transaction is allowed to proceed. However, these prior art methods have serious drawbacks. These systems are manual and consequently prone to human error. Signatures are relatively easy to forge and differences between signatures and photographs may go unnoticed by inattentive merchants. Further, these systems cannot be used with credit card transactions which do not occur in person, for example, transactions which occur via telephone.
Voice verification systems, sometimes known as automatic speaker verification (ASV) systems, attempt to cure the deficiencies of these prior art methods. These systems attempt to match the voice of the person whose identity is undergoing verification with a known voice.
One type of voice recognition system is a text-dependent automatic speaker verification system. The text-dependent ASV system requires that the user speak a specific password or phrase (the "password"). This password is determined by the system or by the user during enrollment. However, in most text-dependent ASV systems, the password is constrained to be within a fixed vocabulary, such as a limited number of numerical digits. The limited number of password phrases gives an imposter a higher probability of discovering a person's password, reducing the reliability of the system. Other text-independent ASV systems of the prior art utilize a user- selectable password. In such systems, the user enjoys the freedom to make-up his/her own password with no constraints on vocabulary words or language. The disadvantage of these types of systems is that they increase the processing requirement of the system because it is much more technically challenging to model and verify a voice pattern of an unknown transcript (i.e. a highly variable context).
Modeling of speech has been done at the phrase, word, and subword level. In recent years, several subword-based speaker verification systems have been proposed using either Hidden Markov Models ("HMM") or Artificial Neural Network ("ANN") references. Modeling at the subword level expands the versatility of the system. Moreover, it is also conjectured that the variations in speaking styles among different speakers can be better captured by modeling at the subword level.
Another challenge posed under real-life operating environments is that noise and background speech/music may be detected and considered as part of the password. Other problems with transmission or communications systems is that channel-specific distortion occurs over channels, such as transducers, telephone lines and telephone equipment which connect users to the system. Further, ASV systems using modeling need to adapt to changes in the user and to prior successful and unsuccessful attempts at verification.
What is needed are reliable systems and methods for automatic speaker verification of user selectable phrases.
What is needed is a user-selectable ASV system in which accuracy is improved over prior ASV systems. What is needed is a word or phrase detector which can identify key portions of spoken password phrases over background noise.
What is needed is channel adaptation to adapt a system in response to signals received over different channels.
What is needed is fusion adaptation to adapt a system in response to previous errors and successes.
What is needed is threshold adaptation to adapt a system in response to previous errors and successes.
What is needed is model adaptation to adapt underlying a system model components in response to previous successes.
SUMMARY OF THE INVENTION
The voice print system of the present invention builds and improves upon existing ASV systems. The voice print system of the present invention is a subword-based, text-dependent automatic speaker verification system that embodies the capability of user-selectable passwords with no constraints on the choice of vocabulary words or the language. Automatic blind speech segmentation allows speech to be segmented into subword units without any linguistic knowledge of the password. Subword modeling is performed using a discriminant training- based classifier, namely a Neural Tree Network (NTN). The present NTN is a hierarchical classifier that combines the properties of decision trees and feed-forward neural networks. The system also takes advantage of such concepts as multiple classifier fusion and data resampling to successfully boost performance.
Key word/ key phrase spotting is used to optimally locate the password. Channel adaptation removes the nonuniform effects of different environments which lead to varying channel characteristics, such as distortion. Channel adaptation is able to remove the characteristics of the test channel and /or enrollment channel to increase accuracy.
Fusion adaptation is used to dynamically change the weight accorded to the individual classifier models, which increases the flexibility of the system. Threshold adaptation dynamically alters the threshold necessary to achieve successful verification. Threshold adaptation is useful to incrementally change false-negative results. Model adaptation gives the system the capability to retrain the classifier models upon the occurrence of subsequent successful verifications.
The voice print system can be employed for user validation for telephone services such as cellular phone services and bill-to-third-party phone services. It can also be used for account validation for information system access. All ASV systems include at least two components, an enrollment component and a testing component. The enrollment component is used to store information concerning a user's voice. This information is then compared to the voice undergoing verification (testing) by the test component. The system of the present invention includes inventive enrollment and testing components, as well as a third, "bootstrap" component. The bootstrap component is used to generate data which assists the enrollment component to model the user's voice. 1. Enrollment Summary.
An enrollment component is used to characterize a known user's voice and store the characteristics in a database, so that this information is available for future comparisons. The system of the present invention utilizes an improved enrollment process. During enrollment, the user speaks the password, which is sampled by the system. Digital to analog conversion (if necessary) is conducted to obtain digital speech samples. Preprocessing is performed to remove unwanted silence and noise from the voice sample, and to indicate portions of the voice sample which correspond to the user's voice.
Next, the transmission channel carrying the user's enrollment voice signal is examined. The characteristics of the enrollment channel are estimated and stored in a database. The database may be indexed by identification information, such as by the user's name, credit card number, account identifier, etc...
Feature extraction is then performed to extract features of the user's voice, such as pitch, spectral frequencies, intonations, etc... Feature extraction may also focus, or capture, desired segments of the voice sample and reject other unwanted segments. The feature extraction process generates a number of vectors relating to features of the voice segment.
Using the feature vectors, a key word/ key phrase reference template may be generated and stored in a voice print database. The reference template is used during testing to locate the spoken password from extraneous speech or noise. Next, segmentation of the voice segment occurs. Segmentation preferably occurs via automatic blind speech segmentation techniques. Alternatively, segmentation may be performed by older manual or semiautomatic techniques. Segmentation divides the voice sample into a number of subwords. The subwords are used in a modeling process. In recent years, several subword-based speaker verification systems have been proposed. The present invention uses subword modeling and may use any of the known techniques, but preferably uses a discriminant training based classifier. The discriminant training based classifier is called a Neural Tree Network (NTN). The NTN is a hierarchical classifier that combines the properties of decision trees and feed-forward Neural Networks. The system also utilizes the principles of multiple classifier fusion and data resampling. A multiple classifier system is a powerful solution for robust pattern classification because it allows for simultaneous use of arbitrary feature descriptors and classification procedures. The additional classifier used herein is the Gaussian Mixture Model (GMM) classifier. In the event that only a small amount of data is available for modeling a speaker, the resulting classifier is very likely to be biased. Data resampling artificially expands the size of the sample pool and therefore improves the generalizations of the classifiers. One of the embodiments of the classifier fusion and data resampling scheme is a "leave-one-out" data resampling method.
A fusion function, which is set at a default value and stored in the database, is used to weigh the individual scored classifiers, and to set a threshold value. The threshold value is stored in the database for use in the verification process. Thus, enrollment produces a voice print database containing an index (such as the user's name or credit card number), along with enrollment channel data, classifier models, feature vectors, segmentation information, multiple trained classifier data, fusion constant, and a recognition threshold.
The threshold is used when a user is undergoing verification (or testing by the test component). A user is verified as the known user when the threshold is reached or exceeded.
2. Test Component Summary.
The test component is the component which performs the verification. During testing or verification, the system first accepts "test speech" and index information from a user claiming to be the person identified by the index information. Voice data indexed in the database is retrieved and used to process the test speech sample.
During verification, the user speaks the password into the system. This "test speech" password undergoes preprocessing, as previously described, with respect to the enrollment component. The next step is to perform channel adaptation.
Channel adaptation, in a preferred embodiment, is performed by removing from the test sample the characteristics of the channel from which the test sample was received. Next, the characteristics of the enrollment channel which were stored by the enrollment component are recalled. The test sample is filtered through the recalled enrollment channel. This type of channel adaptation removes the characteristics from the test channel and supplies the characteristics from the enrollment channel to the test speech so that the test speech matches the transmission channel of the originally enrolled speech.
After channel adaption, feature extraction is performed on the test sample. This occurs as previously described with respect to the enrollment component. After feature extraction, it is desired to locate, or "spot" the phrases in the test speech and simultaneously avoid areas of background noise.
The performance of ASV systems can be significantly degraded by background noise and sounds, such as speech and music, that can lead and/or trail the user's actual spoken password. This is because small differences between the speech and the high volume noise /sounds may lead the preprocessing algorithm to incorrectly treat the background noise and sounds as part of the password. Accordingly, a sample of password including the noise and background sounds will not be recognized. To combat the effects of background noise, the invention uses a key word/ key phrase spotter to identify the password phrase. After key word/ key phrase spotting, automatic speech segmentation occurs. Preferably the automatic speech segmentation is not "blind" segmentation (although "blind" segmentation could be used), but is "force" alignment segmentation. This force segmentation uses the segments previously obtained by the blind segmentation performed in the enrollment component. The test speech is therefore segmented using the segmentation information previously stored. The "force" segmentation results in the identification of subword borders. The subwords undergo multiple classifier fusion.
The multiple classifiers of the enrollment component are used to "score" the subword data, and the scores are the fused, or combined. The result of the fusion is a "final score." The final score is compared to the stored threshold. If the final score exceeds or equals the threshold, the test sample is verified as the user's. If the final score is less than the threshold, the test sample is declared not to be the user's. The final score and date of verification, as well as other related details, may be stored in the database as well. The invention also used a number of adaptation techniques, in addition to channel adaptation. These techniques include fusion adaption, threshold adaption and model adaption.
Fusion adaptation modifies the fusion function for n classifiers,
S(α). The fusion function provides more weight to some classifiers than to others. Fusion adaptation dynamically reallocates the weight between the classifiers, preferably by changing a fusion constant, α.
Threshold adaptation dynamically modifies the stored threshold value over time. The initial threshold is determined during enrollment using voice samples. By further using information on the success of recent verification attempts, the decision threshold can be better estimated.
Model adaptation changes the models learned during the enrollment component dynamically over time, to track aging of the user's voice. For example, every time a user is successfully verified, the test data may be considered as enrollment data, and the classifier trained and modeled using the steps following automatic blind segmentation (in the enrollment component). Model adaptation effectively increases the number of enrollment samples and improves the accuracy of the system.
3. "Bootstrapping" Component Summary.
Bootstrapping is used to generate a pool of speech data representative of the speech of nonspeakers, or "antispeakers." This data is used during enrollment to train the discriminant training-based classifiers. Bootstrapping involves obtaining voice samples from antispeakers, preprocessing the voice samples (as in the enrollment phase), and inverse channel filtering the preprocessed voice samples. Inverse channel filtering removes the characteristics of the channel on which the antispeaker voice sample is obtained. After inverse channel filtering, feature generation and automatic blind voice segmentation occur, as in the enrollment component. The segments and feature vectors are stored in an antispeaker database for use by the enrollment component.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure IA is a diagram of a enrollment component of the present invention.
Figure IB shows pseudo-code for creating a filter to perform the channel estimation shown in Figure IA.
Figure 1C shows pseudo-code for inverting the filter of Figure IB.
Figure 2 is a diagram of a testing component of the present invention.
Figures 3A and 3B are flow diagrams of a channel adaptation module, shown in Figure 2, of the present invention.
Figure 4 is a flow diagram of a key word/ key phrase spotter, shown in Figure 2, of the present invention.
Figure 5 is a diagram of an utterance representation in the feature vector domain. Figure 6 is a diagram of dynamic time warping used to obtain a reference template in the key word/ key phrase spotter of Figure 4.
Figure 7 is a diagram of dynamic time warping distortion, used to match the reference template of Figure 6 to test speech. Figure 8 is a flow diagram of a fusion adaptation module shown in
Figure 2.
Figure 9 is a flow diagram of a threshold adaptation module shown in Figure 2.
Figure 10 is a diagram of a model adaptation module used in the system of Figures 1 and 2.
Figure 11 is a diagram of a bootstrapping component, used to generate antispeaker data in the system of Figure IA.
DESCRIPTION OF THE PREFERRED EMBODIMENT(S)
The system of the present invention includes an enrollment component, a testing component, and a bootstrap component. The enrollment component uses antispeaker data to generate and store information concerning a user's voice. The information concerning the user's voice is compared to the voice undergoing verification (testing) by the test component. The bootstrap component is used to provide initial antispeaker data for use by the enrollment component, such that the enrollment component may properly perform its function of generating data concerning the user's voice.
1. Enrollment Component- Detailed Description.
The enrollment component is used to store information (using supervised learning) about a known user's voice into a voice print database, so that this information is available for future comparisons. In the preferred embodiment, the enrollment component also stores information concerning the channel on which the user provides the speech, the "enrollment channel" into the voice print database. Figure IA shows the enrollment component 10. As shown, the first step 20 is to obtain enrollment speech (the password) and to obtain 26 an index , such as the user's name or credit card number. The enrollment speech may be obtained via a receiver, telephone or other sources, and be received from any transmission media, digital or analog, including terrestrial links, land lines, satellite, microwave, etc... More than one sample of enrollment speech should be supplied, each of which is used to generate multiple data sets. Preferably, four enrollment samples are supplied and processed. The enrollment speech is then analog-to-digital converted 25, if necessary. Analog-to-digital conversion can be performed with standard telephony boards such as those manufactured by Dialogic. A speech encoding method such as ITU G711 standard μ and A law can be used to encode the speech samples. Preferably, a sampling rate of 8000 Hz is used. Alternatively, the speech may be obtained in digital format, such as from an ISDN transmission. In such a case, a telephony board is used to handle Telco signaling protocol.
In the preferred embodiment, the computer processing unit for the speaker verification system is an Intel Pentium platform general purpose computer processing unit (CPU) of at least 100 MHz having about 10MB associated RAM memory and a hard or fixed drive as storage. Alternatively, an additional embodiment could be the Dialogic Antares card.
The digital enrollment speech is then pre-processed 30. Preprocessing 30 may include one or more of the following techniques, as follows:
• Digital filtering using pre-emphasis. In this case, a digital filter H(z) = 1- z-1 is used, where is set between .9 and 1.0.
• Silence removal using energy and zero-crossing statistics. The success of this technique is primarily based on finding a short interval which is guaranteed to be background silence (generally found a few milliseconds at the beginning of the utterance, before the speaker actually starts recording). Thresholds are set using the silence region statistics, in order to discriminate speech and silence frames.
• Silence removal based on an energy histogram. In this method, a histogram of frame energies is generated. A threshold energy value is determined based on the assumption that the biggest peak in the histogram at the lower energy region shall correspond to the background silence frame energies. This threshold energy value is used to perform speech versus silence discrimination.
• DC Bias removal to remove DC bias introduced by analog-to- digital hardware or other components. The mean value of the signal is computed over the entire voice sample and then is subtracted from the voice samples.
In the preferred embodiment, the following preprocessing is conducted: silence removal using the energy histogram technique (20 bins in histogram), signal mean removal to remove DC bias, and signal pre- emphasis using filter = .95. The preprocessing is preferably conducted using hamming windowed analysis frames, with 30 millisecond analysis frames and 10 millisecond shifts between adjacent frames. Following preprocessing 30, channel estimation 40 is performed.
This procedure stores characteristics of the enrollment channel in the voice print database 115. The voice print database 115 may be RAM, ROM, EPROM, EEPROM, hard disk, CD ROM, writable CD ROM, minidisk, a file server, or other storage device. In order to estimate the channel, the distortion present on the channel is considered.
A speech signal with frequency spectrum S(ω) is distorted by a transmission
channel with frequency response X(ω). The frequency spectrum of the distorted
Λ speech S(ω) is given as:
Λ
S(ω) = S(ω)X(ω) If the logarithm and inverse Fourier transform (f Λ )of the magnitude of both sides of the equation are taken, the following equation results:
Λ 7Λ log( I S(ω) I ) = 7Λ log( I S(ω) I ) + log( I X(ω) I )
then, in cepstral domain:
Λ c(n) = c(n) + x(n)
Because cepstrum is defined as the inverse Fourier transform of the logarithm of short-time spectral magnitude. Time invariant convolution distortion X(ω) can be eliminated by Cepstral Mean Subtraction (CMS) or Cepstral Mean Normalization (CMN), which is averaging in the cepstral domain, and subtracting the average component. For example:
Λ ccιean (n) = c(n) + x(n) - mean [c(n)]
Thus, CMS may be conducted on the cepstral features obtained for the voice signal to remove the distortion of the channel.
While CMS may be used alone to remove the effects of the channel distortion, the cepstral mean may include information other than the estimate of the time-invariant convolutional distortion, such as coarse spectral distribution of the speech itself. Pole filtering attempts to decouple the speech information from the channel information in the cepstral mean. Since cepstrum is the weighted combination of LP poles or spectral components, the effect of individual components on the cepstral mean was examined. It was found that broad band-width components exhibited smoother frequency characteristics corresponding to the "roll-off" of channel distortion, assuming that narrow band-width components in the inverse filter were influenced more by speech characteristics. Thus, the narrow band-width LP poles were selectively deflated by broadening their bandwidth and keeping their frequency the same. Therefore, for every frame of speech, the pole filtered cepstral coefficients (PFCC) are computed along with LP-derived cepstral coefficients (LPCC). To achieve cepstral mean subtraction, the mean of the PFCC is subtracted from the LPCC, instead of the regular LPCC mean. This procedure is called pole filtered cepstral mean subtraction (PF-CMS).
To perform PF-CMS, the procedure outlined in the flow chart of Figure IB is followed. With reference to Figure IB, the first block of pseudo-code 42 sets the pole bandwidth threshold. Next z and Z{ are obtained and LPCC and PFCC are evaluated 44. This allows the mean of the PFCC vectors to be computed 46, which may be saved 48 as a channel estimate in the voice print database 115. The PFCC mean may be used to create an LPC filter.
An inverse of this filter may be generated as shown in Figure 1C. First, the PFCC mean is converted from cepstral to the LPC filter coefficient domain 52. Next, the LPC filter may be inverted 54, and speech passed through the inverted filter 56.
Although not preferred, the preprocessed speech during enrollment may be inverse-filtered by inverting the filter of Figure 1C, filter (as described below with respect to Figure 3B). While inverse filtering will theoretically remove the enrollment channel distortion, it is preferred to inverse filter the test speech (on the testing channel) and then feed the test speech through the saved enrollment filter, as described below with reference to Figure 3A.
After preprocessing 30, feature extraction 50 is performed on the processed speech. Feature extraction may occur after (as shown) or simultaneously with the step of channel estimation 40 (in parallel computing embodiments). Spectral features are represented by speech feature vectors determined within each frame of the processed speech signal. In feature extraction 50, spectral feature vectors can be obtained with conventional methods such as linear predictive (LP) analysis to determine LP cepstral coefficients, Fourier Transform Analysis and filter bank analysis. One method of feature extraction 50 is disclosed in U.S. Patent 5,522,012, entitled "Speaker Identification and Verification System," issued on May 28, 1996 and incorporated herein by reference. A preferred method for obtaining spectral feature vectors is performing a 12th order TP-PFC obtained from a 12th order linear prediction (LP) with alpha = 0.7.
The result of feature extraction 50 is that vectors representing a template of the password are generated. This may occur as described with respect to Figure 6. This template is stored 60 in the voice print database 115. Following storage of the template 60, the speech is segmented into sub-words for further processing.
The preferred technique for subword generation 70 is automatic blind speech segmentation, or "Blind Clustering" such as disclosed in U.S. patent application Serial No. 08/827,562 entitled "Blind Clustering of Data With Application to Speech Processing Systems", filed on April 1, 1997, and its corresponding U.S. provisional application no. 60/014,537 entitled
"Blind Speech Segmentation", filed on April 2, 1996, both of which are herein incorporated by reference. During enrollment in the speaker verification system, the automatic blind speech segmentation determines the number of subwords in the password and the location of optimal subword boundaries. Additionally, the subword durations are normalized by the total duration of the voice phrase and stored in the voice print database 115 for subsequent use during testing (force segmentation).
Alternative approaches to subword generation 70 which generate segments of speech may be used with the present invention. A first alternative is the traditional approach, where segmentation and labelling of speech data is performed manually by a trained phonetician using listening and visual cues.
A second alternative to subword generation 70 is automatic hierarchical speech segmentation, which involves a multi-level, fine-to- course segmentation. This segmentation can be displayed in a tree-like fashion called dendogram. The initial segmentation is a fine level with the limiting case being a vector equal to one segment. Thereafter, a segment is chosen to be merged with either its left or right neighbor using a similarity measure. This process is repeated until the entire utterance is described by a single segment.
A third alternative to subword generation 70 is automatic non- hierarchical speech segmentation. This segmentation method attempts to locate the optimal segment boundaries by using a knowledge engineering- based rule set or by extremizing a distortion or score metric.
After subwords are obtained, each sub-word is then modeled 80, 90, preferably with multiple classifier modules. Preferably a first neural tree network (NTN) 80 and a second Gaussian mixture model (GMM) 90 are used. The NTN 80 provides a discriminative-based model and the GMM 90 provides one that is based on a statistical measure. In a preferred embodiment a leave-one-out method data resampling scheme 100 is used. Data resampling 100 is performed by creating multiple subsets of the training data, each of which is created by leaving one data sample out at a time. The subsets of the training data are then used to train multiple models of each of the classifiers, which are stored in the voice print database 115. Thus, Figure IA shows N models for the NTN classifier 80 and N models for the GMM classifier 90. For model #1 of the NTN classifier, a enrollment sample, such as the 1st sample, is left out of the classifier.
In order to train an NTN model 80 for a given speaker, it is necessary to appropriately label the subword data available in the antispeaker database 110. The antispeaker database 110 may be RAM, ROM, EPROM, EEPROM, hard disk, CD ROM, a file server, or other storage device.
The subword data from the speaker being trained is labeled as enrollment speaker data. Because there is a no linguistic labelling information in the antispeaker database 110, the entire database 110 is searched for the closet subword data from other speakers. This data is labeled the anti-speaker data. The mean vector and covariance matrix of the subword segments obtained from subword generation are used to find the "close" subwords. An anti-speaker module 120 searches the antispeaker database 110 to find the "close" subwords of antispeaker data, which are used in the NTN model 20 . Preferably, 20 "close" subwords are identified. The anti-speaker data in the antispeaker database 110 is either manually created, or created using a "bootstrapping" component, described below with reference to Figure 11.
Because a "leave-one-out" system 100 is employed with multiple (N) samples, the classifier models 80, 90 are trained by comparing antispeaker data with N-l samples of enrollment speech. Both modules 80, 90 can determine a score for each spectral feature vector of a subword segment. The individual scores of the NTN 80 and GMM 90 modules can be combined, or "fused" by a classifier fusion module 130 to obtain a composite score for the subword. Since these two modeling approaches tend to have errors that are uncorrelated, it has been found that performance improvements can be obtained by fusing the model outputs
130. In the preferred embodiment, the results of the neural tree network 80 and the Gaussian mixture model 90 are fused 130 using a linear opinion pool, as described below. However, other ways of combining the data can be used with the present invention including a log opinion pool or a "voting" mechanism, wherein hard decisions from both the NTN and
GMM are considered in the voting process.
With continued reference to Figure IA, one of modules used to model the subword segments of the user password is an NTN module 80. The NTN is a hierarchical classifier that uses a tree architecture to implement a sequential linear decision strategy. Specifically, the training data for a NTN consists of data from a target speaker, labeled as one, along with data from other speakers (antispeaker data) that are labeled as zero. The NTN learns to distinguish regions of feature space that belong to the target speaker from those that are more likely to belong to an impostor. These regions of feature space correspond to leaves in the NTN that contain probabilities. These probabilities represent the likelihood of the target speaker having generated data that falls within that region of feature space. In the preferred embodiment, NTN modeling 220 is performed using the following forward pruning criteria: (a) maximum depth of four, (b) pruned nodes containing less than 10% of data vectors at the root. The NTN scores for individual feature vectors are accumulated across subwords by an NTN scoring algorithm 145. The functioning of
NTN networks with respect to speaker recognition is disclosed in K.R. Farrell, R.J. Mammone, and K.T. Assaleh, "Speaker Recognition using Neural Networks and Conventional Classifiers", IEEE Trans. Speech and Audio Processing, 2(1), part 2 (1994), and U.S. Patent Application 08/159,397, filed November 29, 1993, entitled "Rapidly Trainable Neural
Tree Network", U. S. Patent Application Serial No. 08/479,012 entitled "Speaker Verification System," U.S. Patent Application no. 08/827, 562 entitled "Blind Clustering of Data With Application to Speech Processing Systems", filed on April 1, 1997, and its corresponding U.S. Provisional Application no. 60/014,537 entitled "Blind Speech Segmentation", filed on
April 2, 1996, each of which is incorporated herein by reference in its entirety.
As discussed previously, a Gaussian mixture model (GMM) 90 is also used to model each of the subwords. In the GMM, a region of feature space for a target speaker is represented by a set of multivariate Gaussian distributions. In the preferred embodiment, the mean vector and covariance matrix of the subword segments are obtained as a by-product of subword generation using automatic blind speech segmentation and are saved as part of the GMM module, as described in U.S. patent application no. 08/827, 562 entitled "Blind Clustering of Data With Application to
Speech Processing Systems", filed on April 1, 1997, and its corresponding U.S. provisional application no. 60/014,537 entitled "Blind Speech Segmentation", filed on April 2, 1996, both of which are herein incorporated by reference. The GMM probability distribution function is expressed as:
G p(x) = ∑PCω pCx/μ^ Cf*). Each of the G mixture components is defined by a mixture weight
P(ω.) and multi-dimensional normal distribution function p(x/μ. , σ 2), where μ{ is the mean vector and 0 is the covarience matrix. In the preferred embodiment, the normal distribution is constrained to have a diagonal covariance matrix defined by the vector Ot . The PDF is used to produce the sub-word GMM score.
A scoring algorithm 145, 150 is used for each of the NTN and GMM models. The output score (estimated a-posteriori probabilities) of the subword models is combined across all the subwords of the password phrase, so as to yield a composite score for the test utterance.
The scoring algorithm 145, 150 for combining the score the subword models 80, 90 can be based on either of the following schemes:
(a) PHRASE- AVER AGE: Averaging the output scores for the vectors over the entire phrase,
(b) SUBWORD-AVERAGE: Average the score of vectors within a subword, before averaging the (averaged) subword scores, and
(c) SUBWORD-WEIGHING: Same as (b) subword-average scoring, but the (averaged) subword scores are weighted in the final averaging process.
Transitional (or durational) probabilities between the subwords can also be used while computing the composite score for the password phrase. The preferred embodiment is (b) subword-average scoring. The result of scoring provides a GMM score and an NTN score, which must then be combined.
In the preferred embodiment, a classifier fusion module 130 using the linear opinion pool method combines the NTN score and the GMM score. Use of the linear opinion pool is referred to as a data fusion function, because the data from each classifier is "fused," or combined. The data fusion function for n classifiers, S(α), is governed by the following linear opinion pool equation: n
S( ) = G jSi i=l
In this equation S( ) is the probability of the combined system, α are weights, and s;(α) is the probability output by the ith classifier, and n is the number of classifiers; { is between zero and one and the sum of all cij's is equal to one. If two classifiers are used (n=2), Si is the score of the first classifier and s2 is the score of the second classifier. In this instance the equation becomes:
S = αs1 + (l-α)s2
The variable α is set as a constant (although it may be dynamically adapted as discussed below), and functions to provide more influence on one classifier method as opposed to the other. For example, if the NTN method 80 was found to be more accurate, the first classifier Si would be more important, and would be made greater than 0.5, or its previous value. Preferably, α is only incremented or decremented by a small amount, £. Once the variables in the fusion equation are known, a threshold value 140 is output and stored in the voice print database 115. The threshold value output 140 is compared to a "final score" in the testing component to determine whether a test user's voice has so closely matched the model that it can be said that the two voices are from the same person.
2. Testing Component- Detailed Description.
Figure 2 shows a general outline of the testing component 150, which has many features similar to those described with respect to the enrollment component 10 of Figure IA. The testing component 150 is used to determine whether test speech received from a user sufficiently matches identified stored speech characteristics so as to validate that the user is in fact the person whose speech was stored. First, the test speech and index information 160 is supplied to the test component. The index information is used to recall subword /segmentation information and the threshold value 140 from the voice print database 115. The index information may be any non voice data which identifies the user, such as the user's name, credit card number, etc...
After obtaining the test speech and index information, the test speech is preprocessed 170. Preprocessing 170 may be performed as previously described in the enrollment component 10 (Figure IA). Preferably, the same preprocessing 30,170 is conducted on the test speech as was performed during enrollment.
The fact that a speaker's model is conventionally built using enrollment speech that is recorded under a specific, controlled environment implies that the model carries not only the voice print but also the channel print. Therefore, following preprocessing, channel adaptation 180 is performed. Channel adaptation 180 adapts the system to the particular enrollment channel and test channel. Channel adaptation 180 includes processing under both the enrollment component 10 and the test component 150. Figures 3A and 3B show alternatives of channel adaptation 180. As previously mentioned with respect to Figure IA, the enrollment channel is estimated 40 during the enrollment component 10, also shown in Figures 3A and 3B at 300. As shown in Figure 3A, the enrollment channel estimate is also stored 310 in the voice print database 115 during the enrollment component. The enrollment channel may be estimated and stored using the procedures previously discussed with respect to
Figure IA. As shown in Figure 3A, the test channel is estimated 320 during the testing component. The test channel may be estimated by generating a filter using the procedures previously discussed with respect to Figure IB. After generating the filter, the test speech is inverse filtered through the test channel 330. To achieve this, the test speech is passed through the inverse filter of the test channel using the procedure of Figure 1C. This process removes the distortion of the test channel from the test speech. Now, the distortion of the enrollment channel is added to the test speech by filtering the test speech through the enrollment channel. To perform this, the saved enrollment filter is recalled 340 and the test speech is filtered through the enrollment filter 350.
The procedure of Figure 3A stores the enrollment data with the enrollment channel distortion during the enrollment component, and then removes the distortion of the test channel and adds the distortion of the original enrollment channel during testing. As an alternative, shown in Figure 3B, it may be desired to remove the enrollment channel distortion during enrollment, and then remove the test channel distortion during testing.
As shown in Figure 3B, the enrollment channel is estimated 300 during the enrollment component. Next, the enrollment speech is filtered through an inverse of the enrollment channel filter 360. In other words, the enrollment speech is inverse filtered using the techniques previously discussed. During the testing phase the test channel is estimated 370, and an inverse filter constructed using the techniques previously described. The test speech is then filtered through the inverse filter 380.
Using either channel adaptation technique, the system adapts to account for the channel distortion on the enrollment channel and on the test channel. It has been found that the technique shown in Figure 3A is preferred. In the scenario of cellular fraud control, the concept of channel adaptation 180 can be used to validate the user since the channel print carries the characteristics of the particular cellular handset of which the speaker is an authorized user, and therefore creates an association between the voice print and the phone channel print. The combination of voice print and phone print ensures that a particular cellular handset can only be used by its registered subscriber. However, in application such as bill-to— third party phone services where the users are allowed to have access to the service from various locations, an authorized user's request for service may be denied due to the phone print mismatch.
Channel adaptation 180 provides a solution to this problem. It first removes the phone and channel print of the test environment from the test speech by performing an inverse filtering of the channel. Thereafter, channel adaptation can add the phone and channel print of the training environment to the speech so that it looks as if the verification speech is recorded through the training channel.
Channel adaptation 180 in this manner can still be advantageous in cellular fraud control when the channel mismatch is primarily due to variations in the cellular network rather the phone set. The channels can be estimated using techniques such as pole-filtered cepstrum, as described in Figure IB, LP derived cepstrum mean, fast Fourier transform (FFT)- derived cepstrum mean as well as FFT based periodogram of the speech signal. Pole-filtered cepstrum, as shown in Figure IB, is the preferred method.
Referring to Figure 2, feature extraction 190 is performed after preprocessing. Feature extraction 190 may occur immediately after channel adaption 180, or may occur simultaneously with channel adaption 180 (in a multiple processor embodiment). Feature extraction is conducted using any of the techniques previously described with respect to Figure 2, and is preferably conducted using the same techniques performed during enrollment.
Following feature extraction 190, key word /key phrase spotting 200 is performed.
The key- word /key-phrase spotter 200 separates the password (test speech) from the leading and trailing background noise and sounds using a template of the password generated from the enrollment speech. The template is generated by determining the common part in the repeated utterances of the password during the training. During the verification, the template is compared with the entire test utterance to search for the portion of the speech that best matches the password template.
The preferred embodiment of the key-word spotter is based on the dynamic time warping (DTW) method. During the enrollment, the feature vector representation of all the training repetitions of the password are warped to the shortest one using the DTW method. The feature vector based template of the password is generated by taking an average of the warped version of all the password repetitions. During the verification, the word template is compared with the entire test utterance. The part of the test utterance that yields the minimum warping distance to the template is regarded as the actual spoken password. The DTW algorithm uses a reference template and test sample. The reference template is represented as R, were R(n) n=l, • • • , N represents the multidimensional feature vector at the nth analysis frame of the reference speech signal. The test template is represented as T, where T(m) m=l, • • • , M represents the multidimensional feature vector at the mth analysis frame of the test speech utterance. The DTW algorithm attempts to find an optimal path w(.) in the (n,m) plane, such that the path function minimizes the total distance function D.
The path function is: w(n) = m.
N The total distance function D is of the form: D = ∑ d(R( ), T(w(n))). n=l
The expression d(R(n), T(w(n))) is the local distance between the
frame n of the reference pattern and the frame m=w(n) of the test pattern.
To perform key word/ key phrase spotting 200, a template is trained and then the phrase spotted on the test utterance, as shown in Figure 4. As shown in Figure 4, the first step of key word/ key phrase spotting 200 is to select a reference utterance 300. The reference utterance may be one of the utterances collected during enrollment. Figure 5 shows a symbolic representation of any utterance 355 in the feature vector domain. The utterance 355 contains a number of feature vectors 350 and is length
Lx, where x is the number of feature vectors.
The reference utterance 350 could be chosen in a number of ways.
The preferred method is to select the enrollment utterance with the minimum duration. selection = arg min {Li} i = 1, 2, 3, 4... where Lj is the utterance length i of utterances
1=1, 2, 3, 4.
A second approach to select the enrollment utterance with median duration. Thus the enrollment utterances of lengths Lι,L2,L3,L4 are sorted in order of increasing length, for example [ L2, L4, Li, L3] (using the utterances of Figure 6). In this case, L4 is the reference utterance of because it is the median value.
A third approach is to select an utterance with a duration closest to the average duration.
L avg = (Lx + L2 + L3 + L4)/4 selection = arg min | Lavg - L^ | i = 1, 2, 3, 4 i
A fourth approach is to select an utterance with minimum combined distortion with respect to the other utterances. A distortion matrix D is created: d d12 d13 d14
D = d21 d22 d23 d24 d3i d32 d33 d34 d41 d42 d43 d44 The distortion, d^ is the DTW distortion between utterance i and utterance j. The utterance-wise combined distortion is:
4 4 di = ∑ dij + ∑ dji 1 = 1, 2, 3, 4 j=l j=l
The utterance selected is the one with minimum dj.
Selection = arg min di i
After selecting 300 a reference utterance 350, all the remaining utterances are "warped" 310 onto the reference utterance 350 using the
DTW algorithm. Figure 6 shows four utterances 370, 380, 390 and 400 of different lengths (Ll7 L2, L3, L4) "warped" onto the reference utterance 350, which produces four "warped" utterances 410, 420, 430 and 440 of length Lref. The four warped utterances 410, 420, 430 and 440 are averaged 320 to form a reference template 450 of length Lref.
During verification, as shown in Figure 4 and Figure 7, the reference template 450 is slid across the test utterance 460, and the DTW distortion 470 is computed for each point 330. Figure 7 shows the test utterance 460 of length Ltest, and the reference template 450 of length Lref being slid across the test utterance 460. The distortion at each point 470 is also shown on the graph of Figure 7.
Next, with reference to Figure 4 and Figure 7, the key word/ key phrase frames 480 are selected 340 from the portion of the test utterance 460 which produces the least DTW distortion 470. The key word/ key phrase has been spotted and processing can now move to subword generation.
Referring back to Figure 2, after key word/ key phrase spotting, automatic subword generation 210 occurs. Because segmentation was already performed during the enrollment component, subword generation 210 in the testing component is performed based on the subwords /segment model computed in the enrollment phase 10. As previously described with respect to Figure IA, during the enrollment component 10 GMM modeling 90 was performed. The GMM modeling 90 is used in the test component subword generation 210 to "force align" the test phrase into segments corresponding to the previously formed subwords. Using the subword GMMs as reference models, Viterbi or Dynamic programming (DP) based algorithms are used to locate the optimal boundaries for the subword segments. Additionally, the normalized subword duration (stored during enrollment) is used as a constraint for force alignment since it provides stability to the algorithm. Speech segmentation using force alignment is disclosed in U.S. patent application no. 08/827, 562 entitled "Blind Clustering of Data With Application to Speech Processing Systems", filed on April 1, 1997, and its corresponding U.S. provisional application no. 60/014,537 entitled "Blind Speech Segmentation", filed on April 2, 1996, both of which are herein incorporated by reference in their entirety.
After subword generation 210 is performed, scoring 240, 250 using the techniques previously described with respect to Figure 2 (i.e. multiple classifiers such as GMM 230 and NTN 220) is performed on the subwords. Scoring using the NTN and GMM classifiers 220, 230 is disclosed in U.S. patent application Ser. No. 08/ , entitled "Model Adaption System
And Method For Speaker Verification," filed on November 3, 1997 by Kevin Farrell and William Mistretta, U.S. patent application no. 08/827, 562 entitled "Blind Clustering of Data With Application to Speech Processing Systems", filed on April 1, 1997, and its corresponding U.S. provisional application no. 60/014,537 entitled "Blind Speech
Segmentation", filed on April 2, 1996, each of which is herein incorporated by reference in its entirety.
An NTN scoring algorithm 240 and a GMM scoring algorithm 250 are used, as previously described with respect to Figure IA, to provide a GMM score and a NTN score to the classifier fusion module 260.
With continued reference to Figure 2, the classifier fusion module 260 outputs a "final score" 270. The "final score" 270 is then compared 280 to the threshold value 140. If the "final score" 270 is equal to or greater than the threshold value 140 obtained during enrollment, the user is verified. If the "final score" 270 is less than the threshold value 140 then the user is not verified or permitted to complete the transaction requiring verification.
The present invention also employs a number of additional adaptations, in addition to channel adaptation 180.
As previously described, the multiple classifier system uses a classifier fusion module 130, 260 incorporating a fusion function to advantageously combine the strength of the individual classifiers and avoid their weakness. However, the fusion function that is set during the enrollment may not be optimal for the testing in that every single classifier may have its own preferred operating conditions. Therefore, as the operating environment changes, the fusion function changes accordingly in order to achieve the optimal results for fusion. Also, for each user, one classifier may perform better than the other. An adaptable fusion function provides more weight to the better classifier. Fusion adaptation uses predetermined knowledge of the performance of the classifiers to update the fusion function so that the amount of emphasis being put on a particular classifier varies from time to time based on its performance.
As shown in Figure 2, a fusion adaptation module 290 is connected to the classifier fusion module 280. The fusion adaptation module 290 changes the constant, α, in the linear pool data fusion function described previously with respect to Figure 2, which is: n
S(α) = XoqSi i=l In the present invention two classifiers are used (NTN 80, 220 and GMM 90, 230) and s1 is the score of the first classifier and s2 is the score of
the second classifier. In this instance the equation becomes: S = as + (l-a)s
The fusion adaptation module 290 dynamically changes α to weigh either the NTN (si) or GMM (s2) classifiers more than the other, depending on which classifier turns out to be more indicative of a true verification.
The fusion adaptation module 290 is shown in Figure 8. The first step of fusion adaptation is to determine whether the fusion adaptation criteria are met 500. The fusion adaptation criteria are met in any number of circumstances, which may be dependent on the type of voice verification system being implemented in a particular application. For example, the fusion adaptation criteria may be met in the following cases: after every five (or another predetermined number of) successful verifications, if the scores of the classifiers (i.e. the GMM score and the NTN score) differ by more than a predetermined amount, if it is found that the true user was not verified for a predetermined number of attempts (false-negative results), if it is found that an imposter was verified for one or more attempts (false-positive results), or during a time period (i.e. the first week of use by a particular user). In these cases, the system is not working at its optimal efficiency and needs further adaptation to improve. Because fusion adaptation may effect the amount of false-positive results and the amount of false-negative results, the inclusion criteria may be made dependent on the amount of tolerance which is deemed acceptable for these possibilities.
As shown in Figure 8, if the inclusion criteria are met, the classifier closest to the threshold is assessed. Specifically, it is determined whether
Sη is closer to the threshold value than s2510. If Si is closer to the threshold than s2, the constant, α, is increased 520 to provide more weight to s-, . If not, then α is decreased 530 to provide more weight to s2. The amount that α is increased or decreased depends on the particular application, and may be a constant amount or a variable amount, depending on the amount of error in the system, the amount of tolerance for false-positive results, the amount of tolerance for false-negative results, etc... The modified constant, α, is then stored 540 in the voice print database 115 for use in the testing component. Thus, the weighing of the different classifier models may be dynamically changed to adapt the system by changing the fusion constant, α.
Threshold adaptation adapts the threshold value in response to prior final scores. Threshold adaptation module 295 is shown in Figure 2. Figure 9 shows an example of threshold adaptation 600. First, the threshold adaptation inclusion criteria are assessed 610. If the inclusion criteria are not met, the process ends and no threshold adaptation takes place. The inclusion criteria may vary depending on the particular application, as described previously with respect to Figure 8 (fusion adaptation) and Figure 10 (model adaptation). It is also to be noted that threshold adaptation 600 may affect the amount of false-positive results and the amount of false-negative results. Therefore, the inclusion criteria may be made dependent on the amount of tolerance which is deemed acceptable for these possibilities. Threshold adaptation 600 analyzes one or more prior final scores and adapts the threshold in response to the analysis.
With continued reference to Figure 9, after assessing the inclusion criteria 610 , one or more previous final scores, which may include the present final score, are recalled (if necessary) and analyzed 620. The analysis may be simple or complex. For example, the analysis may be the average or mean of all the successful verifications, or, preferably, the analysis may be the average or mean of one or more unsuccessful verifications in which it is known that false-negative results were obtained. The new threshold is calculated 630 from this analysis. For example, if the average of four unsuccessful verifications in which it is known that false-negative results were obtained is 0.4, then the new threshold may be set to 0.3. The analyzation 620 and calculation 630 of a new threshold may depend on the amount of tolerance which is deemed acceptable for false-negative and false-positive results. For example, if false positive results are somewhat tolerable, then the new threshold may be set to the lowest final score in which it is known that a false-negative result occurred.
After calculating the a new threshold, the new threshold is saved 640 for use in current or future testing. Model adaptation adapts the classifier models to subsequent successful verifications. Figure 10 shows one example of model adaptation 540. First, the inclusion criteria for model adaptation are assessed 550. If the inclusion criteria are not met, the process ends and no model adaptation takes place. The inclusion criteria may vary depending on the particular application, as described previously with respect to Figure 8 and fusion adaptation. It is also to be noted that model adaptation 540 may effect the amount of false-positive results and the amount of false- negative results because there is a small chance that the successful test speech is a false-positive. Therefore, the inclusion criteria may be made dependent on the amount of tolerance which is deemed acceptable for these possibilities. Model adaptation 540 uses the test speech as enrollment speech, and retrains the classifier models 80, 90, 220 and 230 with the additional data sample (test speech) in a re-enrollment process that is transparent to the user. Therefore, one of the inclusion criteria is that verification is successful for each test speech used in model adaptation.
With continued reference to Figure 10, after assessing the inclusion criteria 550, the number of samples and their corresponding enrollment speech is identified 560, or recalled from the voice print database 115 if necessary. The previously stored enrollment speech, extracted features, and segmentation (subword) information is recalled from the voice print database 115, along with previous successful test speech, and its associated extracted features.
For example, the previous four test speech samples in which successful verification occurred may be recalled from the voice print database 115, as well as the four initial training samples of enrollment speech. This doubles the number of training samples from four to eight. In order to limit the number of training samples, a "forget" factor may be built into the system, the forget feature may discard one or more samples. For example, only the latest eight samples may be remembered, or only the initial four enrollment speech samples and the newest four successful test samples. The number of samples, and which samples are used, may depend on the tolerance for false-positive results and false-negative results, since the model adaptation will change these probabilities.
After identifying the number of samples and associated speech 560, training the multiple models occur as previously described with respect to
Figure IA. Therefore the remaining portion of Figure 10 corresponds to the multiple classifier model and leave-one-out methodology of Figure IA. A new threshold value will be obtained by the retrained model. Model adaptation 540, as shown in Figure 10, operates in conjunction with the classifiers shown in Figures 1 and 2.
Model adaptation 540 may also occur as described in copending
Provisional Application Serial No. , entitled "Model Adaption
System And Method For Speaker Verification," filed on November 3, 1997 by Kevin Farrell and William Mistretta. Model adaptation 540 is useful for adjusting the system to adapt to gradual changes in the user's voice over long periods of time.
Fusion adaptation 290, model adaptation 540, and threshold adaptation 600 all may effect the number and probability of obtaining false- negative and false-positive results, so should be used with caution. These adaptive techniques may be used in combination with channel adaptation
180, or each other, either simultaneously or at different authorization occurrences. Model adaptation is more dramatic than threshold adaptation or fusion adaptation, which both provide incremental changes to the system.
The voiceprint database 115 may or may not be coresident with the antispeaker database 110. Voice print data stored in the voice print database may include: enrollment channel estimate, classifier models, list of antispeakers selected for training, fusion constant, theshold value, normalized segment durations, and /or other intermediate scores or authorization results used for adaptation.
3. "Bootstrapping" Component. Because the enrollment component 10 uses the "closest" antispeaker data to generate the threshold value 140, the antispeaker database 110 must be initially be filled with antispeaker data. The initial antispeaker data may be generated via artifical simulation techniques, or can be obtained from a pre-existing database, or the database may be "bootstrapped" with data by the bootstrapping component.
Figure 11 shows a bootstrapping component 700. The bootstrapping component 700 first obtains antispeaker speech 710, and then preprocess the speech 720 as previously described with respect to Figure IA. The antispeaker speech may be phrases from any number of speakers who will not be registered in the database as users. Next, the antispeaker speech is inverse-channel filtered 730 to remove the effects of the antispeaker channel as described with respect to Figures 1 and 2. As shown in Figure 11, the processed and filtered antispeaker speech then undergoes feature extraction 770. The feature extraction may occur as previously described with respect to Figure IA. Next, the antispeaker speech undergoes subword generation 750, using the techniques previously described with respect to Figure IA. The preferable method of sub-word generation is automatic blind speech segmentation, discussed previously with respect to Figure IA. The sub-words are then registered as antispeaker data 760 in the database. Thus, the bootstrapping component initializes the database with antispeaker data which then may be compared to enrollment data in the enrollment component.
The present invention provides for an accurate and reliable automatic speaker verification, which uses adaptive techniques to improve performance. A key word/ key phrase spotter 200 and automatic blind speech segmentation improve the usefulness of the system. Adaptation schemes adapt the ASV to changes in success /failures and to changes in the user by using channel adaptation 180, model adaptation 540, fusion adaptation 290, and threshold adaptation 600.
The foregoing description of the present invention has been presented for purposes of illustration and description which is not intended to limit the invention to the specific embodiments described. Consequently, variations and modifications commensurate with the above teachings, and within the skill and knowledge of the relevant art, are part of the scope of the present invention. It is intended that the appended claims be construed to include alternative embodiments to the extent permitted by law.

Claims

CLAIMS:
1. An automatic speaker verification system comprising: a receiver, the receiver obtaining enrollment speech over an enrollment channel; a means, connected to the receiver, for developing an estimate of the enrollment channel; a first storage device, connected to the receiver, for storing the enrollment channel estimate; a means for extracting predetermined features of the enrollment speech; a means, operably connected to the extracting means, for segmenting the predetermined features of the enrollment speech, wherein the features are segmented into a plurality of subwords; at least one classifier, connected to the segmenting means, wherein the classifier models the pluraility of subwords and outputs one or more classifier scores.
2. The automatic speaker verification system of claim 1, further comprising: an analog to digital converter, connected to the receiver, for providing the obtained enrollment speech in a digital format.
3. The automatic speaker verification system of claim 1, wherein at least one classifier is a one neural tree network classifier.
4. The automatic speaker verification system of claim 1, wherein at least one classifier is a Gaussian mixture model classifier.
5. The automatic speaker verification system of claim 1, wherein the classifiers comprise: at least one Gaussian mixture model classifier, the Gaussian mixture model classifer resulting in a first classifier score; and at least one neural tree network classifier, the neural tree network classifer resulting in a second classifier score.
6. The automatic speaker verification system of claim 1, further comprising a means, connected to the classifier, for fusing the classifier scores, wherein the fusing means weighs the scores from the classifier models with a fusion constant and combines the weighted scores resulting in a final score for the combined system.
7. The automatic speaker verification system of claim 6, wherein the weighted scores are variable and are dynamically adapted.
8. The automatic speaker verification system of claim 1, wherein the segmenting means generates subwords using automatic blind speech segmentation.
9. The automatic speaker verification system of claim 1, wherein the estimating means comprises a means for creating a filter representing characteristics of the enrollment channel.
10. The automatic speaker verification system of claim 1, further comprising a second storage device, connected to the classifier, for storing the one or more classifier scores.
11. An automatic speaker verification method, comprising the steps of: obtaining enrollment speech over an enrollment channel; storing an estimate of the enrollment channel; extracting predetermined features of the enrollment speech; segmenting the enrollment speech, wherein the enrollment speech is segmented into a plurality of subwords; and modelling the pluraility of subwords using one or more classifier models resulting in an output of one of more classifier scores.
12. The automatic speaker verification method of claim 11, further comprising the steps of: digitizing the obtained enrollment speech; and preprocessing the digitized enrollment speech.
13. The automatic speaker verification method of claim 11, wherein the step of modeling comprises the step of scoring at least one neural tree network classifier.
14. The automatic speaker verification method of claim 11, wherein the step of modeling further comprises the steps of: . scoring at least one Gaussian mixture model classifier, the Gaussian mixture model classifer resulting in a first classifier score; scoring at least one neural tree network classifier, the Gaussian mixture model classifer resulting in a second classifier score; fusing the first and second classifier scores.
15. The automatic speaker verification method of claim 11, further comprising the steps of: weighing the scores from the classifier models with a fusion constant; and combining the weighted scores resulting in a final score for the combined system.
16. The automatic speaker verification method of claim 15, wherein the fusion constant is variable and is dynamically adapted.
17. The automatic speaker verification method of claim 11, wherein the step of segmenting comprises generating subwords using automatic blind speech segmentation.
18. The automatic speaker verification method of claim 11, wherein the step of storing an estimate of the enrollment channel comprises the step of creating a filter representing characteristics of the enrollment channel.
19. An automatic speaker verification method, comprising the steps of: obtaining enrollment speech over an enrollment channel; storing an estimate of the enrollment channel, the estimate being a filter representing characteristics of the enrollment channel; receiving test speech over a testing channel; inverse filtering the test speech to create filtered test speech; recalling the estimate of the enrollment channel filtering the filtered test speech through the recalled estimate of the enrollment channel to create enrollment filtered test speech; and determining whether the enrollment filtered test speech comes from the same person as the enrollment speech.
20. The automatic speaker verification method of claim 19, wherein the step of storing an estimate of the enrollment channel comprises the step of creating a filter representing characteristics of the enrollment channel.
21. The automatic speaker verification method of claim 19, wherein the step of inverse filtering the test speech comprises the step of creating a filter representing inverse characteristics of the testing channel.
22. An automatic speaker verification method, comprising the steps of: obtaining enrollment speech over an enrollment channel; inverse filtering the enrollment speech to create inverse filtered enrollment speech; receiving test speech over a testing channel; inverse filtering the test speech to create inverse filtered test speech; and determining whether the inverse filtered test speech comes from the same person as the inverse filtered enrollment speech.
23. The automatic speaker verification method of claim 22, wherein the step of inverse filtering the enrollment speech comprises the step of creating a filter representing inverse characteristics of the enrollment channel.
24. The automatic speaker verification method of claim 22, wherein the step of inverse filtering the test speech comprises the step of creating a filter representing inverse characteristics of the testing channel.
25. An automatic speaker verification method, including the steps of: obtaining two or more samples of enrollment speech; processing each sample of enrollment speech to form corresponding utterances; obtaining test speech; identifying one or more key words /key phrases in the test speech, including the steps of: selecting a reference utterance from one of the utterances; warping the remaining samples of the enrollment speech to the reference utterance; averaging one or more of the warped utterances to generate a reference template; calculating a dynamic time warp distortion for the reference template and test speech; and choosing a portion of the test utterance which has the least dynamic time warp distortion; and comparing the identified key word/ key phrases to the enrollment speech to determine whether the test speech and enrollment speech are from the same person.
26. The automatic speaker verification method of claim 25, wherein the step of selecting a reference utterance comprises the step of: choosing the utterance with minimum duration.
27. The automatic speaker verification method of claim 25, wherein the step of selecting a reference utterance comprises the step of: choosing an utterance with median duration.
28. The automatic speaker verification method of claim 25, wherein the step of selecting a reference utterance comprises the step of: choosing an utterance with a duration closest to the average duration.
29. The automatic speaker verification method of claim 25, wherein the step of selecting a reference utterance comprises the step of: choosing an utterance with minimum combined distortion with respect to the other utterances.
30. An automatic speaker verification method, wherein the results of prior verifications are stored, including the steps of: obtaining test speech from a user seeking authorization or identification; generating subwords of the test speech; scoring the subwords against subwords of a known individual using a plurality of modeling classifiers; storing the results of each model classifiers as a classifier score; fusing the results of each classifier score using a fusion constant and weighing function to generate a final score; and comparing final score to a threshold value to determine whether the test speech and enrollment speech are from the known individual.
31. The automatic speaker verification method of claim 30, further comprising the step of: determining that fusion adaptation inclusion criteria are met; and changing the fusion constant to provide more weight to the classifier score which more accurately corresponds to the threshold value.
32. The automatic speaker verification method of claim 30, further comprising the steps of: determining that model adaptation inclusion criteria are met, including that one or more verifications have been successful; and training the model classifiers with previously stored enrollment speech and with speech corresponding to the successful verifications, including the steps of generating a new threshold value; and storing the new threshold value.
33. The automatic speaker verification method of claim 30, further comprising the steps of: determining that threshold adaptation inclusion criteria are met; analyzing the stored final scores; calculating a new threshold value in response to the analyzation; and storing the new threshold value.
34. An automatic speaker verification method, comprising the steps of: obtaining test speech from a user over a test channel; processing the test speech to remove the effects of the test channel; and comparing the processed test speech with speech data from a known user, including the steps of: extracting features of the test speech; generating subwords based on the extracted features; scoring the subwords using one or more model classifiers; fusing the results of the model classifiers to obtain a final score; and verifying the user if the final score is equal to or greater than a threshold value.
35. The automatic speaker verification method of claim 34, wherein the known speech is obtained over an enrollment channel, wherein the step of processing further comprises the step of filtering the test speech through a filter having characteristics of the enrollment channel, and wherein the step of generating subwords further comprises the step of spotting one or more key words/ key phrases in the processed test speech.
36. The automatic speaker verification method of claim 34, further comprising the steps of: training the model classifiers using antispeaker data from nonusers and one or more enrollment speech samples from the user; changing the model classifiers and threshold value, including the step of: determining that the user has been verified; retraining the model classifiers, including the step of using test speech corresponding the verified final score as an enrollment sample; calculating a new threshold value based on the retrained model classifiers.
PCT/US1997/021261 1996-11-22 1997-11-21 Subword-based speaker verification using multiple classifier fusion, with channel, fusion, model, and threshold adaptation WO1998022936A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP97950642A EP0954854A4 (en) 1996-11-22 1997-11-21 Subword-based speaker verification using multiple classifier fusion, with channel, fusion, model, and threshold adaptation
AU53594/98A AU5359498A (en) 1996-11-22 1997-11-21 Subword-based speaker verification using multiple classifier fusion, with channel, fusion, model, and threshold adaptation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US3163996P 1996-11-22 1996-11-22
US60/031,639 1996-11-22

Publications (1)

Publication Number Publication Date
WO1998022936A1 true WO1998022936A1 (en) 1998-05-28

Family

ID=21860598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1997/021261 WO1998022936A1 (en) 1996-11-22 1997-11-21 Subword-based speaker verification using multiple classifier fusion, with channel, fusion, model, and threshold adaptation

Country Status (4)

Country Link
US (2) US6539352B1 (en)
EP (1) EP0954854A4 (en)
AU (1) AU5359498A (en)
WO (1) WO1998022936A1 (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1999021172A2 (en) * 1997-10-20 1999-04-29 Koninklijke Philips Electronics N.V. Pattern recognition enrolment in a distributed system
WO1999027524A1 (en) * 1997-11-21 1999-06-03 Siemens Aktiengesellschaft Method and device for voice recognition
EP0964391A2 (en) * 1998-05-30 1999-12-15 Philips Patentverwaltung GmbH Device for the verification of signals
WO2002103680A2 (en) * 2001-06-19 2002-12-27 Securivox Ltd Speaker recognition system ____________________________________
EP1426924A1 (en) * 2002-12-03 2004-06-09 Alcatel Speaker recognition for rejecting background speakers
WO2005055200A1 (en) * 2003-12-05 2005-06-16 Queensland University Of Technology Model adaptation system and method for speaker recognition
DE102007005704A1 (en) * 2007-02-05 2008-08-07 Voice Trust Ag Digital method for authenticating a person and ordering to carry it out
US9105065B2 (en) 2001-06-08 2015-08-11 Genworth Holdings, Inc. Systems and methods for providing a benefit product with periodic guaranteed income
US9105063B2 (en) 2001-06-08 2015-08-11 Genworth Holdings, Inc. Systems and methods for providing a benefit product with periodic guaranteed minimum income
US10255637B2 (en) 2007-12-21 2019-04-09 Genworth Holdings, Inc. Systems and methods for providing a cash value adjustment to a life insurance policy
RU2697736C1 (en) * 2016-07-15 2019-08-19 ГУГЛ ЭлЭлСи Speaker verification
WO2021139589A1 (en) * 2020-01-10 2021-07-15 华为技术有限公司 Voice processing method, medium, and system
EP3905241A1 (en) * 2017-04-20 2021-11-03 Google LLC Multi-user authentication on a device

Families Citing this family (138)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10257583A (en) * 1997-03-06 1998-09-25 Asahi Chem Ind Co Ltd Voice processing unit and its voice processing method
US6968342B2 (en) * 1997-12-29 2005-11-22 Abel Wolman Energy minimization for data merging and fusion
DE19952049A1 (en) * 1999-10-28 2001-05-10 Siemens Ag Method and arrangement for verifying a speaker using a computer
DE10000973A1 (en) * 2000-01-06 2001-07-12 Deutsche Telekom Ag Voice-controlled identification involves permitting clear identification once user has been recognized, thereby enabling user to be addressed personally
US6728674B1 (en) * 2000-07-31 2004-04-27 Intel Corporation Method and system for training of a classifier
US6999926B2 (en) * 2000-11-16 2006-02-14 International Business Machines Corporation Unsupervised incremental adaptation using maximum likelihood spectral transformation
US7177808B2 (en) * 2000-11-29 2007-02-13 The United States Of America As Represented By The Secretary Of The Air Force Method for improving speaker identification by determining usable speech
US20020078416A1 (en) * 2000-12-01 2002-06-20 Hitachi, Ltd. Method of recording/reproducing digital data and apparatus for same
US20020077828A1 (en) * 2000-12-18 2002-06-20 Building Better Interfaces, Inc. Distributed adaptive heuristic voice recognition technique
JP4759827B2 (en) * 2001-03-28 2011-08-31 日本電気株式会社 Voice segmentation apparatus and method, and control program therefor
ATE335195T1 (en) * 2001-05-10 2006-08-15 Koninkl Philips Electronics Nv BACKGROUND LEARNING OF SPEAKER VOICES
US6999928B2 (en) * 2001-08-21 2006-02-14 International Business Machines Corporation Method and apparatus for speaker identification using cepstral covariance matrices and distance metrics
US7200656B1 (en) * 2001-10-19 2007-04-03 Bbn Technologies Corp. Methods and systems for simultaneously detecting short and long term periodicity for traffic flow identification
US7574597B1 (en) 2001-10-19 2009-08-11 Bbn Technologies Corp. Encoding of signals to facilitate traffic analysis
US7062498B2 (en) * 2001-11-02 2006-06-13 Thomson Legal Regulatory Global Ag Systems, methods, and software for classifying text from judicial opinions and other documents
US7203652B1 (en) * 2002-02-21 2007-04-10 Nuance Communications Method and system for improving robustness in a speech system
AU2003222132A1 (en) * 2002-03-28 2003-10-13 Martin Dunsmuir Closed-loop command and response system for automatic communications between interacting computer systems over an audio communications channel
US7424425B2 (en) * 2002-05-19 2008-09-09 International Business Machines Corporation Optimization of detection systems using a detection error tradeoff analysis criterion
US7305340B1 (en) * 2002-06-05 2007-12-04 At&T Corp. System and method for configuring voice synthesis
US7509257B2 (en) * 2002-12-24 2009-03-24 Marvell International Ltd. Method and apparatus for adapting reference templates
US7222072B2 (en) * 2003-02-13 2007-05-22 Sbc Properties, L.P. Bio-phonetic multi-phrase speaker identity verification
US20040186724A1 (en) * 2003-03-19 2004-09-23 Philippe Morin Hands-free speaker verification system relying on efficient management of accuracy risk and user convenience
US7516071B2 (en) * 2003-06-30 2009-04-07 International Business Machines Corporation Method of modeling single-enrollment classes in verification and identification tasks
US7404087B2 (en) * 2003-12-15 2008-07-22 Rsa Security Inc. System and method for providing improved claimant authentication
EP2290559A1 (en) * 2003-12-31 2011-03-02 Thomson Reuters Global Resources Systems, methods, software and interfaces for integration of case law with legal briefs, litigation documents, and/or other litigation-support documents
JP2005316717A (en) * 2004-04-28 2005-11-10 Sony Corp Memorandum system, portable terminal, computer program, recording medium and memorandum information provision method
US20070033041A1 (en) * 2004-07-12 2007-02-08 Norton Jeffrey W Method of identifying a person based upon voice analysis
US8295446B1 (en) 2004-09-03 2012-10-23 Confinement Telephony Technology, Llc Telephony system and method with enhanced call monitoring, recording and retrieval
US9240188B2 (en) 2004-09-16 2016-01-19 Lena Foundation System and method for expressive language, developmental disorder, and emotion assessment
US10223934B2 (en) 2004-09-16 2019-03-05 Lena Foundation Systems and methods for expressive language, developmental disorder, and emotion assessment, and contextual feedback
US8078465B2 (en) * 2007-01-23 2011-12-13 Lena Foundation System and method for detection and analysis of speech
US9355651B2 (en) 2004-09-16 2016-05-31 Lena Foundation System and method for expressive language, developmental disorder, and emotion assessment
US8938390B2 (en) * 2007-01-23 2015-01-20 Lena Foundation System and method for expressive language and developmental disorder assessment
US20080208578A1 (en) * 2004-09-23 2008-08-28 Koninklijke Philips Electronics, N.V. Robust Speaker-Dependent Speech Recognition System
KR100679042B1 (en) * 2004-10-27 2007-02-06 삼성전자주식회사 Method and apparatus for speech recognition, and navigation system using for the same
US7447633B2 (en) * 2004-11-22 2008-11-04 International Business Machines Corporation Method and apparatus for training a text independent speaker recognition system using speech data with text labels
US7742581B2 (en) 2004-11-24 2010-06-22 Value-Added Communications, Inc. Electronic messaging exchange
US8243891B2 (en) * 2005-01-28 2012-08-14 Value-Added Communications, Inc. Voice message exchange
US9282188B2 (en) 2005-01-28 2016-03-08 Value-Added Communications, Inc. Voice message exchange
US9876915B2 (en) 2005-01-28 2018-01-23 Value-Added Communications, Inc. Message exchange
US7490043B2 (en) * 2005-02-07 2009-02-10 Hitachi, Ltd. System and method for speaker verification using short utterance enrollments
CN101124623B (en) * 2005-02-18 2011-06-01 富士通株式会社 Voice authentication system and method
US20060229879A1 (en) * 2005-04-06 2006-10-12 Top Digital Co., Ltd. Voiceprint identification system for e-commerce
US9571652B1 (en) 2005-04-21 2017-02-14 Verint Americas Inc. Enhanced diarization systems, media and methods of use
US8510215B2 (en) * 2005-04-21 2013-08-13 Victrio, Inc. Method and system for enrolling a voiceprint in a fraudster database
US8639757B1 (en) 2011-08-12 2014-01-28 Sprint Communications Company L.P. User localization using friend location information
US20080312926A1 (en) * 2005-05-24 2008-12-18 Claudio Vair Automatic Text-Independent, Language-Independent Speaker Voice-Print Creation and Speaker Recognition
US7386105B2 (en) * 2005-05-27 2008-06-10 Nice Systems Ltd Method and apparatus for fraud detection
US9300790B2 (en) 2005-06-24 2016-03-29 Securus Technologies, Inc. Multi-party conversation analyzer and logger
US7788101B2 (en) * 2005-10-31 2010-08-31 Hitachi, Ltd. Adaptation method for inter-person biometrics variability
KR100717401B1 (en) * 2006-03-02 2007-05-11 삼성전자주식회사 Method and apparatus for normalizing voice feature vector by backward cumulative histogram
US20070219801A1 (en) * 2006-03-14 2007-09-20 Prabha Sundaram System, method and computer program product for updating a biometric model based on changes in a biometric feature of a user
US20070219792A1 (en) * 2006-03-20 2007-09-20 Nu Echo Inc. Method and system for user authentication based on speech recognition and knowledge questions
US20070239441A1 (en) * 2006-03-29 2007-10-11 Jiri Navratil System and method for addressing channel mismatch through class specific transforms
DE602006010511D1 (en) * 2006-04-03 2009-12-31 Voice Trust Ag Speaker authentication in digital communication networks
CN101051463B (en) * 2006-04-06 2012-07-11 株式会社东芝 Verification method and device identified by speaking person
EP1860647B1 (en) * 2006-05-24 2009-12-23 Voice.Trust Ag Robust speaker recognition
US20080010065A1 (en) * 2006-06-05 2008-01-10 Harry Bratt Method and apparatus for speaker recognition
US7822605B2 (en) * 2006-10-19 2010-10-26 Nice Systems Ltd. Method and apparatus for large population speaker identification in telephone interactions
US20080260169A1 (en) * 2006-11-06 2008-10-23 Plantronics, Inc. Headset Derived Real Time Presence And Communication Systems And Methods
US9591392B2 (en) * 2006-11-06 2017-03-07 Plantronics, Inc. Headset-derived real-time presence and communication systems and methods
EP2126901B1 (en) * 2007-01-23 2015-07-01 Infoture, Inc. System for analysis of speech
US8099288B2 (en) * 2007-02-12 2012-01-17 Microsoft Corp. Text-dependent speaker verification
US7953216B2 (en) 2007-05-04 2011-05-31 3V Technologies Incorporated Systems and methods for RFID-based access management of electronic devices
KR100870724B1 (en) 2007-05-25 2008-11-27 인하대학교 산학협력단 System for detecting image using t-test and method therefor
US8050919B2 (en) * 2007-06-29 2011-11-01 Microsoft Corporation Speaker recognition via voice sample based on multiple nearest neighbor classifiers
JP5089295B2 (en) * 2007-08-31 2012-12-05 インターナショナル・ビジネス・マシーンズ・コーポレーション Speech processing system, method and program
CN101465123B (en) * 2007-12-20 2011-07-06 株式会社东芝 Verification method and device for speaker authentication and speaker authentication system
US8696458B2 (en) * 2008-02-15 2014-04-15 Thales Visionix, Inc. Motion tracking system and method using camera and non-camera sensors
US8121967B2 (en) * 2008-06-18 2012-02-21 International Business Machines Corporation Structural data classification
JP2010008601A (en) * 2008-06-25 2010-01-14 Fujitsu Ltd Guide information display device, guide information display method, and program
US8370925B2 (en) 2008-07-29 2013-02-05 International Business Machines Corporation User policy manageable strength-based password aging
US8190437B2 (en) * 2008-10-24 2012-05-29 Nuance Communications, Inc. Speaker verification methods and apparatus
JP5042194B2 (en) * 2008-10-27 2012-10-03 インターナショナル・ビジネス・マシーンズ・コーポレーション Apparatus and method for updating speaker template
US8442824B2 (en) 2008-11-26 2013-05-14 Nuance Communications, Inc. Device, system, and method of liveness detection utilizing voice biometrics
US9767806B2 (en) * 2013-09-24 2017-09-19 Cirrus Logic International Semiconductor Ltd. Anti-spoofing
US8762149B2 (en) * 2008-12-10 2014-06-24 Marta Sánchez Asenjo Method for verifying the identity of a speaker and related computer readable medium and computer
US9621714B2 (en) 2009-01-27 2017-04-11 Value-Added Communications, Inc. System and method for electronic notification in institutional communication
EP2216775B1 (en) * 2009-02-05 2012-11-21 Nuance Communications, Inc. Speaker recognition
US8687839B2 (en) * 2009-05-21 2014-04-01 Digimarc Corporation Robust signatures derived from local nonlinear filters
US8417525B2 (en) * 2010-02-09 2013-04-09 International Business Machines Corporation Adaptive voice print for conversational biometric engine
US8775179B2 (en) * 2010-05-06 2014-07-08 Senam Consulting, Inc. Speech-based speaker recognition systems and methods
US9318114B2 (en) * 2010-11-24 2016-04-19 At&T Intellectual Property I, L.P. System and method for generating challenge utterances for speaker verification
US8818810B2 (en) 2011-12-29 2014-08-26 Robert Bosch Gmbh Speaker verification in a health monitoring system
US9390445B2 (en) 2012-03-05 2016-07-12 Visa International Service Association Authentication using biometric technology through a consumer device
US9336302B1 (en) 2012-07-20 2016-05-10 Zuci Realty Llc Insight and algorithmic clustering for automated synthesis
US9672815B2 (en) * 2012-07-20 2017-06-06 Interactive Intelligence Group, Inc. Method and system for real-time keyword spotting for speech analytics
US9368116B2 (en) 2012-09-07 2016-06-14 Verint Systems Ltd. Speaker separation in diarization
DK2713367T3 (en) * 2012-09-28 2017-02-20 Agnitio S L Speech Recognition
US10134400B2 (en) 2012-11-21 2018-11-20 Verint Systems Ltd. Diarization using acoustic labeling
SG11201504186UA (en) 2012-12-19 2015-07-30 Visa Int Service Ass System and method for voice authentication
US9230550B2 (en) * 2013-01-10 2016-01-05 Sensory, Incorporated Speaker verification and identification using artificial neural network-based sub-phonetic unit discrimination
US9646605B2 (en) * 2013-01-22 2017-05-09 Interactive Intelligence Group, Inc. False alarm reduction in speech recognition systems using contextual information
US8694315B1 (en) * 2013-02-05 2014-04-08 Visa International Service Association System and method for authentication using speaker verification techniques and fraud model
CN103258536B (en) * 2013-03-08 2015-10-21 北京理工大学 A kind of extensive speaker's identification method
CN103198833B (en) * 2013-03-08 2015-10-21 北京理工大学 A kind of high precision method for identifying speaker
US9237232B1 (en) * 2013-03-14 2016-01-12 Verint Americas Inc. Recording infrastructure having biometrics engine and analytics service
US9466292B1 (en) * 2013-05-03 2016-10-11 Google Inc. Online incremental adaptation of deep neural networks using auxiliary Gaussian mixture models in speech recognition
US9460722B2 (en) 2013-07-17 2016-10-04 Verint Systems Ltd. Blind diarization of recorded calls with arbitrary number of speakers
US9711148B1 (en) 2013-07-18 2017-07-18 Google Inc. Dual model speaker identification
US9984706B2 (en) * 2013-08-01 2018-05-29 Verint Systems Ltd. Voice activity detection using a soft decision mechanism
US10237399B1 (en) 2014-04-01 2019-03-19 Securus Technologies, Inc. Identical conversation detection method and apparatus
US8812320B1 (en) 2014-04-01 2014-08-19 Google Inc. Segment-based speaker verification using dynamically generated phrases
US9621713B1 (en) 2014-04-01 2017-04-11 Securus Technologies, Inc. Identical conversation detection method and apparatus
US10540979B2 (en) * 2014-04-17 2020-01-21 Qualcomm Incorporated User interface for secure access to a device using speaker verification
US9922048B1 (en) 2014-12-01 2018-03-20 Securus Technologies, Inc. Automated background check via facial recognition
US9875743B2 (en) 2015-01-26 2018-01-23 Verint Systems Ltd. Acoustic signature building for a speaker from multiple sessions
WO2017008075A1 (en) * 2015-07-09 2017-01-12 Board Of Regents, The University Of Texas System Systems and methods for human speech training
CN106601238A (en) * 2015-10-14 2017-04-26 阿里巴巴集团控股有限公司 Application operation processing method and application operation processing device
GB2583988B (en) * 2016-06-06 2021-03-31 Cirrus Logic Int Semiconductor Ltd Voice user interface
US10141009B2 (en) 2016-06-28 2018-11-27 Pindrop Security, Inc. System and method for cluster-based audio event detection
EP3267438B1 (en) * 2016-07-05 2020-11-25 Nxp B.V. Speaker authentication with artificial neural networks
US9824692B1 (en) 2016-09-12 2017-11-21 Pindrop Security, Inc. End-to-end speaker recognition using deep neural network
US10325601B2 (en) 2016-09-19 2019-06-18 Pindrop Security, Inc. Speaker recognition in the call center
WO2018053531A1 (en) * 2016-09-19 2018-03-22 Pindrop Security, Inc. Dimensionality reduction of baum-welch statistics for speaker recognition
CA3179080A1 (en) 2016-09-19 2018-03-22 Pindrop Security, Inc. Channel-compensated low-level features for speaker recognition
US10614813B2 (en) * 2016-11-04 2020-04-07 Intellisist, Inc. System and method for performing caller identity verification using multi-step voice analysis
US10755718B2 (en) * 2016-12-07 2020-08-25 Interactive Intelligence Group, Inc. System and method for neural network based speaker classification
US11205103B2 (en) 2016-12-09 2021-12-21 The Research Foundation for the State University Semisupervised autoencoder for sentiment analysis
US10397398B2 (en) 2017-01-17 2019-08-27 Pindrop Security, Inc. Authentication using DTMF tones
CN108335755B (en) * 2017-01-19 2022-03-04 京东方科技集团股份有限公司 Data analysis method and device
US10749827B2 (en) 2017-05-11 2020-08-18 Global Tel*Link Corporation System and method for inmate notification and training in a controlled environment facility
DK179496B1 (en) * 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
US10529357B2 (en) 2017-12-07 2020-01-07 Lena Foundation Systems and methods for automatic determination of infant cry and discrimination of cry from fussiness
CN108364006B (en) * 2018-01-17 2022-03-08 超凡影像科技股份有限公司 Medical image classification device based on multi-mode deep learning and construction method thereof
US10715522B2 (en) * 2018-01-31 2020-07-14 Salesforce.Com Voiceprint security with messaging services
US11538128B2 (en) 2018-05-14 2022-12-27 Verint Americas Inc. User interface for fraud alert management
US10887452B2 (en) 2018-10-25 2021-01-05 Verint Americas Inc. System architecture for fraud detection
WO2020159917A1 (en) 2019-01-28 2020-08-06 Pindrop Security, Inc. Unsupervised keyword spotting and word discovery for fraud analytics
US11019201B2 (en) 2019-02-06 2021-05-25 Pindrop Security, Inc. Systems and methods of gateway detection in a telephone network
US11289098B2 (en) * 2019-03-08 2022-03-29 Samsung Electronics Co., Ltd. Method and apparatus with speaker recognition registration
WO2020184753A1 (en) * 2019-03-12 2020-09-17 엘지전자 주식회사 Artificial intelligence apparatus for performing voice control by using voice extraction filter, and method therefor
WO2020198354A1 (en) 2019-03-25 2020-10-01 Pindrop Security, Inc. Detection of calls from voice assistants
US11115521B2 (en) 2019-06-20 2021-09-07 Verint Americas Inc. Systems and methods for authentication and fraud detection
US11868453B2 (en) 2019-11-07 2024-01-09 Verint Americas Inc. Systems and methods for customer authentication based on audio-of-interest
CN111048100B (en) * 2019-11-21 2023-09-08 深圳市东进银通电子有限公司 Big data parallelization voiceprint recognition system and method
CN113744727B (en) * 2021-07-16 2023-12-26 厦门快商通科技股份有限公司 Model training method, system, terminal equipment and storage medium
CN113938785A (en) * 2021-11-24 2022-01-14 英华达(上海)科技有限公司 Noise reduction processing method, device, equipment, earphone and storage medium

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5850360B2 (en) * 1978-05-12 1983-11-10 株式会社日立製作所 Preprocessing method in speech recognition device
US5548647A (en) * 1987-04-03 1996-08-20 Texas Instruments Incorporated Fixed text speaker verification method and apparatus
US5528731A (en) * 1993-11-19 1996-06-18 At&T Corp. Method of accommodating for carbon/electret telephone set variability in automatic speaker verification
US5638486A (en) * 1994-10-26 1997-06-10 Motorola, Inc. Method and system for continuous speech recognition using voting techniques
US5596679A (en) * 1994-10-26 1997-01-21 Motorola, Inc. Method and system for identifying spoken sounds in continuous speech by comparing classifier outputs
US5862519A (en) * 1996-04-02 1999-01-19 T-Netix, Inc. Blind clustering of data with application to speech processing systems
JPH10105191A (en) * 1996-09-30 1998-04-24 Toshiba Corp Speech recognition device and microphone frequency characteristic converting method
US5950157A (en) * 1997-02-28 1999-09-07 Sri International Method for establishing handset-dependent normalizing models for speaker recognition
US6266633B1 (en) * 1998-12-22 2001-07-24 Itt Manufacturing Enterprises Noise suppression and channel equalization preprocessor for speech and speaker recognizers: method and apparatus

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
IEEE SIGNAL PROCESSING MAGAZINE, September 1996, MAMMONE R.J. et al., "Robust Speaker Recognition", p. 58-71. *
MODERN METHODS OF SPEECH PROCESSING, FARRELL K.R. et al., "Data Fusion Techniques for Speaker Recognition", ed. by R.P. RAMACHANDRAN et al., KLUWER, 1995, p. 279-292. *
PROC. ICASSP 96, May 1996, SHARMA M., "Subword-Based Text-Dependent Speaker Verification System with User-Selectable Passwords", p. 93-96. *
See also references of EP0954854A4 *

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1999021172A3 (en) * 1997-10-20 1999-07-01 Koninkl Philips Electronics Nv Pattern recognition enrolment in a distributed system
US6363348B1 (en) 1997-10-20 2002-03-26 U.S. Philips Corporation User model-improvement-data-driven selection and update of user-oriented recognition model of a given type for word recognition at network server
WO1999021172A2 (en) * 1997-10-20 1999-04-29 Koninklijke Philips Electronics N.V. Pattern recognition enrolment in a distributed system
US7035799B1 (en) 1997-11-21 2006-04-25 Siemens Aktiengesellschaft Method and device for voice recognition
WO1999027524A1 (en) * 1997-11-21 1999-06-03 Siemens Aktiengesellschaft Method and device for voice recognition
EP0964391A2 (en) * 1998-05-30 1999-12-15 Philips Patentverwaltung GmbH Device for the verification of signals
EP0964391A3 (en) * 1998-05-30 2000-01-05 Philips Patentverwaltung GmbH Device for the verification of signals
US10055795B2 (en) 2001-06-08 2018-08-21 Genworth Holdings, Inc. Systems and methods for providing a benefit product with periodic guaranteed minimum income
US9105063B2 (en) 2001-06-08 2015-08-11 Genworth Holdings, Inc. Systems and methods for providing a benefit product with periodic guaranteed minimum income
US9105065B2 (en) 2001-06-08 2015-08-11 Genworth Holdings, Inc. Systems and methods for providing a benefit product with periodic guaranteed income
WO2002103680A3 (en) * 2001-06-19 2003-03-13 Securivox Ltd Speaker recognition system ____________________________________
CN100377209C (en) * 2001-06-19 2008-03-26 塞库里沃克斯公司 Speaker recognition systems
WO2002103680A2 (en) * 2001-06-19 2002-12-27 Securivox Ltd Speaker recognition system ____________________________________
EP1426924A1 (en) * 2002-12-03 2004-06-09 Alcatel Speaker recognition for rejecting background speakers
WO2005055200A1 (en) * 2003-12-05 2005-06-16 Queensland University Of Technology Model adaptation system and method for speaker recognition
DE102007005704A1 (en) * 2007-02-05 2008-08-07 Voice Trust Ag Digital method for authenticating a person and ordering to carry it out
DE102007005704B4 (en) * 2007-02-05 2008-10-30 Voice Trust Ag Digital method for authenticating a person and ordering to carry it out
US10255637B2 (en) 2007-12-21 2019-04-09 Genworth Holdings, Inc. Systems and methods for providing a cash value adjustment to a life insurance policy
US10403291B2 (en) 2016-07-15 2019-09-03 Google Llc Improving speaker verification across locations, languages, and/or dialects
RU2697736C1 (en) * 2016-07-15 2019-08-19 ГУГЛ ЭлЭлСи Speaker verification
US11017784B2 (en) 2016-07-15 2021-05-25 Google Llc Speaker verification across locations, languages, and/or dialects
US11594230B2 (en) 2016-07-15 2023-02-28 Google Llc Speaker verification
EP3905241A1 (en) * 2017-04-20 2021-11-03 Google LLC Multi-user authentication on a device
KR20210134821A (en) * 2017-04-20 2021-11-10 구글 엘엘씨 Multi-user authentication on a device
US11238848B2 (en) 2017-04-20 2022-02-01 Google Llc Multi-user authentication on a device
KR102458805B1 (en) 2017-04-20 2022-10-25 구글 엘엘씨 Multi-user authentication on a device
US11721326B2 (en) 2017-04-20 2023-08-08 Google Llc Multi-user authentication on a device
US11727918B2 (en) 2017-04-20 2023-08-15 Google Llc Multi-user authentication on a device
WO2021139589A1 (en) * 2020-01-10 2021-07-15 华为技术有限公司 Voice processing method, medium, and system

Also Published As

Publication number Publication date
EP0954854A4 (en) 2000-07-19
AU5359498A (en) 1998-06-10
US6539352B1 (en) 2003-03-25
EP0954854A1 (en) 1999-11-10
US6760701B2 (en) 2004-07-06
US20030009333A1 (en) 2003-01-09

Similar Documents

Publication Publication Date Title
US6760701B2 (en) Subword-based speaker verification using multiple-classifier fusion, with channel, fusion, model and threshold adaptation
US6519561B1 (en) Model adaptation of neural tree networks and other fused models for speaker verification
US5862519A (en) Blind clustering of data with application to speech processing systems
EP0870300B1 (en) Speaker verification system
Furui Recent advances in speaker recognition
Reynolds et al. Robust text-independent speaker identification using Gaussian mixture speaker models
EP1399915B1 (en) Speaker verification
EP0744734B1 (en) Speaker verification method and apparatus using mixture decomposition discrimination
EP0822539B1 (en) Two-staged cohort selection for speaker verification system
EP1159737B9 (en) Speaker recognition
AU2002311452A1 (en) Speaker recognition system
EP0891618A1 (en) Speech processing
WO2002029785A1 (en) Method, apparatus, and system for speaker verification based on orthogonal gaussian mixture model (gmm)
Olsson Text dependent speaker verification with a hybrid HMM/ANN system
WO1999059136A1 (en) Channel estimation system and method for use in automatic speaker verification systems
Bourlard et al. Speaker verification: A quick overview
Xafopoulos Speaker Verification (an overview)
Lee et al. Continuants based neural speaker verification system
Fakotakis et al. High performance text-independent speaker recognition system based on voiced/unvoiced segmentation and multiple neural nets.
Nickel et al. Robust speaker verification with principal pitch components
Xafopoulos Speaker Verification
Hlaing et al. Voice Authorization using Voice Recognition and Verification Method
VALSAMAKIS Speaker Identification and Verification Using Gaussian Mixture Models
Jayarathna Speaker Recognition using Voice Biometrics
P-SCHMM means of equation (3) performs worse than posterior probability for speaker verification while it performs slightly better for text verification. This is probably due the lower sensitivity of the median with respect to outlying points in the test lists.

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE GH HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZW AM AZ BY KG KZ MD RU TJ TM

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH KE LS MW SD SZ UG ZW AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 1997950642

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

WWP Wipo information: published in national office

Ref document number: 1997950642

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: CA

WWW Wipo information: withdrawn in national office

Ref document number: 1997950642

Country of ref document: EP