CA2351705C - System and method for automating transcription services - Google Patents
System and method for automating transcription services Download PDFInfo
- Publication number
- CA2351705C CA2351705C CA002351705A CA2351705A CA2351705C CA 2351705 C CA2351705 C CA 2351705C CA 002351705 A CA002351705 A CA 002351705A CA 2351705 A CA2351705 A CA 2351705A CA 2351705 C CA2351705 C CA 2351705C
- Authority
- CA
- Canada
- Prior art keywords
- file
- training
- current
- text
- written text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
- G10L15/063—Training
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
- G10L15/063—Training
- G10L2015/0631—Creating reference templates; Clustering
Abstract
A system for automating transcription services for multiple voice users including a manual transcription station, a speech recognition program and a routing program. The system establishes a profile for each of the voice users containing a training status which is selected from the group of enrollment, training, automated and stop automation. When the system receives a voice dictation file from a current voice user (206) based on the training status (207), the system routes the voice dictation file to a manual transcription station (301) and the speech recognition program. A
human transcriptionist (302) creates transcribed files for each received voice dictation file. The speech recognition program automatically creates a written text for each received voice dictation file if the training status of the current user is training or automated. A verbatim file (304) is manually established if the training status of the current user is enrollment or training, and the speech recognition program is trained with an acoustic model for the current user using the verbatim file and the voice dictation file (308) if the training status of the current user is enrollment or training. The transcribed file is returned to the current user if the training status of the current user is enrollment or training, or, the written text is returned if the training status of the current user is automated.
human transcriptionist (302) creates transcribed files for each received voice dictation file. The speech recognition program automatically creates a written text for each received voice dictation file if the training status of the current user is training or automated. A verbatim file (304) is manually established if the training status of the current user is enrollment or training, and the speech recognition program is trained with an acoustic model for the current user using the verbatim file and the voice dictation file (308) if the training status of the current user is enrollment or training. The transcribed file is returned to the current user if the training status of the current user is enrollment or training, or, the written text is returned if the training status of the current user is automated.
Description
SYSTEM AND METHOD FOR AUTOMATING TRANSCRIPTION SERVICES
Background of the Invention 1. Field of the Invention The present invention relates in general to computer speech recognition systems and in particular, to a system and method for automating the text transcription of voice dictation by various end users.
Background of the Invention 1. Field of the Invention The present invention relates in general to computer speech recognition systems and in particular, to a system and method for automating the text transcription of voice dictation by various end users.
2. Background Art Speech recognition programs are well known in the art.
While these programs are ultimately useful in automatically converting speech into text, many users are dissuaded from using these programs because they require each user to spend a significant amount of time training the system. Usually this training begins by having each user read a series of pre-selected materials for approximately 20 minutes. Then, as the user continues to use the program, as words are improperly transcribed the user is expected to stop and train the program as to the intended word thus advancing the ultimate accuracy of the acoustic model. Unfortunately, most professionals (doctors, dentists, veterinarians, lawyers) and business executives are unwilling to spend the time developing the necessary acoustic model to truly benefit from the automated transcription.
Accordingly, the present invention seeks to provide a system that offers transparent training of the speech recognition program to the end-users.
There are systems for using computers for routing transcription from a group of end users. Most often these systems are used in large multi-user settings such as hospitals.
In those systems, a voice user dictates into a general-purpose computer or other recording device and the resulting file is transferred automatically to a human transcriptionist. The human transcriptionist transcribes the file, which is then returned to the original "author" for review. These systems have the perpetual overhead of employing a sufficient number of human transcriptionists to transcribe all of the dictation files.
Accordingly, the present invention seeks to provide an automated means of translating speech into text wherever suitable so as to minimize the number of human transcriptionists necessary to transcribe audio files coming into the system.
It is an associated aspect to provide a simplified means for providing verbatim text files for training a user's acoustic model for the speech recognition portion of the system.
It is another associated aspect of the present invention to automate a preexisting speech recognition program toward further minimizing the number of operators necessary to operate the system.
These and other aspects will be apparent to those of ordinary skill in the art having the present drawings, specification and claims before them.
WO 00/31724 PCT/US99/27539 _
While these programs are ultimately useful in automatically converting speech into text, many users are dissuaded from using these programs because they require each user to spend a significant amount of time training the system. Usually this training begins by having each user read a series of pre-selected materials for approximately 20 minutes. Then, as the user continues to use the program, as words are improperly transcribed the user is expected to stop and train the program as to the intended word thus advancing the ultimate accuracy of the acoustic model. Unfortunately, most professionals (doctors, dentists, veterinarians, lawyers) and business executives are unwilling to spend the time developing the necessary acoustic model to truly benefit from the automated transcription.
Accordingly, the present invention seeks to provide a system that offers transparent training of the speech recognition program to the end-users.
There are systems for using computers for routing transcription from a group of end users. Most often these systems are used in large multi-user settings such as hospitals.
In those systems, a voice user dictates into a general-purpose computer or other recording device and the resulting file is transferred automatically to a human transcriptionist. The human transcriptionist transcribes the file, which is then returned to the original "author" for review. These systems have the perpetual overhead of employing a sufficient number of human transcriptionists to transcribe all of the dictation files.
Accordingly, the present invention seeks to provide an automated means of translating speech into text wherever suitable so as to minimize the number of human transcriptionists necessary to transcribe audio files coming into the system.
It is an associated aspect to provide a simplified means for providing verbatim text files for training a user's acoustic model for the speech recognition portion of the system.
It is another associated aspect of the present invention to automate a preexisting speech recognition program toward further minimizing the number of operators necessary to operate the system.
These and other aspects will be apparent to those of ordinary skill in the art having the present drawings, specification and claims before them.
WO 00/31724 PCT/US99/27539 _
- 3 - -~ummarx czf the Invention The present invention comprises, in part, a system for substantially automating transcription services for one or more voice users. The system includes means for receiving a voice dictation file from a current user and an audio player used to audibly reproduce said voice dictation file. Both of these system elements can be implemented on the same or different general-purpose computers. Additionally, the voice dictation file receiving means may include an audio recorder, means for operably connecting to a separate digital recording device and/or means for reading audio files from removable magnetic and other computer media.
Each of the general purpose computers implementing the system may be remotely located from the other computers but in operable connection to each other by way of a computer network, direct telephone connection, via email or other Internet based transfer.
The system further includes means for manually inputting and creating a transcribed file based on humanly perceived contents of the voice dictation file.
Thus, for certain voice dictation files, a human transcriptionist manually transcribes a textual version of the audio -- using a text editor or word processor --based on the output of the output of the audio player.
The system also includes means for automatically converting the voice dictation file into written text.
The automatic speech converting means may be a preexisting speech recognition program, such as Dragon Systems' Naturally Speaking, IBM's Via Voice or Philips Corporation's Magic Speech. In such a case, the WO 00131724 PCTIUS99/27539 _ _ _ automatic speech converting means includes means for automating responses to a series of interactive inquiries from the preexisting speech recognition program. In one embodiment, the system also includes means for manually selecting a specialized language model.
The system further includes means for manually editing the resulting written text to create a verbatim text of the voice dictation file. At the outset of a users use of the system, this verbatim text will have to be created completely manually. However, after the automatic speech converting means has begun to sufficiently develop that user's acoustic model a more automated means can be used.
In a preferred embodiment, that manual editing means includes means for sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched words culled from the copy of said written text. The manual editing means further includes means for incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list. Finally, the preferred manual editing means includes means for correcting the current unmatched word in the second buffer, which includes means for displaying the current unmatched word in a manner substantially visually isolated from other text in the written text and means for playing a portion of the voice dictation recording from said first buffer associated with said current unmatched word. In one embodiment, the manual input
Each of the general purpose computers implementing the system may be remotely located from the other computers but in operable connection to each other by way of a computer network, direct telephone connection, via email or other Internet based transfer.
The system further includes means for manually inputting and creating a transcribed file based on humanly perceived contents of the voice dictation file.
Thus, for certain voice dictation files, a human transcriptionist manually transcribes a textual version of the audio -- using a text editor or word processor --based on the output of the output of the audio player.
The system also includes means for automatically converting the voice dictation file into written text.
The automatic speech converting means may be a preexisting speech recognition program, such as Dragon Systems' Naturally Speaking, IBM's Via Voice or Philips Corporation's Magic Speech. In such a case, the WO 00131724 PCTIUS99/27539 _ _ _ automatic speech converting means includes means for automating responses to a series of interactive inquiries from the preexisting speech recognition program. In one embodiment, the system also includes means for manually selecting a specialized language model.
The system further includes means for manually editing the resulting written text to create a verbatim text of the voice dictation file. At the outset of a users use of the system, this verbatim text will have to be created completely manually. However, after the automatic speech converting means has begun to sufficiently develop that user's acoustic model a more automated means can be used.
In a preferred embodiment, that manual editing means includes means for sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched words culled from the copy of said written text. The manual editing means further includes means for incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list. Finally, the preferred manual editing means includes means for correcting the current unmatched word in the second buffer, which includes means for displaying the current unmatched word in a manner substantially visually isolated from other text in the written text and means for playing a portion of the voice dictation recording from said first buffer associated with said current unmatched word. In one embodiment, the manual input
4 PCTNS99I27S39 _ _ _ means further includes means for alternatively viewing the current unmatched word in context within the written text. For instance, the operator may wish to view the unmatched within the sentence in which it appears or perhaps with only is immediately adjacent words. Thus, the manner substantially visual isolation can be manually selected from the group containing word-by-word display, sentence-by-sentence display, and said current unmatched word display. The manual editing means portion of the complete system may also be utilized as a separate apparatus.
The system may also include means for determining an accuracy rate for the current user. In one approach, this accuracy determination can be made by determining the ratio of the number of words in the sequential list of unmatched words to the number of wards in the written text.
The system additionally includes means for training the automatic speech converting means to achieve higher accuracy for the current user. In particular, the training means uses the verbatim text created by the manual editing means and the voice dictation file **.
The training means may also comprise a preexisting training portion of the preexisting speech recognition program. Thus, the training means would also include means for automating responses to a series of interactive inquiries from the preexisting training portion of the speech recognition program.
The system finally includes means for controlling the flow of the voice dictation file based upon the training status of the current user. said voice dictation WO 00/31724 PCT/US99/27539 _ _ 6 - _ file to at 3east one of said manual input means and said automatic speech converting means. The control means reads and modifies a user's training status such that it is an appropriate selection from the group of pre-y enrollment, enrollment, training, automation and stop automation. During a user's pre-enrollment phase the control means further includes means for creating a user identification and acoustic model within the automatic speech converting means. During this phase, the control means routes the voice dictation file to the automatic speech converting means and the manual input means, routes the written text and the transcribed file to the manual editing means, routes the verbatim text to the training means and routes the transcribed file back to the current user as a finished text.
During the training phase, the control means routes (1) the voice dictation file to the automatic speech converting means and the manual input means, (2) routes the written text and the transcribed file to the manual editing means, (3) routes the verbatim text to the training means and (4) routes the transcribed file back to the current user as a finished text.
During the automation stage, the control means routes (1) the voice dictation file only to the automatic speech converting means and (2) the written text back to the current user as a finished text.
The present application also discloses a method fox automating transcription services for one or more voice users in a system including a manual transcription station and a speech recognition program. The method comprising the steps of: (2) establishing a profile for WO 40/31724 PCTIUS99I2~S39 _ _ _ each of the voice users, the profile containing a training status; {2) receiving a voice dictation file from a current voice user; (3) choosing the training status of the current voice user from the group of enrollment, training, automated and stop automation; (4) routing the voice dictation file to at least one of the manual transcription station and the speech recognition program based on the training status; (5) receiving the voice dictation file in at least one of the manual transcription station and the speech recognition program;
(6) creating a transcribed file at the manual transcription station for each received voice dictation file; (7) automatically creating a written text with the speech recognition program for each received voice dictation file if the training status of the current user is training or automated; (8) manually establishing a verbatim file if the training status of the current user is enrollment or training; (9) training the speech recognition program with an acoustic model for the current user using the verbatim file and the voice dictation file if the training status of the current user is enrollment or training; {10) returning the transcribed file to the current user if the training status of the current user is enrollment or training; and (11) returning the written text to the current user if the training status of the current user is automated.
This method may also include the various substeps in the step of manually establishing a verbatim file which include: (1) assisting an operator to establish the verbatim file if the training status of the current user is training by: (a) sequentially comparing a copy of the written text with the transcribed file resulting in a WO 00/31724 PCTIUS99127539 _ _ g _ _ sequential list of unmatched words culled from the copy of the written text, the sequential list having-a beginning, an end and a current unmatched word, the current unmatched word being successively advanced from the beginning to the end; (b) incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; (c} displaying the current unmatched word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and (d) correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
The invention also includes a method for assisting an operator to establish a verbatim file from a written text created by a speech recognition program and a transcribed file created by a human transcriptionist, the method comprising the steps of: (1) sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched words culled from the copy of the written text, the sequential list having a beginning, an end and a current unmatched word, the current unmatched ward being successively advanced from the beginning to the end; (2) incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; (3) displaying the current unmatched g _ _.
word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and (4) correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
Brief Description c~~ the Drawings Fig. I of the drawings is a block diagram of one potential embodiment of the present system for substantially automating transcription services for one or more voice users;
Fig. 3.b of the drawings is a block diagram of a general-purpose computer which may be used as a dictation station, a transcription station and the control means within the present system;
Fig. 2a of the drawings is a flow diagram of the main loop of the control means of the present system;
Fig. 2b of the drawings is a flow diagram of the enrollment stage portion of the control means of the present system;
Fig. 2c of the drawings is a flow diagram of the training stage portion of the control means of the present system;
Fig. 2d of the drawings is a flow diagram of the automation stage portion ~f the control means of the present system;
Fig. 3 of the drawings is a directory structure used by the control means in the present system;
Fig. 4 of the drawings is a block diagram of a portion of a preferred embodiment of the manual editing means; and Fig. 5 of the drawings is an elevation view of the remainder of a preferred embodiment of the manual editing means.
Best Modes of Practicing the Invention While the present invention may be embodied in many different forms, there is shown in the drawings and discussed herein a few specific embodiments with the understanding that the present disclosure is to be considered only as an exemplification of the principles of the invention and is not intended to limit the invention to the embodiments illustrated.
Fig. 1 of the drawings generally shows one potential embodiment of the present system for substantially automating transcription services for one or more voice users. The present system must include some means for receiving a voice dictation file from a current user.
This voice dictation file receiving means can be a digital audio recorder, an analog audio recorder, or standard means for receiving computer files on magnetic media or via a data connection.
As shown, in one embodiment, the system 100 includes multiple digital recording stations 10, 11, 12 and 13.
Each digital recording station has at least a digital audio recorder and means for identifying the current voice user.
WO 00/31724 PCT/US99I27539 _ Preferably, each of these digital recording stations is implemented on a general-purpose computer (such as computer 20), although a specialized computer could be developed for this specific purpose. The general-purpose computer, though has the added advantage of being adaptable to varying uses in addition to operating within the present system 100. In general, the general-purpose computer should have, among other elements, a microprocessor (such as the Intel Corporation PENTIUM, Cyrix K6 or Motorola 68000 series); volatile and non-volatile memory; one or more mass storage devices (i.e.
HDD (not shown), floppy drive 21, and other removable media devices 22 such as a CD-ROM drive, DITTO, ZIP or JAZ drive (from Iomega Corporation) and the like);
various user input devices, such as a mouse 23, a keyboard 24, or a microphone 25; and a video display system 26. In one embodiment, the general-purpose computer is controlled by the WINDOWS 9.x operating system. It is contemplated, however, that the present system would work equally well using a MACINTOSH computer or even another operating system such as a WINDOWS CE, UNIX or a JAVA based operating system, to name a few.
Regardless of the particular computer platform used, in an embodiment utilizing an analog audio input (via microphone 25) the general-purpose computer must include a sound-card (not shown). Of course, in an embodiment with a digital input no sound card would be necessary.
In the embodiment shown in Fig. 1, digital audio recording stations 10, 11, 12 and 13 are loaded and configured to run digital audio recording software an a PENTIUM-based computer system operating under WINDOWS
WO OOJ3I724 PCT/US99J27539 _ 12 _ _ 9.x. Such digital recording software is available as a utility in the WINDOWS 9.x operating system or from various third party vendor such as The Programmers' Consortium, Tnc. of Oakton, Virginia (VOICEDOC), Syntrillium Corporation of Phoenix, Arizona {COOL EDIT}
or Dragon Systems Corporation (Dragon Naturally Speaking Professional Edition). These various software programs produce a voice dictation file in the form of a "WAV"
file. However, as would be known to those skilled in the art, other audio file formats, such as MP3 or DSS, could also be used to format the voice dictation file, without departing from the spirit of the present invention. In one embodiment where VOICEDOC software is used that software also automatically assigns a file handle to the WAV file, however, it would be known to those of ordinary skill in the art to save an audio file on a computer system using standard operating system file management methods.
Another means for receiving a voice dictation file is dedicated digital recorder 14, such as the Olympus Digital Voice Recorder D-1000 manufactured by the Olympus Corporation. Thus, if the current voice user is mare comfortable with a more conventional type of dictation device, they can continue to use a dedicated digital recorder 14. In order to harvest the digital audio text file, upon completion of a recording, dedicated digital recorder 14 would be operably connected to one of the digital audio recording stations, such as I3, toward downloading the digital a-udio file into that general-purpose computer. With this approach, for instance, no audio card would be required.
- 13 _ _ Another alternative for receiving the voice dictation file may consist of using one farm or another of removable magnetic media containing a pre-recorded audio file. With this alternative an operator would input the removable magnetic media into one of the digital audio recording stations toward uploading the audio file into the system.
In some cases it may be necessary to pre-process the audio files to make them acceptable for processing by the speech recognition software. For instance, a DSS file format may have to be changed to a WAV file format, or the sampling rate of a digital audio file may have to be upsampled or downsampled. For instance, in use the Olympus Digital Voice Recorder with Dragon Naturally Speaking, Olympus' 8MHz rate needs to be upsampled to 11 MHz. Software to accomplish such pre-processing is available from a variety of sources including Syntrillium Corporation and Olympus Corporation.
The other aspect of the digital audio~recording stations is some means for identifying the current voice user. The identifying means may include keyboard 24 upon which the user (or a separate operator} can input the current user's unique identification code. Of course, the user identification can be input using a myriad of computer input devices such as pointing devices (e. g.
mouse 23), a touch screen (not shown}, a light pen (not shown), bar-code reader (not shown} or audio cues via microphone 25, to name a few.
In the case of a first time user the identifying means may also assign that user an identification number after receiving potentially identifying information from .. CA 02351705 2001-05-16 WO 00/31724 PCT/US99/27539 _ that user, including: (1) name; (2) address; (3) occupation; (4) vocal dialect or accent; etc. As discussed in association with the control means, based upon this input information, a voice user profile and a sub-directory within the control means are established.
Thus, regardless of the particular identification means used, a user identification must be established for each voice user and subsequently provided with a corresponding digital audio file for each use such that the control means can appropriately route and the system ultimately transcribe the audio.
In one embodiment of the present invention, the identifying means may also seek the manual selection of a specialty vocabulary. It is contemplated that the specialty vocabulary sets may be general for various users such as medical (i.e. Radiology, Orthopedic Surgery, Gynecology} and legal (i.e. corporate, patent, litigation} or highly specific such that within each specialty the vocabulary parameters could be further limited based on the particular circumstances of a particular dictation file. For instance, if the current voice user is a Radiologist dictating the reading of a abdominal CAT scan the nomenclature is highly specialized and different from the nomenclature for a renal ultrasound. By narrowly segmenting each selectable vocabulary set an increase in the accuracy of the automatic speech converter is likely.
As shown in Fig. 1, the digital audio recording stations may be operably connected to system 100 as part of computer network 30 or, alternatively, they may be operably connected to the system via Internet host 15.
WO 00/31724 PCT/US99/27539 _ 15 _ _ As shown in Fig. lb, the general-purpose computer can be connected to both network jack 27 and telephone jack.
With the use of an Internet host, connection may be accomplished by e-mailing the audio file via the Internet. Another method fox completing such connection is by way of direct modem connection via remote control software, such as PC ANYWHERE, which is available from Symantec Corporation of Cupertino, California. It is also possible, if the IP address of digital audio recording station 10 or Internet host I5 is known, to transfer the audio file using basic file transfer protocol. Thus, as can be seen from the foregoing, the present system allows great flexibility for voice users to provide audio input into the system.
Control means 200 controls the flow of voice dictation file based upon the training status of the current voice user. As shown in Figs. 2a, 2b, 2c, 2d, control means 200 comprises a software program operating on general purpose computer 40. In particular, the program is initialized in step 201 where variable are set, buffers cleared and the particular configuration for this particular installation of the control means is loaded. Control means continually monitors a target directory (such as "current" (shown in Fig. 3)) to determine whether a new file has been moved into the target, step 202. Once a new file is found (such as "5723.id" (shown in Fig. 3)), a determination is made as to whether or not the current user 5 (shown in Fig. 1) is a new user, step 203. -For each new user (as indicated by the existence of a ".pro" file in the "current" subdirectory), a new WO 00/31?24 PCT/US99I27539 _ _ 1~ _ _ subdirectory is established, step 204 (such as the "usern" subdirectory (shown in Fig. 3)). This subdirectory is used to store all of the audio files ("xxxx.wav"), written text ("xxxx.wrt"), verbatim text ("xxxx.vb"), transcription text ("xxxx.txt") and user profile ("usern.pro") for that particular user. Each particular job is assigned a unique number "xxxx" such that all of the files associated with a job can be associated by that number. With this directory structure, the number of users is practically limited only by storage space within general-purpose computer 40.
Now that the user subdirectory has been established, the user profile is moved to the subdirectory, step 205.
The contents of this user profile may vary between systems. The contents of one potential user profile is shown in Fig. 3 as containing: the user name, address, occupation and training status. Aside from the training status variable, which is necessary, the other data is useful in routing and transcribing the audio files.
The control means, having selected one set of files by the handle, determines the identity of the current user by comparing the ".id" file with its "user.tbl,"
step 206. Now that the user is known the user profile may be parsed from that user's subdirectory and the current training status determined, step 207. Steps 208-211 are the triage of the current training status is one of: enrollment, training, automate, and stop automation.
Enrollment is the first stage in automating transcription services. As shown in Fig. 2b, the audio file is sent to transcription, step 301. In particular, the "xxxx.wav" file is transferred to transcriptionist stations 50 and 51. In a preferred embodiment, both stations are general-purpose computers, which run both an audio player and manual input means. The audio player is likely to be a digital audio player, although it is possible that an analog audio file could be transferred to the stations. Various audio players are commonly available including a utility in the WINDOWS 9.x operating system and various other third parties such from The Programmers' Consortium, Inc. of Oakton, Virginia (VOICESCRIBE). Regardless of the audio player used to play the audio file, manual input means is running on the computer at the same time. This manual input means may comprise any of text editor or word processor (such as MS WORD, WordPerfect, AmiPro or Word Pad) in combination with a keyboard, mouse, or other user-interface device. In one embodiment of the present invention, this manual input means may, itself, also be speech recognition software, such as Naturally Speaking from Dragon Systems of Newton, Massachusetts, Via Voice from TBM Corporation of Armonk, New York, or Speech Magic from Philips Corporation of Atlanta, Georgia. Human transcriptionist 6 listens to the audio file created by current user 5 and as is known, manually inputs the perceived contents of that recorded text, thus establishing the transcribed file, step 302. Being human, human transcriptionist 6 is likely to impose experience, education and biases on the text and thus not input a verbatim transcript of the audio file. Upon completion of the human transcription, the human transcriptionist 6 saves the file and indicates that it is ready for transfer to the current users subdirectory as "xxxx.txt", step 303.
WD 00/31724 PCT/US99127539 _ _ 1 g _ _.
Inasmuch as this current user is only at the enrollment stage, a human operator will have to listen to the audio file and manually compare it to the transcribed file and create a verbatim file, step 304. That verbatim S file "xxxx.vb" is also transferred to the current user's subdirectory, step 305. Now that verbatim text is available, control means 200 starts the automatic speech conversion means, step 306. This automatic speech conversion means may be a preexisting program, such as Dragon System's Naturally Speaking, IBM's Via Voice or Philips' Speech Magic, to name a few. Alternatively, it could be a unique program that is designed to specifically perform automated speech recagnition.
In a preferred embodiment, Dragon Systems' Naturally Speaking has been used by running an executable simultaneously with Naturally Speaking that feeds phantom keystrokes and mauling operations through the WIN32API, such that Naturally Speaking believes that it is interacting with a human being, when in fact it is being controlled by control means 200. Such techniques are well known in the computer software testing art and, thus, will not be discussed in detail. It should suffice to say that by watching the application flow of any speech recognition program, an executable to mimic the interactive manual steps can be created.
If the current user is a new user, the speech recognition program will need to establish the new user, step 307. Control means provides the necessary information from the user profile found in the current user's subdirectory. All speech recognition require significant training to establish an acoustic model of a .. CA 02351705 2001-05-16 WO 00/31724 PCT/US99/27539 _ _ 19 _ _ particular user. In the case of Dragon, initially the program seeks approximately 20 minutes of audio usually obtained by the user reading a canned text provided by Dragon Systems. There is also functionality built into Dragon that allows "mobile training." Using this feature, the verbatim file and audio file are fed into the speech recognition program to beginning training the acoustic model far that user, step 308. Regardless of the length of that audio file, control means 200 closes the speech recognition program at the completion of the file, step 309.
As the enrollment step is too soon to use the automatically created text, a copy of the transcribed file is sent to the current user using the address information contained in the user profile, step 310.
This address can be a street address or an e-mail address. Following that transmission, the program returns to the main loop on Fig. 2a.
After a certain number of minutes of training have been conducted for a particular user, that user's training status may be changed from enrollment to training. The border for this change is subjective, but perhaps a good rule of thumb is once Dragon appears to be creating written text with 80% accuracy or more, the switch between states can be made. Thus, for such a user the next transcription event will prompt control means 200 into the training state. As shown in Fig. 2c, steps 401-403 are the same human transcription steps as steps 301-303 in the enrollment phase. Once the transcribed file is established, control means 200 starts the automatic speech conversion means (or speech recognition ... CA 02351705 2001-05-16 WO 00/31724 PCT/US99/27539 _ program) and selects the current user, step 404. The audio file is fed into the speech recognition program and a written text is established within the program buffer, step 405. In the case of Dragon, this buffer is given the same file handle on very instance of the program.
Thus, that buffer can be easily copied using standard operating system commands and manual editing can begin, step 406.
Manual editing is not an easy task. Human beings are prone to errors. Thus, the present invention also includes means for improving on that task. As shown in Fig. 4, the transcribed file ("3333.txt") and the copy of the written text ("3333.wrt") are sequentially compared word by word 406a toward establishing sequential list of unmatched words 406b that are culled from the copy of the written text. This list has a beginning and an end and pointer 406c to the current unmatched word. Underlying the sequential list is another list of objects which contains the original unmatched words, as well as the words immediately before and after that unmatched word, the starting location in memory of each unmatched word in the sequential list of unmatched words 406b and the length of the unmatched word.
As shown in Fig. 5, the unmatched word pointed at by pointer 406c from list 406b is displayed in substantial visual isolation from the other text in the copy of the written text on a standard computer monitor 500 in an active window 501. As shown in Fig. 5, the context of the unmatched word can be selected by the operator to be shown within the sentence it resides, word by word or in WO 00!31724 PCTNS99/27539 phrase context, by clicking on buttons 514, 515, and 516, respectively.
Associated with active window 501 is background window 502, which contains the copy of the written text file. As shown in background window 502, a incremental search has located (see pointer 503) the next occurrence of the current unmatched word "cash." Contemporaneously therewith, within window 505 containing the buffer from the speech recognition program, the same incremental search has located (see pointer 506) the next occurrence of the current unmatched word. A human user will likely only being viewing active window 501 activate the audio replay from the speech recognition program by clicking on "play" button 510, which plays the audio synchronized to IS the text at pointer 506. Based on that snippet of speech, which can be played aver and over by clicking on the play button, the human user can manually input the correction to the current unmatched word via keyboard, mousing actions, or possibly even audible cues to another speech recognition program running within this window.
In the present example, even if the choice of isolated context offered by buttons 514, 515 and 516, it may still be difficult to determine the correct verbatim word out-of -context, accordingly there is a switch window button 513 that will move background window 502 to the foreground with visible pointer 503 indicating the current location within the copy of the written text.
The user can then return to the active window and input the correct word, "trash." This change will only effect the copy of the written text displayed in background window 502.
._ CA 02351705 2001-05-16 WO 00!31724 PCTIU599127539 When the operator is ready for the next unmatched word, the operator clicks on the advance button 511, which advances pointer 406c dawn the list of unmatched words and activates the incremental search in both window 502 and 505. This unmatched word is now displayed in isolation and the operator can play the synchronized speech from the speech recognition program and correct this word as well. If at any point in the operation, the operator would like to return to a previous unmatched i0 word, the operator clicks on the reverse button 512, which moves pointer 406c back a word in the list and causes a backward incremental search to occur. This is accomplished by using the underlying list of objects which contains the original unmatched words. This list IS is traversed in object by object fashion, but alternatively each of the records could be padded such that each item has the same word size to assist in bi-directional traversing of the list. As the unmatched words in this underlying list are read only it is 20 possible to return to the original unmatched word such that the operator can determine if a different correction should have been made.
Ultimately, the copy of the written text is finally corrected resulting in a verbatim copy, which is saved to 25 the user's subdirectory. The verbatim file is also passed to the speech recognition program for training, step 407. The new (and improved) acoustic model is saved, step 408, and the speech recognition program is closed, step 409, As the-system is still in training, 30 the transcribed file is returned to the user, as in step 310 from the enrollment phase.
.. CA 02351705 2001-05-16 WO 00/31724 PCTNS99/27539 :.
_ 23 _ _ As shown in Fig. 4, the system may also include means for determining the accuracy rate from the output of the sequential comparing means. Specifically, by counting the number of words in the written text and the number of words in list 406b the ratio of words in said sequential list to words in said written text can be determined, thus providing an accuracy percentage. As before, it is a matter of choice when to advance users from one stage to another. Once that goal is reached, the user's profile is changed to the next stage, step 211.
Once the user has reached the automation stage, the greatest benefits of the present system can be achieved.
The speech recognition software is started, step 600, and the current user selected, step 601. Tf desired, a particularized vocabulary may be selected, step 602.
Then automatic conversion of the digital audio file recorded by the current user may commence, step 603.
When completed, the written file is transmitted to the user based on the information contained in the user profile, step 604 and the program is returned to the main loop.
Unfortunately, there may be instances where the voice users cannot use automated transcription for a period of time (during an illness, after dental work, etc.) because their acoustic model has been temporarily (or even permanently) altered. In that case, the system administrator may set the training status variable to a stop automation state in which steps 301, 302, 303, 305 and 310 (see Fig. 2b) are the only steps performed.
.. CA 02351705 2001-05-16 The foregoing description and drawings merely explain and illustrate the invention and the invention is not limited thereto. Those of the skill in the art who have the disclosure before them will be able to make modifications and variations therein without departing from the scope of the present invention. Far instance, it is possible to implement all of the elements of the present system on a single general-purpose computer by essentially time sharing the machine between the voice user, transcriptionist and the speech recognition program. The resulting cost saving makes this system accessible to more types of office situations not simply large medical clinics, hospital, law firms or other large entities.
The system may also include means for determining an accuracy rate for the current user. In one approach, this accuracy determination can be made by determining the ratio of the number of words in the sequential list of unmatched words to the number of wards in the written text.
The system additionally includes means for training the automatic speech converting means to achieve higher accuracy for the current user. In particular, the training means uses the verbatim text created by the manual editing means and the voice dictation file **.
The training means may also comprise a preexisting training portion of the preexisting speech recognition program. Thus, the training means would also include means for automating responses to a series of interactive inquiries from the preexisting training portion of the speech recognition program.
The system finally includes means for controlling the flow of the voice dictation file based upon the training status of the current user. said voice dictation WO 00/31724 PCT/US99/27539 _ _ 6 - _ file to at 3east one of said manual input means and said automatic speech converting means. The control means reads and modifies a user's training status such that it is an appropriate selection from the group of pre-y enrollment, enrollment, training, automation and stop automation. During a user's pre-enrollment phase the control means further includes means for creating a user identification and acoustic model within the automatic speech converting means. During this phase, the control means routes the voice dictation file to the automatic speech converting means and the manual input means, routes the written text and the transcribed file to the manual editing means, routes the verbatim text to the training means and routes the transcribed file back to the current user as a finished text.
During the training phase, the control means routes (1) the voice dictation file to the automatic speech converting means and the manual input means, (2) routes the written text and the transcribed file to the manual editing means, (3) routes the verbatim text to the training means and (4) routes the transcribed file back to the current user as a finished text.
During the automation stage, the control means routes (1) the voice dictation file only to the automatic speech converting means and (2) the written text back to the current user as a finished text.
The present application also discloses a method fox automating transcription services for one or more voice users in a system including a manual transcription station and a speech recognition program. The method comprising the steps of: (2) establishing a profile for WO 40/31724 PCTIUS99I2~S39 _ _ _ each of the voice users, the profile containing a training status; {2) receiving a voice dictation file from a current voice user; (3) choosing the training status of the current voice user from the group of enrollment, training, automated and stop automation; (4) routing the voice dictation file to at least one of the manual transcription station and the speech recognition program based on the training status; (5) receiving the voice dictation file in at least one of the manual transcription station and the speech recognition program;
(6) creating a transcribed file at the manual transcription station for each received voice dictation file; (7) automatically creating a written text with the speech recognition program for each received voice dictation file if the training status of the current user is training or automated; (8) manually establishing a verbatim file if the training status of the current user is enrollment or training; (9) training the speech recognition program with an acoustic model for the current user using the verbatim file and the voice dictation file if the training status of the current user is enrollment or training; {10) returning the transcribed file to the current user if the training status of the current user is enrollment or training; and (11) returning the written text to the current user if the training status of the current user is automated.
This method may also include the various substeps in the step of manually establishing a verbatim file which include: (1) assisting an operator to establish the verbatim file if the training status of the current user is training by: (a) sequentially comparing a copy of the written text with the transcribed file resulting in a WO 00/31724 PCTIUS99127539 _ _ g _ _ sequential list of unmatched words culled from the copy of the written text, the sequential list having-a beginning, an end and a current unmatched word, the current unmatched word being successively advanced from the beginning to the end; (b) incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; (c} displaying the current unmatched word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and (d) correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
The invention also includes a method for assisting an operator to establish a verbatim file from a written text created by a speech recognition program and a transcribed file created by a human transcriptionist, the method comprising the steps of: (1) sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched words culled from the copy of the written text, the sequential list having a beginning, an end and a current unmatched word, the current unmatched ward being successively advanced from the beginning to the end; (2) incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; (3) displaying the current unmatched g _ _.
word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and (4) correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
Brief Description c~~ the Drawings Fig. I of the drawings is a block diagram of one potential embodiment of the present system for substantially automating transcription services for one or more voice users;
Fig. 3.b of the drawings is a block diagram of a general-purpose computer which may be used as a dictation station, a transcription station and the control means within the present system;
Fig. 2a of the drawings is a flow diagram of the main loop of the control means of the present system;
Fig. 2b of the drawings is a flow diagram of the enrollment stage portion of the control means of the present system;
Fig. 2c of the drawings is a flow diagram of the training stage portion of the control means of the present system;
Fig. 2d of the drawings is a flow diagram of the automation stage portion ~f the control means of the present system;
Fig. 3 of the drawings is a directory structure used by the control means in the present system;
Fig. 4 of the drawings is a block diagram of a portion of a preferred embodiment of the manual editing means; and Fig. 5 of the drawings is an elevation view of the remainder of a preferred embodiment of the manual editing means.
Best Modes of Practicing the Invention While the present invention may be embodied in many different forms, there is shown in the drawings and discussed herein a few specific embodiments with the understanding that the present disclosure is to be considered only as an exemplification of the principles of the invention and is not intended to limit the invention to the embodiments illustrated.
Fig. 1 of the drawings generally shows one potential embodiment of the present system for substantially automating transcription services for one or more voice users. The present system must include some means for receiving a voice dictation file from a current user.
This voice dictation file receiving means can be a digital audio recorder, an analog audio recorder, or standard means for receiving computer files on magnetic media or via a data connection.
As shown, in one embodiment, the system 100 includes multiple digital recording stations 10, 11, 12 and 13.
Each digital recording station has at least a digital audio recorder and means for identifying the current voice user.
WO 00/31724 PCT/US99I27539 _ Preferably, each of these digital recording stations is implemented on a general-purpose computer (such as computer 20), although a specialized computer could be developed for this specific purpose. The general-purpose computer, though has the added advantage of being adaptable to varying uses in addition to operating within the present system 100. In general, the general-purpose computer should have, among other elements, a microprocessor (such as the Intel Corporation PENTIUM, Cyrix K6 or Motorola 68000 series); volatile and non-volatile memory; one or more mass storage devices (i.e.
HDD (not shown), floppy drive 21, and other removable media devices 22 such as a CD-ROM drive, DITTO, ZIP or JAZ drive (from Iomega Corporation) and the like);
various user input devices, such as a mouse 23, a keyboard 24, or a microphone 25; and a video display system 26. In one embodiment, the general-purpose computer is controlled by the WINDOWS 9.x operating system. It is contemplated, however, that the present system would work equally well using a MACINTOSH computer or even another operating system such as a WINDOWS CE, UNIX or a JAVA based operating system, to name a few.
Regardless of the particular computer platform used, in an embodiment utilizing an analog audio input (via microphone 25) the general-purpose computer must include a sound-card (not shown). Of course, in an embodiment with a digital input no sound card would be necessary.
In the embodiment shown in Fig. 1, digital audio recording stations 10, 11, 12 and 13 are loaded and configured to run digital audio recording software an a PENTIUM-based computer system operating under WINDOWS
WO OOJ3I724 PCT/US99J27539 _ 12 _ _ 9.x. Such digital recording software is available as a utility in the WINDOWS 9.x operating system or from various third party vendor such as The Programmers' Consortium, Tnc. of Oakton, Virginia (VOICEDOC), Syntrillium Corporation of Phoenix, Arizona {COOL EDIT}
or Dragon Systems Corporation (Dragon Naturally Speaking Professional Edition). These various software programs produce a voice dictation file in the form of a "WAV"
file. However, as would be known to those skilled in the art, other audio file formats, such as MP3 or DSS, could also be used to format the voice dictation file, without departing from the spirit of the present invention. In one embodiment where VOICEDOC software is used that software also automatically assigns a file handle to the WAV file, however, it would be known to those of ordinary skill in the art to save an audio file on a computer system using standard operating system file management methods.
Another means for receiving a voice dictation file is dedicated digital recorder 14, such as the Olympus Digital Voice Recorder D-1000 manufactured by the Olympus Corporation. Thus, if the current voice user is mare comfortable with a more conventional type of dictation device, they can continue to use a dedicated digital recorder 14. In order to harvest the digital audio text file, upon completion of a recording, dedicated digital recorder 14 would be operably connected to one of the digital audio recording stations, such as I3, toward downloading the digital a-udio file into that general-purpose computer. With this approach, for instance, no audio card would be required.
- 13 _ _ Another alternative for receiving the voice dictation file may consist of using one farm or another of removable magnetic media containing a pre-recorded audio file. With this alternative an operator would input the removable magnetic media into one of the digital audio recording stations toward uploading the audio file into the system.
In some cases it may be necessary to pre-process the audio files to make them acceptable for processing by the speech recognition software. For instance, a DSS file format may have to be changed to a WAV file format, or the sampling rate of a digital audio file may have to be upsampled or downsampled. For instance, in use the Olympus Digital Voice Recorder with Dragon Naturally Speaking, Olympus' 8MHz rate needs to be upsampled to 11 MHz. Software to accomplish such pre-processing is available from a variety of sources including Syntrillium Corporation and Olympus Corporation.
The other aspect of the digital audio~recording stations is some means for identifying the current voice user. The identifying means may include keyboard 24 upon which the user (or a separate operator} can input the current user's unique identification code. Of course, the user identification can be input using a myriad of computer input devices such as pointing devices (e. g.
mouse 23), a touch screen (not shown}, a light pen (not shown), bar-code reader (not shown} or audio cues via microphone 25, to name a few.
In the case of a first time user the identifying means may also assign that user an identification number after receiving potentially identifying information from .. CA 02351705 2001-05-16 WO 00/31724 PCT/US99/27539 _ that user, including: (1) name; (2) address; (3) occupation; (4) vocal dialect or accent; etc. As discussed in association with the control means, based upon this input information, a voice user profile and a sub-directory within the control means are established.
Thus, regardless of the particular identification means used, a user identification must be established for each voice user and subsequently provided with a corresponding digital audio file for each use such that the control means can appropriately route and the system ultimately transcribe the audio.
In one embodiment of the present invention, the identifying means may also seek the manual selection of a specialty vocabulary. It is contemplated that the specialty vocabulary sets may be general for various users such as medical (i.e. Radiology, Orthopedic Surgery, Gynecology} and legal (i.e. corporate, patent, litigation} or highly specific such that within each specialty the vocabulary parameters could be further limited based on the particular circumstances of a particular dictation file. For instance, if the current voice user is a Radiologist dictating the reading of a abdominal CAT scan the nomenclature is highly specialized and different from the nomenclature for a renal ultrasound. By narrowly segmenting each selectable vocabulary set an increase in the accuracy of the automatic speech converter is likely.
As shown in Fig. 1, the digital audio recording stations may be operably connected to system 100 as part of computer network 30 or, alternatively, they may be operably connected to the system via Internet host 15.
WO 00/31724 PCT/US99/27539 _ 15 _ _ As shown in Fig. lb, the general-purpose computer can be connected to both network jack 27 and telephone jack.
With the use of an Internet host, connection may be accomplished by e-mailing the audio file via the Internet. Another method fox completing such connection is by way of direct modem connection via remote control software, such as PC ANYWHERE, which is available from Symantec Corporation of Cupertino, California. It is also possible, if the IP address of digital audio recording station 10 or Internet host I5 is known, to transfer the audio file using basic file transfer protocol. Thus, as can be seen from the foregoing, the present system allows great flexibility for voice users to provide audio input into the system.
Control means 200 controls the flow of voice dictation file based upon the training status of the current voice user. As shown in Figs. 2a, 2b, 2c, 2d, control means 200 comprises a software program operating on general purpose computer 40. In particular, the program is initialized in step 201 where variable are set, buffers cleared and the particular configuration for this particular installation of the control means is loaded. Control means continually monitors a target directory (such as "current" (shown in Fig. 3)) to determine whether a new file has been moved into the target, step 202. Once a new file is found (such as "5723.id" (shown in Fig. 3)), a determination is made as to whether or not the current user 5 (shown in Fig. 1) is a new user, step 203. -For each new user (as indicated by the existence of a ".pro" file in the "current" subdirectory), a new WO 00/31?24 PCT/US99I27539 _ _ 1~ _ _ subdirectory is established, step 204 (such as the "usern" subdirectory (shown in Fig. 3)). This subdirectory is used to store all of the audio files ("xxxx.wav"), written text ("xxxx.wrt"), verbatim text ("xxxx.vb"), transcription text ("xxxx.txt") and user profile ("usern.pro") for that particular user. Each particular job is assigned a unique number "xxxx" such that all of the files associated with a job can be associated by that number. With this directory structure, the number of users is practically limited only by storage space within general-purpose computer 40.
Now that the user subdirectory has been established, the user profile is moved to the subdirectory, step 205.
The contents of this user profile may vary between systems. The contents of one potential user profile is shown in Fig. 3 as containing: the user name, address, occupation and training status. Aside from the training status variable, which is necessary, the other data is useful in routing and transcribing the audio files.
The control means, having selected one set of files by the handle, determines the identity of the current user by comparing the ".id" file with its "user.tbl,"
step 206. Now that the user is known the user profile may be parsed from that user's subdirectory and the current training status determined, step 207. Steps 208-211 are the triage of the current training status is one of: enrollment, training, automate, and stop automation.
Enrollment is the first stage in automating transcription services. As shown in Fig. 2b, the audio file is sent to transcription, step 301. In particular, the "xxxx.wav" file is transferred to transcriptionist stations 50 and 51. In a preferred embodiment, both stations are general-purpose computers, which run both an audio player and manual input means. The audio player is likely to be a digital audio player, although it is possible that an analog audio file could be transferred to the stations. Various audio players are commonly available including a utility in the WINDOWS 9.x operating system and various other third parties such from The Programmers' Consortium, Inc. of Oakton, Virginia (VOICESCRIBE). Regardless of the audio player used to play the audio file, manual input means is running on the computer at the same time. This manual input means may comprise any of text editor or word processor (such as MS WORD, WordPerfect, AmiPro or Word Pad) in combination with a keyboard, mouse, or other user-interface device. In one embodiment of the present invention, this manual input means may, itself, also be speech recognition software, such as Naturally Speaking from Dragon Systems of Newton, Massachusetts, Via Voice from TBM Corporation of Armonk, New York, or Speech Magic from Philips Corporation of Atlanta, Georgia. Human transcriptionist 6 listens to the audio file created by current user 5 and as is known, manually inputs the perceived contents of that recorded text, thus establishing the transcribed file, step 302. Being human, human transcriptionist 6 is likely to impose experience, education and biases on the text and thus not input a verbatim transcript of the audio file. Upon completion of the human transcription, the human transcriptionist 6 saves the file and indicates that it is ready for transfer to the current users subdirectory as "xxxx.txt", step 303.
WD 00/31724 PCT/US99127539 _ _ 1 g _ _.
Inasmuch as this current user is only at the enrollment stage, a human operator will have to listen to the audio file and manually compare it to the transcribed file and create a verbatim file, step 304. That verbatim S file "xxxx.vb" is also transferred to the current user's subdirectory, step 305. Now that verbatim text is available, control means 200 starts the automatic speech conversion means, step 306. This automatic speech conversion means may be a preexisting program, such as Dragon System's Naturally Speaking, IBM's Via Voice or Philips' Speech Magic, to name a few. Alternatively, it could be a unique program that is designed to specifically perform automated speech recagnition.
In a preferred embodiment, Dragon Systems' Naturally Speaking has been used by running an executable simultaneously with Naturally Speaking that feeds phantom keystrokes and mauling operations through the WIN32API, such that Naturally Speaking believes that it is interacting with a human being, when in fact it is being controlled by control means 200. Such techniques are well known in the computer software testing art and, thus, will not be discussed in detail. It should suffice to say that by watching the application flow of any speech recognition program, an executable to mimic the interactive manual steps can be created.
If the current user is a new user, the speech recognition program will need to establish the new user, step 307. Control means provides the necessary information from the user profile found in the current user's subdirectory. All speech recognition require significant training to establish an acoustic model of a .. CA 02351705 2001-05-16 WO 00/31724 PCT/US99/27539 _ _ 19 _ _ particular user. In the case of Dragon, initially the program seeks approximately 20 minutes of audio usually obtained by the user reading a canned text provided by Dragon Systems. There is also functionality built into Dragon that allows "mobile training." Using this feature, the verbatim file and audio file are fed into the speech recognition program to beginning training the acoustic model far that user, step 308. Regardless of the length of that audio file, control means 200 closes the speech recognition program at the completion of the file, step 309.
As the enrollment step is too soon to use the automatically created text, a copy of the transcribed file is sent to the current user using the address information contained in the user profile, step 310.
This address can be a street address or an e-mail address. Following that transmission, the program returns to the main loop on Fig. 2a.
After a certain number of minutes of training have been conducted for a particular user, that user's training status may be changed from enrollment to training. The border for this change is subjective, but perhaps a good rule of thumb is once Dragon appears to be creating written text with 80% accuracy or more, the switch between states can be made. Thus, for such a user the next transcription event will prompt control means 200 into the training state. As shown in Fig. 2c, steps 401-403 are the same human transcription steps as steps 301-303 in the enrollment phase. Once the transcribed file is established, control means 200 starts the automatic speech conversion means (or speech recognition ... CA 02351705 2001-05-16 WO 00/31724 PCT/US99/27539 _ program) and selects the current user, step 404. The audio file is fed into the speech recognition program and a written text is established within the program buffer, step 405. In the case of Dragon, this buffer is given the same file handle on very instance of the program.
Thus, that buffer can be easily copied using standard operating system commands and manual editing can begin, step 406.
Manual editing is not an easy task. Human beings are prone to errors. Thus, the present invention also includes means for improving on that task. As shown in Fig. 4, the transcribed file ("3333.txt") and the copy of the written text ("3333.wrt") are sequentially compared word by word 406a toward establishing sequential list of unmatched words 406b that are culled from the copy of the written text. This list has a beginning and an end and pointer 406c to the current unmatched word. Underlying the sequential list is another list of objects which contains the original unmatched words, as well as the words immediately before and after that unmatched word, the starting location in memory of each unmatched word in the sequential list of unmatched words 406b and the length of the unmatched word.
As shown in Fig. 5, the unmatched word pointed at by pointer 406c from list 406b is displayed in substantial visual isolation from the other text in the copy of the written text on a standard computer monitor 500 in an active window 501. As shown in Fig. 5, the context of the unmatched word can be selected by the operator to be shown within the sentence it resides, word by word or in WO 00!31724 PCTNS99/27539 phrase context, by clicking on buttons 514, 515, and 516, respectively.
Associated with active window 501 is background window 502, which contains the copy of the written text file. As shown in background window 502, a incremental search has located (see pointer 503) the next occurrence of the current unmatched word "cash." Contemporaneously therewith, within window 505 containing the buffer from the speech recognition program, the same incremental search has located (see pointer 506) the next occurrence of the current unmatched word. A human user will likely only being viewing active window 501 activate the audio replay from the speech recognition program by clicking on "play" button 510, which plays the audio synchronized to IS the text at pointer 506. Based on that snippet of speech, which can be played aver and over by clicking on the play button, the human user can manually input the correction to the current unmatched word via keyboard, mousing actions, or possibly even audible cues to another speech recognition program running within this window.
In the present example, even if the choice of isolated context offered by buttons 514, 515 and 516, it may still be difficult to determine the correct verbatim word out-of -context, accordingly there is a switch window button 513 that will move background window 502 to the foreground with visible pointer 503 indicating the current location within the copy of the written text.
The user can then return to the active window and input the correct word, "trash." This change will only effect the copy of the written text displayed in background window 502.
._ CA 02351705 2001-05-16 WO 00!31724 PCTIU599127539 When the operator is ready for the next unmatched word, the operator clicks on the advance button 511, which advances pointer 406c dawn the list of unmatched words and activates the incremental search in both window 502 and 505. This unmatched word is now displayed in isolation and the operator can play the synchronized speech from the speech recognition program and correct this word as well. If at any point in the operation, the operator would like to return to a previous unmatched i0 word, the operator clicks on the reverse button 512, which moves pointer 406c back a word in the list and causes a backward incremental search to occur. This is accomplished by using the underlying list of objects which contains the original unmatched words. This list IS is traversed in object by object fashion, but alternatively each of the records could be padded such that each item has the same word size to assist in bi-directional traversing of the list. As the unmatched words in this underlying list are read only it is 20 possible to return to the original unmatched word such that the operator can determine if a different correction should have been made.
Ultimately, the copy of the written text is finally corrected resulting in a verbatim copy, which is saved to 25 the user's subdirectory. The verbatim file is also passed to the speech recognition program for training, step 407. The new (and improved) acoustic model is saved, step 408, and the speech recognition program is closed, step 409, As the-system is still in training, 30 the transcribed file is returned to the user, as in step 310 from the enrollment phase.
.. CA 02351705 2001-05-16 WO 00/31724 PCTNS99/27539 :.
_ 23 _ _ As shown in Fig. 4, the system may also include means for determining the accuracy rate from the output of the sequential comparing means. Specifically, by counting the number of words in the written text and the number of words in list 406b the ratio of words in said sequential list to words in said written text can be determined, thus providing an accuracy percentage. As before, it is a matter of choice when to advance users from one stage to another. Once that goal is reached, the user's profile is changed to the next stage, step 211.
Once the user has reached the automation stage, the greatest benefits of the present system can be achieved.
The speech recognition software is started, step 600, and the current user selected, step 601. Tf desired, a particularized vocabulary may be selected, step 602.
Then automatic conversion of the digital audio file recorded by the current user may commence, step 603.
When completed, the written file is transmitted to the user based on the information contained in the user profile, step 604 and the program is returned to the main loop.
Unfortunately, there may be instances where the voice users cannot use automated transcription for a period of time (during an illness, after dental work, etc.) because their acoustic model has been temporarily (or even permanently) altered. In that case, the system administrator may set the training status variable to a stop automation state in which steps 301, 302, 303, 305 and 310 (see Fig. 2b) are the only steps performed.
.. CA 02351705 2001-05-16 The foregoing description and drawings merely explain and illustrate the invention and the invention is not limited thereto. Those of the skill in the art who have the disclosure before them will be able to make modifications and variations therein without departing from the scope of the present invention. Far instance, it is possible to implement all of the elements of the present system on a single general-purpose computer by essentially time sharing the machine between the voice user, transcriptionist and the speech recognition program. The resulting cost saving makes this system accessible to more types of office situations not simply large medical clinics, hospital, law firms or other large entities.
Claims (32)
1. A system for automating transcription services for one or more voice users, said system comprising:
- means for receiving a voice dictation file from a current user, said current user being one of said one or more voice users;
- an audio player used to audibly reproduce said voice dictation file;
- means for manually inputting and creating a transcribed file based on humanly perceived contents of said voice dictation file;
- means for automatically converting said voice dictation file into written text;
- means for manually editing a copy of said written text to create a verbatim text of said voice dictation file;
- means for training said automatic converting means to achieve higher accuracy with said voice dictation file of current user; and - means for controlling flow of said voice dictation file based upon a training status of said current user, whereby said controlling means sends said voice dictation file to at least one of said manual input means and said automatic speech converting means.
- means for receiving a voice dictation file from a current user, said current user being one of said one or more voice users;
- an audio player used to audibly reproduce said voice dictation file;
- means for manually inputting and creating a transcribed file based on humanly perceived contents of said voice dictation file;
- means for automatically converting said voice dictation file into written text;
- means for manually editing a copy of said written text to create a verbatim text of said voice dictation file;
- means for training said automatic converting means to achieve higher accuracy with said voice dictation file of current user; and - means for controlling flow of said voice dictation file based upon a training status of said current user, whereby said controlling means sends said voice dictation file to at least one of said manual input means and said automatic speech converting means.
2. The invention according to Claim 1 wherein said written text is at least temporarily synchronized to said voice dictation file, said manual editing means comprises:
- means for sequentially comparing a copy of said written text with said transcribed file resulting in a sequential list of unmatched words culled from said copy of said written text, said sequential list having a beginning, an end and a current unmatched word, said current unmatched word being successively advanced from said beginning to said end;
- means for incrementally searching for said current unmatched word contemporaneously within a first buffer associated with a speech recognition program containing said written text and a second buffer associated with said sequential list; and - means for correcting said current unmatched word in said second buffer, said correcting means including means for displaying said current unmatched word in a manner substantially visually isolated from other text in said copy of said written text and means for playing a portion of a synchronized voice dictation recording from said first buffer associated with said current unmatched word.
- means for sequentially comparing a copy of said written text with said transcribed file resulting in a sequential list of unmatched words culled from said copy of said written text, said sequential list having a beginning, an end and a current unmatched word, said current unmatched word being successively advanced from said beginning to said end;
- means for incrementally searching for said current unmatched word contemporaneously within a first buffer associated with a speech recognition program containing said written text and a second buffer associated with said sequential list; and - means for correcting said current unmatched word in said second buffer, said correcting means including means for displaying said current unmatched word in a manner substantially visually isolated from other text in said copy of said written text and means for playing a portion of a synchronized voice dictation recording from said first buffer associated with said current unmatched word.
3. The invention according to Claim 2 wherein said correcting means further includes means for alternatively viewing said current unmatched word in context within said copy of said written text.
4. The invention according to Claim 2 further including means for determining an accuracy rate for said current user.
5. The invention according to Claim 3 wherein said sequential list and said written text each have a respective number of words, said accuracy rate determining means determining a ratio of said number of words in said sequential list to said number of words in said written text.
6. The invention according to Claim 5 wherein said training status is based on said accuracy rate.
7. The invention according to Claim 1 wherein said automatic converting means comprises a preexisting speech recognition program intended for human interactive use, said automatic converting means includes means for automating responses to a series of interactive inquiries from said preexisting speech recognition program.
8. The invention according to Claim 7 wherein said training means comprises a preexisting training portion of said preexisting speech recognition program intended for human interactive use, said training means includes means for automating responses to a series of interactive inquiries from said preexisting training portion of said preexisting speech recognition program.
9. The invention according to Claim 1 wherein said training means comprises a preexisting training program intended for human interactive use, said training means includes means for automating responses to a series of interactive inquiries from said preexisting training program.
10. The invention according to Claim 1 wherein said control means reads and modifies a user profile associated with said current user, each of said user profiles including said training status of said current user.
11. The invention according to Claim 10 wherein said training status is selected from the group of pre-enrollment, enrollment, training, automation and stop automation.
12. The invention according to Claim 11 when said training status is pre-enrollment said control means further includes means for creating a user identification and acoustic model within said automatic speech converting means.
13. The invention according to Claim 11 when said training status is enrollment said control means routes said voice 1 dictation file to said automatic converting means and said manual input means, routes said written text and said transcribed file to said manual editing means, routes said verbatim text to said training means and routes said transcribed file back to said current user as a finished text.
14. The invention according to Claim 11 when said training status is training said control means routes said voice dictation file to said automatic converting means and said manual input means, routes said written text and said transcribed file to said manual editing means, routes said verbatim text to said training means and routes said transcribed file back to said current user as a finished text.
15. The invention according to Claim 11 when said training status is automation said control means routes said voice dictation file only to said automatic speech converting means and routes said written text back to said current user as a finished text.
16. The invention according to Claim 10 further including means for determining an accuracy rate for said current user, wherein said training status is based in part upon said accuracy rate.
17. The invention according to Claim 1 further comprising means for manually selecting a specialized language model.
18. The invention according to Claim 17 wherein said manual language model selection means is associated with said voice dictation file receiving means such that said current user can select said specialized language model.
19. The invention according to Claim 1 wherein said voice dictation file receiving means is implemented by a general purpose computer.
20. The invention according to Claim 19 wherein said general purpose computer implementing said voice dictation file receiving means is remotely located, said voice dictation file receiving means is operably connected to said audio player and said automatic speech conversion means by a computer network.
21. The invention according to Claim 19 wherein said general purpose computer implementing said voice dictation file receiving means is remotely located, said general purpose computer further including means for transferring said voice dictation file via the Internet.
22. The invention according to Claim 1 wherein said audio player is implemented by a general purpose computer.
23. The invention according to Claim 22 wherein said general purpose computer implementing said audio player is remotely located, said audio player is operably connected to said voice dictation file receiving means and said automatic speech conversion means by a computer network.
24. The invention according to Claim 22 wherein said general purpose computer implementing said audio player is remotely located, said general purpose computer further including means for receiving said voice dictation file and transmitting said transcribed text via the Internet.
25. An apparatus for substantially simplifying the production of a verbatim text of a voice dictation recording based upon a transcribed file produced by a human transcriptionist and a written text produced by a speech recognition program, wherein said written text is at least temporarily synchronized to said voice dictation recording, said apparatus comprising:
- means for sequentially comparing a copy of said written text with said transcribed file resulting in a sequential list of unmatched words culled from said copy of said written text, said sequential list having a beginning, an end and a current unmatched word, said current unmatched word being successively advanced from said beginning to said end;
- means for incrementally searching for said current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing said written text and a second buffer associated with said sequential list; and - means for correcting said current unmatched word in said second buffer, said correcting means including means for displaying said current unmatched word in a manner substantially visually isolated from other text in said copy of said written text and means for playing a portion of said synchronized voice dictation recording from said first buffer associated with said current unmatched word.
- means for sequentially comparing a copy of said written text with said transcribed file resulting in a sequential list of unmatched words culled from said copy of said written text, said sequential list having a beginning, an end and a current unmatched word, said current unmatched word being successively advanced from said beginning to said end;
- means for incrementally searching for said current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing said written text and a second buffer associated with said sequential list; and - means for correcting said current unmatched word in said second buffer, said correcting means including means for displaying said current unmatched word in a manner substantially visually isolated from other text in said copy of said written text and means for playing a portion of said synchronized voice dictation recording from said first buffer associated with said current unmatched word.
26. The invention according to Claim 25 wherein said correcting means further includes means for alternatively viewing said current unmatched word in context within said copy of said written text.
27. The invention according to Claim 25 wherein said manner substantially visually isolated from other text can be manually selected from the group containing word-by-word display, sentence-by-sentence display and said current unmatched word display.
28. The invention according to Claim 25 further including means for determining an accuracy rate for said current user.
29. The invention according to Claim 28 wherein said sequential list and written text each have a respective number of words, said accuracy rate determining means determining a ratio of said number of words in said sequential list to said number of words in said written text.
30. A method for automating transcription services for one or more voice users in a system including a manual transcription station and a speech recognition program, said method comprising the steps of:
- establishing a profile for each of the voice users, the profile containing a training status;
- receiving a voice dictation file from a current voice user;
- choosing the training status of the current voice user from the group of enrollment, training, automated and stop automation;
- routing the voice dictation file to at least one of the manual transcription station and the speech recognition program based on the training status;
- receiving the voice dictation file in at least one of the manual transcription station and the speech recognition program;
- creating a transcribed file at the manual transcription station for each received voice dictation file;
- automatically creating a written text with the speech recognition program for each received voice dictation file if the training status of the current user is training or automated;
- manually establishing a verbatim file if the training status of the current user is enrollment or training;
- training the speech recognition program with an acoustic model for the current user using the verbatim file and the voice dictation file if the training status of the current user is enrollment or training;
- returning the transcribed file to the current user if the training status of the current user is enrollment or training; and - returning the written text to the current user if the training status of the current user is automated.
- establishing a profile for each of the voice users, the profile containing a training status;
- receiving a voice dictation file from a current voice user;
- choosing the training status of the current voice user from the group of enrollment, training, automated and stop automation;
- routing the voice dictation file to at least one of the manual transcription station and the speech recognition program based on the training status;
- receiving the voice dictation file in at least one of the manual transcription station and the speech recognition program;
- creating a transcribed file at the manual transcription station for each received voice dictation file;
- automatically creating a written text with the speech recognition program for each received voice dictation file if the training status of the current user is training or automated;
- manually establishing a verbatim file if the training status of the current user is enrollment or training;
- training the speech recognition program with an acoustic model for the current user using the verbatim file and the voice dictation file if the training status of the current user is enrollment or training;
- returning the transcribed file to the current user if the training status of the current user is enrollment or training; and - returning the written text to the current user if the training status of the current user is automated.
31. The invention according to Claim 30 wherein said step of manually establishing a verbatim file includes the sub-steps of:
- assisting an operator to establish the verbatim file if the training status of the current user is training by:
- sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched words culled from the copy of the written text, the sequential list having a beginning, an end and a current unmatched word, the current unmatched word being successively advanced from the beginning to the end;
- incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; and - displaying the current unmatched word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and - correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
- assisting an operator to establish the verbatim file if the training status of the current user is training by:
- sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched words culled from the copy of the written text, the sequential list having a beginning, an end and a current unmatched word, the current unmatched word being successively advanced from the beginning to the end;
- incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; and - displaying the current unmatched word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and - correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
32. A method for assisting an operator to establish a verbatim file from a written text created by a speech recognition program and a transcribed file created by a human transcriptionist, the method comprising the steps of:
- sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched wards culled from the copy of the written text, the sequential list having a beginning, an end and a current unmatched word, the current unmatched word being successively advanced from the beginning to the end;
- incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; and - displaying the current unmatched word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
- sequentially comparing a copy of the written text with the transcribed file resulting in a sequential list of unmatched wards culled from the copy of the written text, the sequential list having a beginning, an end and a current unmatched word, the current unmatched word being successively advanced from the beginning to the end;
- incrementally searching for the current unmatched word contemporaneously within a first buffer associated with the speech recognition program containing the written text and a second buffer associated with the sequential list; and - displaying the current unmatched word in a manner substantially visually isolated from other text in the copy of the written text and playing a portion of the synchronized voice dictation recording from the first buffer associated with the current unmatched word; and correcting the current unmatched word to be a verbatim representation of the portion of the synchronized voice dictation recording.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US09/197,313 US6122614A (en) | 1998-11-20 | 1998-11-20 | System and method for automating transcription services |
| US09/197,313 | 1998-11-20 | ||
| PCT/US1999/027539 WO2000031724A1 (en) | 1998-11-20 | 1999-11-19 | System and method for automating transcription services |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2351705A1 CA2351705A1 (en) | 2000-06-02 |
| CA2351705C true CA2351705C (en) | 2005-11-15 |
Family
ID=22728894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002351705A Expired - Fee Related CA2351705C (en) | 1998-11-20 | 1999-11-19 | System and method for automating transcription services |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US6122614A (en) |
| AU (1) | AU1824700A (en) |
| CA (1) | CA2351705C (en) |
| WO (1) | WO2000031724A1 (en) |
Families Citing this family (233)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6311157B1 (en) * | 1992-12-31 | 2001-10-30 | Apple Computer, Inc. | Assigning meanings to utterances in a speech recognition system |
| US7249026B1 (en) * | 1993-03-24 | 2007-07-24 | Engate Llc | Attorney terminal having outline preparation capabilities for managing trial proceedings |
| US6604124B1 (en) * | 1997-03-13 | 2003-08-05 | A:\Scribes Corporation | Systems and methods for automatically managing work flow based on tracking job step completion status |
| US6571211B1 (en) | 1997-11-21 | 2003-05-27 | Dictaphone Corporation | Voice file header data in portable digital audio recorder |
| US6671567B1 (en) | 1997-11-21 | 2003-12-30 | Dictaphone Corporation | Voice file management in portable digital audio recorder |
| US7203288B1 (en) * | 1997-11-21 | 2007-04-10 | Dictaphone Corporation | Intelligent routing of voice files in voice data management system |
| DE19809563A1 (en) * | 1998-03-05 | 1999-09-09 | Siemens Ag | Medical work station for treating patient |
| US6360237B1 (en) * | 1998-10-05 | 2002-03-19 | Lernout & Hauspie Speech Products N.V. | Method and system for performing text edits during audio recording playback |
| US6839669B1 (en) * | 1998-11-05 | 2005-01-04 | Scansoft, Inc. | Performing actions identified in recognized speech |
| US6324507B1 (en) * | 1999-02-10 | 2001-11-27 | International Business Machines Corp. | Speech recognition enrollment for non-readers and displayless devices |
| US6961699B1 (en) * | 1999-02-19 | 2005-11-01 | Custom Speech Usa, Inc. | Automated transcription system and method using two speech converting instances and computer-assisted correction |
| US20050261907A1 (en) | 1999-04-12 | 2005-11-24 | Ben Franklin Patent Holding Llc | Voice integration platform |
| US6408272B1 (en) * | 1999-04-12 | 2002-06-18 | General Magic, Inc. | Distributed voice user interface |
| US6298326B1 (en) * | 1999-05-13 | 2001-10-02 | Alan Feller | Off-site data entry system |
| US6493672B2 (en) | 1999-05-26 | 2002-12-10 | Dictaphone Corporation | Automatic distribution of voice files |
| US6374223B1 (en) * | 1999-06-11 | 2002-04-16 | Lucent Technologies, Inc. | Internet to voice mail messaging |
| US6477493B1 (en) * | 1999-07-15 | 2002-11-05 | International Business Machines Corporation | Off site voice enrollment on a transcription device for speech recognition |
| US6904405B2 (en) * | 1999-07-17 | 2005-06-07 | Edwin A. Suominen | Message recognition using shared language model |
| US6704709B1 (en) * | 1999-07-28 | 2004-03-09 | Custom Speech Usa, Inc. | System and method for improving the accuracy of a speech recognition program |
| US7689416B1 (en) | 1999-09-29 | 2010-03-30 | Poirier Darrell A | System for transferring personalize matter from one computer to another |
| US6415258B1 (en) * | 1999-10-06 | 2002-07-02 | Microsoft Corporation | Background audio recovery system |
| US7017161B1 (en) | 1999-10-11 | 2006-03-21 | Dictaphone Corporation | System and method for interfacing a radiology information system to a central dictation system |
| US6789060B1 (en) * | 1999-11-01 | 2004-09-07 | Gene J. Wolfe | Network based speech transcription that maintains dynamic templates |
| US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
| US6738784B1 (en) * | 2000-04-06 | 2004-05-18 | Dictaphone Corporation | Document and information processing system |
| EP1317749B1 (en) * | 2000-07-31 | 2007-05-09 | Eliza Corporation | Method of and system for improving accuracy in a speech recognition system |
| US7401023B1 (en) * | 2000-09-06 | 2008-07-15 | Verizon Corporate Services Group Inc. | Systems and methods for providing automated directory assistance using transcripts |
| US7236932B1 (en) * | 2000-09-12 | 2007-06-26 | Avaya Technology Corp. | Method of and apparatus for improving productivity of human reviewers of automatically transcribed documents generated by media conversion systems |
| US7792676B2 (en) * | 2000-10-25 | 2010-09-07 | Robert Glenn Klinefelter | System, method, and apparatus for providing interpretive communication on a network |
| US6980953B1 (en) * | 2000-10-31 | 2005-12-27 | International Business Machines Corp. | Real-time remote transcription or translation service |
| US6832189B1 (en) * | 2000-11-15 | 2004-12-14 | International Business Machines Corporation | Integration of speech recognition and stenographic services for improved ASR training |
| US20020072914A1 (en) * | 2000-12-08 | 2002-06-13 | Hiyan Alshawi | Method and apparatus for creation and user-customization of speech-enabled services |
| US7558735B1 (en) * | 2000-12-28 | 2009-07-07 | Vianeta Communication | Transcription application infrastructure and methodology |
| US8213910B2 (en) * | 2001-02-09 | 2012-07-03 | Harris Technology, Llc | Telephone using a connection network for processing data remotely from the telephone |
| US6826241B2 (en) | 2001-02-21 | 2004-11-30 | Motorola, Inc. | Apparatus and method for filtering maximum-length-code signals in a spread spectrum communication system |
| WO2002071390A1 (en) * | 2001-03-01 | 2002-09-12 | Ordinate Corporation | A system for measuring intelligibility of spoken language |
| ATE300084T1 (en) * | 2001-03-16 | 2005-08-15 | Koninkl Philips Electronics Nv | TRANSCRIPTION SERVICE WITH CANCEL OF AUTOMATIC TRANSCRIPTION |
| US20020169615A1 (en) * | 2001-03-23 | 2002-11-14 | Irwin Kruger | Computerized voice-controlled system for compiling quality control data |
| JP5093966B2 (en) | 2001-03-29 | 2012-12-12 | ニュアンス コミュニケーションズ オーストリア ゲーエムベーハー | Alignment of voice cursor and text cursor during editing |
| US6834264B2 (en) | 2001-03-29 | 2004-12-21 | Provox Technologies Corporation | Method and apparatus for voice dictation and document production |
| US7200565B2 (en) * | 2001-04-17 | 2007-04-03 | International Business Machines Corporation | System and method for promoting the use of a selected software product having an adaptation module |
| DE60220794T2 (en) * | 2001-06-21 | 2008-03-06 | Koninklijke Philips Electronics N.V. | METHOD FOR TRAINING A CUSTOMER-ORIENTED APPLICATION DEVICE THROUGH LANGUAGE INPUTS, WITH PROGRESSING OUT BY AN ANIMATED CHARACTER WITH DIFFERENT TIRED STATE, WHICH ARE EACH ASSIGNED TO PROGRESS, AND DEVICE FOR CARRYING OUT THE PROCESS |
| US7668718B2 (en) * | 2001-07-17 | 2010-02-23 | Custom Speech Usa, Inc. | Synchronized pattern recognition source data processed by manual or automatic means for creation of shared speaker-dependent speech user profile |
| US20030050777A1 (en) * | 2001-09-07 | 2003-03-13 | Walker William Donald | System and method for automatic transcription of conversations |
| US7133829B2 (en) * | 2001-10-31 | 2006-11-07 | Dictaphone Corporation | Dynamic insertion of a speech recognition engine within a distributed speech recognition system |
| US7146321B2 (en) * | 2001-10-31 | 2006-12-05 | Dictaphone Corporation | Distributed speech recognition system |
| US20030101054A1 (en) * | 2001-11-27 | 2003-05-29 | Ncc, Llc | Integrated system and method for electronic speech recognition and transcription |
| US6766294B2 (en) * | 2001-11-30 | 2004-07-20 | Dictaphone Corporation | Performance gauge for a distributed speech recognition system |
| US6785654B2 (en) | 2001-11-30 | 2004-08-31 | Dictaphone Corporation | Distributed speech recognition system with speech recognition engines offering multiple functionalities |
| US6990445B2 (en) * | 2001-12-17 | 2006-01-24 | Xl8 Systems, Inc. | System and method for speech recognition and transcription |
| US20030128856A1 (en) * | 2002-01-08 | 2003-07-10 | Boor Steven E. | Digitally programmable gain amplifier |
| US7257531B2 (en) * | 2002-04-19 | 2007-08-14 | Medcom Information Systems, Inc. | Speech to text system using controlled vocabulary indices |
| US7236931B2 (en) | 2002-05-01 | 2007-06-26 | Usb Ag, Stamford Branch | Systems and methods for automatic acoustic speaker adaptation in computer-assisted transcription systems |
| US7292975B2 (en) * | 2002-05-01 | 2007-11-06 | Nuance Communications, Inc. | Systems and methods for evaluating speaker suitability for automatic speech recognition aided transcription |
| US7590534B2 (en) * | 2002-05-09 | 2009-09-15 | Healthsense, Inc. | Method and apparatus for processing voice data |
| US7260534B2 (en) * | 2002-07-16 | 2007-08-21 | International Business Machines Corporation | Graphical user interface for determining speech recognition accuracy |
| CN1675642A (en) * | 2002-08-20 | 2005-09-28 | 皇家飞利浦电子股份有限公司 | Method to route jobs |
| US7016844B2 (en) * | 2002-09-26 | 2006-03-21 | Core Mobility, Inc. | System and method for online transcription services |
| US7539086B2 (en) * | 2002-10-23 | 2009-05-26 | J2 Global Communications, Inc. | System and method for the secure, real-time, high accuracy conversion of general-quality speech into text |
| US7774694B2 (en) * | 2002-12-06 | 2010-08-10 | 3M Innovation Properties Company | Method and system for server-based sequential insertion processing of speech recognition results |
| US7444285B2 (en) * | 2002-12-06 | 2008-10-28 | 3M Innovative Properties Company | Method and system for sequential insertion of speech recognition results to facilitate deferred transcription services |
| US20050096910A1 (en) * | 2002-12-06 | 2005-05-05 | Watson Kirk L. | Formed document templates and related methods and systems for automated sequential insertion of speech recognition results |
| US20040138898A1 (en) * | 2003-01-10 | 2004-07-15 | Elbrader Robert E. | Methods and apparatus for making and keeping records |
| US7516070B2 (en) * | 2003-02-19 | 2009-04-07 | Custom Speech Usa, Inc. | Method for simultaneously creating audio-aligned final and verbatim text with the assistance of a speech recognition program as may be useful in form completion using a verbal entry method |
| WO2004097791A2 (en) * | 2003-04-29 | 2004-11-11 | Custom Speech Usa, Inc. | Methods and systems for creating a second generation session file |
| US7046777B2 (en) | 2003-06-02 | 2006-05-16 | International Business Machines Corporation | IVR customer address acquisition method |
| KR100600522B1 (en) * | 2003-12-16 | 2006-07-13 | 에스케이 주식회사 | Quality of service ensuring call routing system using agents and automatic speech reconition enging and method thereof |
| US7764771B2 (en) * | 2003-12-24 | 2010-07-27 | Kimberly-Clark Worldwide, Inc. | Method of recording invention disclosures |
| US7660715B1 (en) * | 2004-01-12 | 2010-02-09 | Avaya Inc. | Transparent monitoring and intervention to improve automatic adaptation of speech models |
| WO2005086021A2 (en) * | 2004-03-02 | 2005-09-15 | Melingo, Ltd. | Embedded translation document method and system |
| US7319961B2 (en) * | 2004-03-04 | 2008-01-15 | Saudi Arabian Oil Company | Voice recognition technology to capture geoscience data |
| US8504369B1 (en) | 2004-06-02 | 2013-08-06 | Nuance Communications, Inc. | Multi-cursor transcription editing |
| GB2432704B (en) * | 2004-07-30 | 2009-12-09 | Dictaphone Corp | A system and method for report level confidence |
| US7584103B2 (en) * | 2004-08-20 | 2009-09-01 | Multimodal Technologies, Inc. | Automated extraction of semantic content and generation of a structured document from speech |
| US8335688B2 (en) | 2004-08-20 | 2012-12-18 | Multimodal Technologies, Llc | Document transcription system training |
| US8412521B2 (en) * | 2004-08-20 | 2013-04-02 | Multimodal Technologies, Llc | Discriminative training of document transcription system |
| US20060111917A1 (en) * | 2004-11-19 | 2006-05-25 | International Business Machines Corporation | Method and system for transcribing speech on demand using a trascription portlet |
| US7836412B1 (en) | 2004-12-03 | 2010-11-16 | Escription, Inc. | Transcription editing |
| US20060173829A1 (en) * | 2005-01-10 | 2006-08-03 | Neeman Yoni M | Embedded translation-enhanced search |
| US7613610B1 (en) * | 2005-03-14 | 2009-11-03 | Escription, Inc. | Transcription data extraction |
| US7447636B1 (en) * | 2005-05-12 | 2008-11-04 | Verizon Corporate Services Group Inc. | System and methods for using transcripts to train an automated directory assistance service |
| WO2006125346A1 (en) * | 2005-05-27 | 2006-11-30 | Intel Corporation | Automatic text-speech mapping tool |
| US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
| US8032372B1 (en) | 2005-09-13 | 2011-10-04 | Escription, Inc. | Dictation selection |
| CN100351820C (en) * | 2005-09-30 | 2007-11-28 | 迈世亚（北京）科技有限公司 | Method for duplicating musical composition in CD into MP3 realplayer |
| US20070078806A1 (en) * | 2005-10-05 | 2007-04-05 | Hinickle Judith A | Method and apparatus for evaluating the accuracy of transcribed documents and other documents |
| US7783488B2 (en) * | 2005-12-19 | 2010-08-24 | Nuance Communications, Inc. | Remote tracing and debugging of automatic speech recognition servers by speech reconstruction from cepstra and pitch information |
| US8036889B2 (en) * | 2006-02-27 | 2011-10-11 | Nuance Communications, Inc. | Systems and methods for filtering dictated and non-dictated sections of documents |
| US7653543B1 (en) | 2006-03-24 | 2010-01-26 | Avaya Inc. | Automatic signal adjustment based on intelligibility |
| US8121838B2 (en) | 2006-04-11 | 2012-02-21 | Nuance Communications, Inc. | Method and system for automatic transcription prioritization |
| US7831423B2 (en) * | 2006-05-25 | 2010-11-09 | Multimodal Technologies, Inc. | Replacing text representing a concept with an alternate written form of the concept |
| WO2007150005A2 (en) | 2006-06-22 | 2007-12-27 | Multimodal Technologies, Inc. | Automatic decision support |
| US8286071B1 (en) * | 2006-06-29 | 2012-10-09 | Escription, Inc. | Insertion of standard text in transcriptions |
| US7925508B1 (en) | 2006-08-22 | 2011-04-12 | Avaya Inc. | Detection of extreme hypoglycemia or hyperglycemia based on automatic analysis of speech patterns |
| US7962342B1 (en) | 2006-08-22 | 2011-06-14 | Avaya Inc. | Dynamic user interface for the temporarily impaired based on automatic analysis for speech patterns |
| US8521510B2 (en) | 2006-08-31 | 2013-08-27 | At&T Intellectual Property Ii, L.P. | Method and system for providing an automated web transcription service |
| US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
| US7899670B1 (en) * | 2006-12-21 | 2011-03-01 | Escription Inc. | Server-based speech recognition |
| US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
| US8041344B1 (en) | 2007-06-26 | 2011-10-18 | Avaya Inc. | Cooling off period prior to sending dependent on user's state |
| US20090037171A1 (en) * | 2007-08-03 | 2009-02-05 | Mcfarland Tim J | Real-time voice transcription system |
| US20090125299A1 (en) * | 2007-11-09 | 2009-05-14 | Jui-Chang Wang | Speech recognition system |
| CA2710310A1 (en) * | 2007-12-21 | 2009-07-02 | Nvoq Incorporated | Distributed dictation/transcription system |
| US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
| US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
| US8639512B2 (en) * | 2008-04-23 | 2014-01-28 | Nvoq Incorporated | Method and systems for measuring user performance with speech-to-text conversion for dictation systems |
| US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
| CA2665009C (en) * | 2008-05-23 | 2018-11-27 | Accenture Global Services Gmbh | System for handling a plurality of streaming voice signals for determination of responsive action thereto |
| CA2665014C (en) | 2008-05-23 | 2020-05-26 | Accenture Global Services Gmbh | Recognition processing of a plurality of streaming voice signals for determination of responsive action thereto |
| CA2665055C (en) * | 2008-05-23 | 2018-03-06 | Accenture Global Services Gmbh | Treatment processing of a plurality of streaming voice signals for determination of responsive action thereto |
| US20100030549A1 (en) | 2008-07-31 | 2010-02-04 | Lee Michael M | Mobile device having human language translation capability with positional feedback |
| US20100169092A1 (en) * | 2008-11-26 | 2010-07-01 | Backes Steven J | Voice interface ocx |
| US9959870B2 (en) | 2008-12-11 | 2018-05-01 | Apple Inc. | Speech recognition involving a mobile device |
| US20100299131A1 (en) * | 2009-05-21 | 2010-11-25 | Nexidia Inc. | Transcript alignment |
| US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
| US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
| US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
| US10255566B2 (en) | 2011-06-03 | 2019-04-09 | Apple Inc. | Generating and processing task items that represent tasks to perform |
| US9431006B2 (en) | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
| TWI383376B (en) * | 2009-08-14 | 2013-01-21 | Kuo Ping Yang | Method and system for voice communication |
| US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
| US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
| US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
| US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
| DE202011111062U1 (en) | 2010-01-25 | 2019-02-19 | Newvaluexchange Ltd. | Device and system for a digital conversation management platform |
| US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
| US9009040B2 (en) * | 2010-05-05 | 2015-04-14 | Cisco Technology, Inc. | Training a transcription system |
| US8959102B2 (en) | 2010-10-08 | 2015-02-17 | Mmodal Ip Llc | Structured searching of dynamic structured document corpuses |
| US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
| US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
| US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
| US8994660B2 (en) | 2011-08-29 | 2015-03-31 | Apple Inc. | Text correction processing |
| US9002699B2 (en) * | 2011-11-14 | 2015-04-07 | Microsoft Technology Licensing, Llc | Adaptive input language switching |
| US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
| US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
| US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
| US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
| US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
| US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
| US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
| WO2014052326A2 (en) * | 2012-09-25 | 2014-04-03 | Nvoq Incorporated | Apparatus and methods for managing resources for a system using voice recognition |
| KR20230137475A (en) | 2013-02-07 | 2023-10-04 | 애플 인크. | Voice trigger for a digital assistant |
| US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
| WO2014144579A1 (en) | 2013-03-15 | 2014-09-18 | Apple Inc. | System and method for updating an adaptive speech recognition model |
| AU2014233517B2 (en) | 2013-03-15 | 2017-05-25 | Apple Inc. | Training an at least partial voice command system |
| WO2014197336A1 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
| US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
| WO2014197334A2 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
| WO2014197335A1 (en) | 2013-06-08 | 2014-12-11 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
| US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
| EP3937002A1 (en) | 2013-06-09 | 2022-01-12 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
| AU2014278595B2 (en) | 2013-06-13 | 2017-04-06 | Apple Inc. | System and method for emergency calls initiated by voice command |
| DE112014003653B4 (en) | 2013-08-06 | 2024-04-18 | Apple Inc. | Automatically activate intelligent responses based on activities from remote devices |
| US20150073790A1 (en) * | 2013-09-09 | 2015-03-12 | Advanced Simulation Technology, inc. ("ASTi") | Auto transcription of voice networks |
| FR3011374B1 (en) * | 2013-09-30 | 2015-10-23 | Peugeot Citroen Automobiles Sa | METHOD OF CORRECTING A SEQUENCE OF WORDS ACQUIRED BY VOICE RECOGNITION |
| CN104575496A (en) * | 2013-10-14 | 2015-04-29 | 中兴通讯股份有限公司 | Method and device for automatically sending multimedia documents and mobile terminal |
| US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
| US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
| US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
| US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
| US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
| US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
| US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
| US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
| US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
| US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
| US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
| AU2015266863B2 (en) | 2014-05-30 | 2018-03-15 | Apple Inc. | Multi-command single utterance input method |
| US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
| US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
| US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
| US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
| US20160019101A1 (en) * | 2014-07-21 | 2016-01-21 | Ryan Steelberg | Content generation and tracking application, engine, system and method |
| US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
| US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
| US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
| US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
| US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
| US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
| US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
| US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
| US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
| US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
| US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
| US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
| US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
| US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
| US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
| US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
| US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
| US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
| US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
| US9578173B2 (en) | 2015-06-05 | 2017-02-21 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
| US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
| US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
| US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
| CN106340295B (en) * | 2015-07-06 | 2019-10-22 | 无锡天脉聚源传媒科技有限公司 | A kind of receiving method and device of speech recognition result |
| US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
| US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
| US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
| US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
| US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
| US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
| US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
| US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
| US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
| US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
| US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
| US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
| US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
| US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
| DK179588B1 (en) | 2016-06-09 | 2019-02-22 | Apple Inc. | Intelligent automated assistant in a home environment |
| US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
| US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
| US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
| US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
| US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
| DK179343B1 (en) | 2016-06-11 | 2018-05-14 | Apple Inc | Intelligent task discovery |
| DK179049B1 (en) | 2016-06-11 | 2017-09-18 | Apple Inc | Data driven natural language event detection and classification |
| DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
| DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
| US10706210B2 (en) * | 2016-08-31 | 2020-07-07 | Nuance Communications, Inc. | User interface for dictation application employing automatic speech recognition |
| US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
| US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
| US20180315428A1 (en) * | 2017-04-27 | 2018-11-01 | 3Play Media, Inc. | Efficient transcription systems and methods |
| DK201770439A1 (en) | 2017-05-11 | 2018-12-13 | Apple Inc. | Offline personal assistant |
| DK179496B1 (en) | 2017-05-12 | 2019-01-15 | Apple Inc. | USER-SPECIFIC Acoustic Models |
| DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
| DK201770431A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
| DK201770432A1 (en) | 2017-05-15 | 2018-12-21 | Apple Inc. | Hierarchical belief states for digital assistants |
| DK179560B1 (en) | 2017-05-16 | 2019-02-18 | Apple Inc. | Far-field extension for digital assistant services |
| CN108986793A (en) * | 2018-09-28 | 2018-12-11 | 北京百度网讯科技有限公司 | translation processing method, device and equipment |
| US11594242B2 (en) * | 2021-05-03 | 2023-02-28 | Gulfstream Aerospace Corporation | Noise event location and classification in an enclosed area |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4375083A (en) * | 1980-01-31 | 1983-02-22 | Bell Telephone Laboratories, Incorporated | Signal sequence editing method and apparatus with automatic time fitting of edited segments |
| US4430726A (en) * | 1981-06-18 | 1984-02-07 | Bell Telephone Laboratories, Incorporated | Dictation/transcription method and arrangement |
| US5179627A (en) * | 1987-02-10 | 1993-01-12 | Dictaphone Corporation | Digital dictation system |
| AU2868092A (en) * | 1991-09-30 | 1993-05-03 | Riverrun Technology | Method and apparatus for managing information |
| IT1256823B (en) * | 1992-05-14 | 1995-12-21 | Olivetti & Co Spa | PORTABLE CALCULATOR WITH VERBAL NOTES. |
| GB2302199B (en) * | 1996-09-24 | 1997-05-14 | Allvoice Computing Plc | Data processing method and apparatus |
| US5875448A (en) * | 1996-10-08 | 1999-02-23 | Boys; Donald R. | Data stream editing system including a hand-held voice-editing apparatus having a position-finding enunciator |
| US5995936A (en) * | 1997-02-04 | 1999-11-30 | Brais; Louis | Report generation system and method for capturing prose, audio, and video by voice command and automatically linking sound and image to formatted text locations |
| US5909667A (en) * | 1997-03-05 | 1999-06-01 | International Business Machines Corporation | Method and apparatus for fast voice selection of error words in dictated text |
-
1998
- 1998-11-20 US US09/197,313 patent/US6122614A/en not_active Expired - Fee Related
-
1999
- 1999-11-19 CA CA002351705A patent/CA2351705C/en not_active Expired - Fee Related
- 1999-11-19 WO PCT/US1999/027539 patent/WO2000031724A1/en active Application Filing
- 1999-11-19 AU AU18247/00A patent/AU1824700A/en not_active Abandoned
Also Published As
| Publication number | Publication date |
|---|---|
| US6122614A (en) | 2000-09-19 |
| AU1824700A (en) | 2000-06-13 |
| CA2351705A1 (en) | 2000-06-02 |
| WO2000031724A1 (en) | 2000-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2351705C (en) | System and method for automating transcription services | |
| US6961699B1 (en) | Automated transcription system and method using two speech converting instances and computer-assisted correction | |
| EP1183680B1 (en) | Automated transcription system and method using two speech converting instances and computer-assisted correction | |
| US7006967B1 (en) | System and method for automating transcription services | |
| US7979281B2 (en) | Methods and systems for creating a second generation session file | |
| US7516070B2 (en) | Method for simultaneously creating audio-aligned final and verbatim text with the assistance of a speech recognition program as may be useful in form completion using a verbal entry method | |
| US20020095290A1 (en) | Speech recognition program mapping tool to align an audio file to verbatim text | |
| US6490558B1 (en) | System and method for improving the accuracy of a speech recognition program through repetitive training | |
| US20030004724A1 (en) | Speech recognition program mapping tool to align an audio file to verbatim text | |
| US20080255837A1 (en) | Method for locating an audio segment within an audio file | |
| US20050131559A1 (en) | Method for locating an audio segment within an audio file | |
| CA2502412A1 (en) | A method for comparing a transcribed text file with a previously created file | |
| US7120581B2 (en) | System and method for identifying an identical audio segment using text comparison | |
| WO2000046787A2 (en) | System and method for automating transcription services | |
| AU2004233462B2 (en) | Automated transcription system and method using two speech converting instances and computer-assisted correction | |
| GB2390930A (en) | Foreign language speech recognition | |
| WO2001009877A9 (en) | System and method for improving the accuracy of a speech recognition program | |
| WO2001093058A1 (en) | System and method for comparing text generated in association with a speech recognition program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| FZDC | Correction of dead application (reinstatement) | ||
| MKLA | Lapsed | ||
| MKLA | Lapsed |
Effective date: 20121119 |