Systems and Methods for Concurrent Signal Recognition

• US 20130132082A1
• Filed: 02/21/2011
• Published: 05/23/2013
• Est. Priority Date: 02/21/2011
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Smaragdis, Paris
• Granted Patent: US 9,047,867 B2
• Time in Patent Office: Days
• US Class Current: 704/240
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence

Abstract

Methods and systems for recognition of concurrent, superimposed, or otherwise overlapping signals are described. A Markov Selection Model is introduced that, together with probabilistic decomposition methods, enable recognition of simultaneously emitted signals from various sources. For example, a signal mixture may include overlapping speech from different persons. In some instances, recognition may be performed without the need to separate signals or sources. As such, some of the techniques described herein may be useful in automatic transcription, noise reduction, teaching, electronic games, audio search and retrieval, medical and scientific applications, etc.

Claims

1. A non-transitory computer-readable storage medium having instructions stored thereon that, upon execution by a computer system, cause the computer system to perform operations comprising:

identifying a plurality of models, wherein each model corresponds to a respective utterance spoken by a person;

receiving a representation of a speech mixture including utterances concurrently spoken by at least two persons;

combining spectral vectors of the plurality of models into a set of spectral vectors;

calculating mixture weights for one or more vectors of the set of spectral vectors based, at least in part, on the representation of the speech mixture; and

identifying a concurrently spoken utterance in the speech mixture based, at least in part, on the mixture weights.

2. The non-transitory computer-readable storage medium of claim 1, wherein identifying further comprises recognizing without separating utterances concurrently spoken by the at least two persons.

3. The non-transitory computer-readable storage medium of claim 1, wherein the operations further comprise calculating a likelihood that a given model expresses the given utterance based, at least in part, on the mixture weights.

4. The non-transitory computer-readable storage medium of claim 3, wherein recognizing the given utterance comprises determining that the likelihood that the given model expresses the given utterance is greater than a likelihood that another model expresses the given utterance.

5. The non-transitory computer-readable storage medium of claim 1, wherein a models corresponding to an utterance is a Markov Selection Model including one or more dictionaries, each dictionary including one or more spectral vectors.

6. The non-transitory computer-readable storage medium of claim 5, wherein the utterance is expressed by the Markov Selection Model as a linear combination of the one or more spectral vectors.

7. The non-transitory computer-readable storage medium of claim 1, wherein a number of dictionaries for a model is determined dependent, at least in part, on a number of phonemes within an utterance described by the model.

8. The non-transitory computer-readable storage medium of claim 1, wherein one or more of the plurality of models is generated from on training data for one or more utterances.

9. The non-transitory computer-readable storage medium of claim 1, wherein the operations further comprise identifying a person who spoke the given utterance.

10. A method, comprising:

performing, by one or more computing devices;

identifying a first model corresponding to a first signal emitted by a first source, wherein the first model includes a first set of dictionaries and each of the first set of dictionaries includes a first set of spectral vectors;

identifying a second model corresponding to a second signal emitted by a second source, wherein the second model includes a second set of dictionaries and each of the second set of dictionaries includes a second set of spectral vectors;

receiving a representation of a signal mixture, wherein the signal mixture includes signals emitted by the first and second sources at least partially simultaneously;

combining spectral vectors of the first and second models into a superset of spectral vectors;

calculating a weight for each spectral vector of the superset of spectral vectors with respect to the signal mixture; and

recognizing at least one of the first and second signals within the signal mixture based, at least in part, on the calculated weights.

11. The method of claim 10, wherein the first signal emitted by the first source comprises a first utterance spoken by a first person and the second signal emitted by the second source comprises a second utterance spoken by a second person.

12. The method of claim 10, wherein recognizing further comprises recognizing without separating a signal emitted by the first source from another signal emitted by the second source.

13. The method of claim 10, wherein the performing further comprises calculating a likelihood that at least one of the first and second models expresses a portion of the signal mixture based, at least in part, on the calculated weights.

14. The method of claim 13, wherein the performing further comprises recognizing the first signal within the signal mixture in response to determining that a likelihood that the first model expresses the portion of the signal mixture is greater than a likelihood that the second model expresses the portion of the signal mixture.

15. The method of claim 10, wherein the first signal is expressed as a linear combination of the first set of spectral vectors.

16. A system, comprising:

at least one processor; and

a memory coupled to the at least one processor, wherein the memory stores program instructions, and wherein the program instructions are executable by the at least one processor to perform operations including;

receiving a representation of a signal mixture including a first signal emitted by a first source and a second signal emitted by a second source, wherein, within the signal mixture, the first and second signals overlap in time; and

recognizing the first signal within the signal mixture without separating the first signal from the second signal.

17. The method of claim 16, wherein the first signal emitted by the first source comprises a first utterance spoken by a first person and the second signal emitted by the second source comprises a second utterance spoken by a second person.

18. The system of claim 16, the operations further including:

storing a first Markov Selection Model corresponding to the first signal, wherein the first model includes a first set of dictionaries and each dictionary includes a first set of spectral vectors; and

storing a second Markov Selection Model corresponding to the second signal, wherein the second model includes a second set of dictionaries and each dictionary includes a second set of spectral vectors.

19. The system of claim 18, the operations further including:

combining the first and second sets of spectral vectors; and

calculating a weight for each spectral vector of the combination of spectral vectors with respect to the signal mixture, wherein recognizing the first signal within the signal mixture comprises recognizing the first signal based, at least in part, on the calculated weights.

20. The system of claim 19, the operations further including:

determining that a likelihood that the first model expresses the first signal within the signal mixture is greater than a likelihood that the second model expresses the second signal within the signal mixture based, at least in part, on the calculated weights.