Audio-visual speaker identification using coupled hidden markov models

• US 20050027530A1
• Filed: 07/31/2003
• Published: 02/03/2005
• Est. Priority Date: 07/31/2003
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Fu, Tieyan, Liang, Luhong, Liu, Xiaoxing, Nefian, Ara Victor, Pi, Xiaobo
• Application Number:
  US10/631,424
• Publication Number:
  US 20050027530A1
• Time in Patent Office: Days
• US Class Current: 704/256
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 17/10 Multimodal systems, i.e. based on the integration of multiple recognition engines or fusion of expert systems
• CPC Class Codes: G10L 17/16 Hidden Markov models [HMM]

Abstract

A phoneme and a viseme of a person may be modeled using a coupled hidden Markov model. The coupled hidden Markov model and a second model may be compared to identify the person.

Claims

1. A method comprising:

modeling a phoneme and a viseme of a person using a coupled hidden Markov model; and

comparing the coupled hidden Markov model and a second model to identify the person.

2. The method of claim 1 including utilizing a speaker-independent model having parameters and adapting the parameters to a speaker-dependent model.

3. The method of claim 2 wherein utilizing the speaker-independent model includes using estimation-maximization, and adapting the parameters includes using a maximum a posteriori method.

4. The method of claim 1 further including identifying the person based on a likelihood that the coupled hidden Markov model matches the second model.

5. The method of claim 1 further including modeling silence between consecutive words using a coupled hidden Markov model.

6. The method of claim 1 further including modeling silence between consecutive sentences using a coupled hidden Markov model.

7. An article comprising a medium storing instructions that, if executed, enable a processor-based system to:

model a phoneme and a viseme of a person using a coupled hidden Markov model; and

compare the coupled hidden Markov model and a second model to identify the person.

8. The article of claim 7 further storing instructions that, if executed, enable the system to utilize a speaker-independent model having parameters and to adapt the parameters to a speaker-dependent model.

9. The article of claim 7 further storing instructions that, if executed, enable the system to utilize a speaker-independent model using estimation-maximization and to adapt the parameters to a speaker-dependent model using a maximum a posteriori method.

10. The article of claim 7 further storing instructions that, if executed, enable the system to identify the person based on a likelihood that the coupled hidden Markov model matches the second model.

11. The article of claim 7 further storing instructions that, if executed, enable the system to model silence between consecutive words using a coupled hidden Markov model.

12. The article of claim 7 further storing instructions that, if executed, enable the system to model silence between consecutive sentences using a coupled hidden Markov model.

13. An apparatus comprising:

a model trainer to model a phoneme and a viseme of a person using a coupled hidden Markov model; and

a graph decoder to compare the coupled hidden Markov model and a second model to identify the person.

14. The apparatus of claim 13 further including a feature extractor to detect the viseme of the person.

15. The apparatus of claim 13 including the model trainer to utilize a speaker-independent model having parameters and to adapt the parameters to a speaker-dependent model.

16. The apparatus of claim 13 including the model trainer to utilize a speaker-independent model using estimation-maximization and to adapt the parameters to a speaker-dependent model using a maximum a posteriori method.

17. The apparatus of claim 13 including the graph decoder to identify the person based on a likelihood that the coupled hidden Markov model matches the second model.

18. The apparatus of claim 13 including the model trainer to model silence between consecutive words using a coupled hidden Markov model.

19. The apparatus of claim 13 including the model trainer to model silence between consecutive sentences using a coupled hidden Markov model.

20. A system comprising:

a processor-based device;

a graphics controller coupled to the processor-based device to receive data from the processor-based device; and

a storage coupled to the processor-based device storing instructions that, if executed, enable the processor-based device to;

model a phoneme and a viseme of a person using a coupled hidden Markov model, and

compare the coupled hidden Markov model and a second model to identify the person.

21. The system of claim 20 further storing instructions that, if executed, enable the processor-based device to utilize a speaker-independent model having parameters and to adapt the parameters to a speaker-dependent model.

22. The system of claim 20 further storing instructions that, if executed, enable the processor-based device to utilize a speaker-independent model using estimation-maximization and to adapt the parameters to a speaker-dependent model using a maximum a posteriori method.

23. The system of claim 20 further storing instructions that, if executed, enable the processor-based device to identify the person based on a likelihood that the coupled hidden Markov model matches the second model.

24. The system of claim 20 further storing instructions that, if executed, enable the processor-based device to model silence between consecutive words using a coupled hidden Markov model.

25. The system of claim 20 further storing instructions that, if executed, enable the processor-based device to model silence between consecutive sentences using a coupled hidden Markov model.