Head pose tracking system

• US 7,515,173 B2
• Filed: 05/23/2002
• Issued: 04/07/2009
• Est. Priority Date: 05/23/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yang, Ruigang, Zhang, Zhengyou
• Primary Examiner(s): Ramakrishnaiah; Melur
• Application Number:
  US10/154,892
• Publication Number:
  US 20030218672A1
• Time in Patent Office: 2,511 Days
• Field of Search: 348 1401- 1409, 348/14.1, 348/14.11, 348/14.12, 348/14.13, 348/14.16, 348/169, 382/118, 382/154
• US Class Current: 348/14.16
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Video images representative of a conferee'"'"'s head are received and evaluated with respect to a reference model to monitor a head position of the conferee. A personalized face model of the conferee is captured to track head position of the conferee. In a stereo implementation, first and second video images representative of a first conferee taken from different views are concurrently captured. A head position of the first conferee is tracked from the first and second video images. The tracking of head-position through a personalized model-based approach can be used in a number of applications such as human-computer interaction and eye-gaze correction for video conferencing.

Claims

1. A method, comprising:

receiving initial video images representative of a conferee'"'"'s head;

evaluating the initial video images to form an initial reference model having initial features to monitor a head position of the conferee;

tracking the head position of the conferee using updates to the initial features as subsequent video images are received; and

activating an auto-recovery tracking process using the initial reference model that does not included the updates when tracking of the conferee'"'"'s head is temporarily interrupted.

2. The method as recited in claim 1, wherein the reference model is a three-dimensional personalized face model of the conferee.

3. The method as recited in claim 1, further comprising building the reference model during a learning stage and storing the reference model in a database.

4. The method as recited in claim 1, further comprising assigning markers to the reference model and searching the received video images of the conferee'"'"'s head for points that match the markers in the reference model.

5. The method as recited in claim 1, wherein the reference model includes a triangular mesh, approximating features of the conferee'"'"'s head, wherein vertexes of the triangular mesh include geometric and semantic information, which correspond to the features.

6. One or more computer-readable media comprising computer-executable instructions that, when executed, perform the method as recited in claim 1.

7. A system, comprising:

means for receiving initial video images representative of a conferee'"'"'s head;

means for evaluating the initial video images to form an initial reference model having initial features to monitor a head position of the conferee;

means for tracking the head position of the conferee using updates to the initial features as subsequent video images are received; and

means for activating an auto-recovery tracking process using the initial reference model that does not included the updates when tracking of the conferee'"'"'s head is temporarily interrupted.

8. The system as recited in claim 7, wherein the reference model is a three-dimensional personalized face model of the conferee.

9. The system as recited in claim 7, further comprising means for building the reference model during a learning stage and storing the reference image in a database.

10. The system as recited in claim 7, further comprising means for assigning markers to the reference model and means for searching the received video images of the conferee'"'"'s head for points that match the markers in the reference model.

11. The system as recited in claim 7, wherein the reference model includes a triangular mesh, approximating features of the conferee'"'"'s head, wherein vertexes of the triangular mesh include geometric and semantic information, which correspond to the features.

12. One or more computer-readable media having stored thereon computer executable instructions that, when executed by one or more processors, causes the one or more processors of a computer system to:

store a personalized face model of a conferee;

receive initial video images representative of the conferee'"'"'s face;

evaluate the initial video images to form an initial reference model having initial features to monitor a head position of the conferee;

track the head position of the conferee using updates to the initial features as subsequent video images are received; and

activate an auto-recovery tracking process using the initial reference model that does not included the updates, if the tracking of the conferee'"'"'s head is temporarily interrupted.

13. One or more computer-readable media as recited in claim 12, wherein the reference model is a three-dimensional personalized face model of the conferee.

14. One or more computer-readable media as recited in claim 12, further comprising computer executable instructions that, when executed, direct the computer system to build the personalized face model during a learning stage and store the personalized face model in a database.

15. One or more computer-readable media as recited in claim 12, further comprising computer executable instructions that, when executed, direct the computer system to assign markers to the personalized face model and search the received video images of the conferee'"'"'s face for points that match the markers in the personalized face model.

16. One or more computer-readable media as recited in claim 12, wherein the reference model includes a triangular mesh, approximating features of the conferee'"'"'s face, wherein vertexes of the triangular mesh include geometric and semantic information, which correspond to the features.