Method of video coding the movement of a human face from a sequence of images

• US 7,046,728 B1
• Filed: 06/30/2000
• Issued: 05/16/2006
• Est. Priority Date: 06/30/2000
• Status: Expired due to Fees

• Current Assignee: Indian Institute of Technology Delhi, Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Acharya, Tinku, Sengupta, Somnath, Suryanarayana, A. Rama
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US09/608,989
• Time in Patent Office: 2,146 Days
• Field of Search: 348 17- 20, 375/240.08, 375/240.09
• US Class Current: 375/240.08
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame

Abstract

Briefly, in accordance with one embodiment, a technique for coding the movement of a head or face from a sequence of images is disclosed. A variety of potential alternative embodiments are discussed.

Claims

1. A method of video coding the movement of a face from a sequence of images comprising:

selecting a limited number of feature points from an image of the face where minimal or no local motion is observed to be video coded;

estimating spatio-temporal rates of change in intensity at the selected feature points using at least two images from the sequence;

estimating the translation and rotation of the face using the feature points and using the estimated spatio-temporal rates; and

coding the estimated translation and rotation.

2. The method of claim 1, wherein prior to selecting the feature points, for a particular image, the face is approximated using triangular planar patches.

3. The method of claim 2, wherein a feature point is selected from each planar triangular patch.

4. The method of claim 1, wherein the translation and rotation of the face are estimated from the feature points and from the estimated spatio-temporal rates using a least mean square estimation method.

5. A method of video coding the movement of a head from a sequence of images comprising:

coding the head from at least one of the images based, at least in part, on a limited number of selected feature points from an image of the head where minimal or no local motion is observed, employing a three-dimensional based coding technique to produce a three-dimensional (3D) model; and

estimating the movement of the head in other images of the sequence using the 3D model of the head, wherein the movement of the head is estimated as translations and rotations based at least in part on estimates of spatio-temporal rates of change in intensity at the selected feature points.

6. The method of claim 5, wherein the movement of the head is estimated by treating the 3D model of the head as a rigid body in the sequence of images.

7. A system comprising:

an imager; and

a computing platform;

said imager and computing platform being coupled to communicate electronically;

wherein said computing platform being adapted so that, in operation, the movement of a face from a sequence of images is coded by;

selecting a limited number of feature points from an image of the face where minimal or no local motion is observed to be video coded;

estimating spatio-temporal rates of change in intensity at the selected feature points using at least two images from the sequence;

estimating the translation and rotation of the face using the feature points and using the estimated spatio-temporal rates; and

coding the estimated translation and rotation.

8. The system of claim 7, wherein said computing platform is adapted so that, in operation, the movement of a face from a sequence of images is further coded by, prior to selecting the feature points, for a particular image, approximating the face using triangular planar patches.

9. The system of claim 8, wherein said computing platform is adapted so that, in operation, the movement of a face from a sequence of images is further coded by selecting a feature point from each planar triangular patch.

10. The system of claim 7, wherein said computing platform is adapted so that, in operation, the movement of a face from a sequence of images is further coded by estimating the translation and rotation of the face from the feature points and from the estimated spatio-temporal rates using a least mean square estimation method.

11. A system comprising:

an imager; and

a computing platform;

said imager and computing platform being coupled to communicate electronically;

wherein said computing platform being adapted so that, in operation, the movement of a head from a sequence of images is represented by;

coding the head from at least one of the images based, at least in part, on a limited number of selected feature points from an image of the head where minimal or no local motion is observed, employing a three-dimensional based coding technique to produce a three-dimensional (3D) model; and

estimating the movement of the head in other images of the sequence using the 3D model of the head, wherein said computing platform is adapted so that, in operation, the movement of the head is estimated as translations and rotations based at least in part on estimates of spatio-temporal rates of change in intensity at the selected feature points.

12. The system of claim 11, wherein said computing platform is adapted so that, in operation, the movement of the head in other images of the sequence is estimated by treating the 3D model of the head as a rigid body in the sequence of images.

13. An article comprising:

a storage medium, said storage medium having stored thereon instructions, said instructions, when executed by a computing platform, resulting in the movement of a face from a sequence of images being coded by;

selecting a limited number of feature points from an image of the face where minimal or no local motion is observed to be video coded;

estimating spatio-temporal rates of change in intensity at the selected feature points using at least two images from the sequence;

estimating the translation and rotation of the face using the feature points and using the estimated spatio-temporal rates; and

coding the estimated translation and rotation.

14. The article of claim 13, wherein said instructions, when executed, further result in, prior to selecting the feature points, for a particular image, approximating the face using triangular planar patches.

15. The article of claim 14, wherein said instructions, when executed, further result in selecting a feature point from each planar triangular patch.

16. The article of claim 13, wherein said instructions, when executed, further result in, estimating the translation and rotation of the face from the feature points and from the estimated spatio-temporal rates using a least mean square estimation method.

17. An article comprising:

a storage medium, said storage medium having stored thereon instructions, said instructions, when executed by a computing platform, result in the movement of a head from a sequence of images being represented by;

coding the head from at least one of the images based, at least in part, on a limited number of selected feature points from an image of the head where minimal or no local motion is observed, employing a three-dimensional based coding technique to produce a three-dimensional (3D) model; and

estimating the movement of the head in other images of the sequence using the 3D model of the head, wherein said instructions, when executed, result in the movement of the head being estimated as translations and rotations based at least in part on estimates of spatio-temporal rates of change in intensity at the selected feature points.

18. The article of claim 17, wherein said instructions, when executed, further result in the

the movement of the head in other images of the sequence being estimated by treating the 3D model of the head as a rigid body in the sequence of images.