Gesture detection from digital video images

• US 7,224,830 B2
• Filed: 02/04/2003
• Issued: 05/29/2007
• Est. Priority Date: 02/04/2003
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Nefian, Ara V., Cavin, Robert D.
• Primary Examiner(s): Sherali; Ishrat
• Assistant Examiner(s): CARTER, AARON W
• Application Number:
  US10/358,761
• Publication Number:
  US 20040151366A1
• Time in Patent Office: 1,575 Days
• Field of Search: 382/154, 382/285, 715/863, 348 42- 60
• US Class Current: 382/154
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Human gestures are detected and/or tracked from a pair of digital video images. The pair of images may be used to provide a set of observation vectors that provide a three dimensional position of a subject'"'"'s upper body. The likelihood of each observation vector representing an upper body component may be determined. Initialization of the model for detecting and tracking gestures may include a set of assumptions regarding the initial position of the subject in a set of foreground observation vectors.

Claims

1. A method comprising:

detecting a set of foreground pixels from a plurality of observation vectors obtained from a pair of digital video images, the observation vectors including positional information with respect to a distance from an image capture device and color information, the observation vectors created from a disparity map based on measured differences between the pair of video images, at least some of the foreground pixels representing an upper body;

selecting an anthropological measure to limit the pixels representing at least one component of the upper body;

determining the coordinates for at least one joint of the upper body; and

calculating the observation likelihood of the upper body component from the observation vectors.

2. The method of claim 1 further comprising calculating the observation likelihood for a first, second and third upper body component using a Bayesian network having nodes representing anthropological measures for the first, second and third upper body components, the coordinates for the at least one joint, and parameters of the first, second and third upper body components.

3. The method of claim 2 further comprising estimating the position of the first upper body component from at least some of the foreground pixels, and calculating the observation likelihood of the second and third upper body components based at least in part on the estimated position of the first upper body component, the first upper body component comprising a neck.

4. The method of claim 1 wherein the upper body component is a torso.

5. The method of claim 1 wherein the upper body component is a head.

6. The method of claim 1 farther comprising storing the anthropological measures in memory.

7. The method of claim 1 further comprising identifying an upper body gesture.

16. The method of claim 3, wherein the second and third upper body components comprise a head and a torso.

17. The method of claim 16, further comprising calculating the observation likelihood for an arm and band after the observation likelihood for at least two of the first, second, and third upper body components is less than a convergence threshold.

8. A system comprising:

a stereo digital video input port to generate a first set and a second set of pixels depicting an upper body; and

a processor coupled to the stereo digital video input port to create a disparity map from the first set and second set of pixels, and to create a plurality of observation vectors each including positional information with respect to a distance from a stereo camera and color information, the observation vectors created from the disparity map, and to determine the probability that the sets of pixels represent a selected upper body component based on the observation vectors.

9. The system of claim 8 further comprising the stereo camera coupled to the stereo digital video input port.

10. The system of claim 8 further comprising system memory to store a set of anthropological measures.

11. The system of claim 8 wherein the processor is to calculate the probability that the sets of pixels represent a plurality of upper body components using a Bayesian network having nodes to represent anthropological measures for first, second and third upper body components, a position of at least one joint of the upper body, and parameters of the first, second and third upper body components.

12. The system of claim 11 wherein the plurality of upper body components include at least a head and a torso, and a neck, and wherein the probability of the head and the torso is based on the probability of the neck.

13. A processor-based system to:

detect a set of foreground pixels from a plurality of observation vectors obtained from a pair of video images of an upper body, the observation vectors including positional information with respect to a distance from an image capture device and color information, the observation vectors created from a disparity map based on a measured difference between the pair of video images;

select an anthropological measure stored in system memory limiting the parameters of an upper body component;

determine the coordinates for at least one upper body joint from the foreground pixels, the coordinates limited by the anthropological measures; and

calculate the observation likelihood for the upper body component from the observation vectors.

14. The processor-based system of claim 13 wherein determining the coordinates for at least one upper body joint comprises selecting a plane in the foreground pixels, the normal to the plane facing the upper body.

15. The processor-based system of claim 13 wherein calculating the observation likelihood comprises calculating the observation likelihood for a plurality of upper body components including at least a head, a torso, and a neck, and wherein the probability of the head and the torso is based on the probability of the neck.