System for fast, probabilistic skeletal tracking
First Claim
1. In a system including a computing environment coupled to a capture device for capturing image data from a field of view of the capture device, the image data representing a position of a user, a method of estimating user body position comprising:
- (a) receiving image data from the field of view;
(b) applying one or more computer models for generating body part proposals from the image data, at least one computer model accounting for missing joint information by using a position of the missing joint identified in a past frame as a basis for the position of the missing joint in the current frame; and
(c) analyzing the one or more computer models produced in said step (b) by one or more methodologies to choose at least one computer model of the one or more computer models estimated to provide the best body part proposal.
2 Assignments
0 Petitions
Accused Products
Abstract
A system and method are disclosed for recognizing and tracking a user'"'"'s skeletal joints with a NUI system. The system includes one or more experts for proposing one or more skeletal hypotheses each representing a user pose within a given frame. Each expert is generally computationally inexpensive. The system further includes an arbiter for resolving the skeletal hypotheses from the experts into a best state estimate for a given frame. The arbiter may score the various skeletal hypotheses based on different methodologies. The one or more skeletal hypotheses resulting in the highest score may be returned as the state estimate for a given frame. It may happen that the experts and arbiter are unable to resolve a single state estimate with a high degree of confidence for a given frame. It is a further goal of the present system to capture any such uncertainty as a factor in how a state estimate is to be used.
-
Citations
19 Claims
-
1. In a system including a computing environment coupled to a capture device for capturing image data from a field of view of the capture device, the image data representing a position of a user, a method of estimating user body position comprising:
-
(a) receiving image data from the field of view; (b) applying one or more computer models for generating body part proposals from the image data, at least one computer model accounting for missing joint information by using a position of the missing joint identified in a past frame as a basis for the position of the missing joint in the current frame; and (c) analyzing the one or more computer models produced in said step (b) by one or more methodologies to choose at least one computer model of the one or more computer models estimated to provide the best body part proposal. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A software pipeline for generating a state estimate for a given frame of captured image data, the state estimate representing an estimate of a position of a user within a field of view captured within the image data, comprising:
-
one or more experts for receiving information including one or more body part proposals and generating a plurality of computer models, each computer model representing an estimation of the position of the user in the given frame of captured image data at least one of the experts generating skeletal hypotheses from at least one of a tree structure of a human body including a torso and limbs as branches, a head triangle including a triangle formed by a head and shoulders and a torso volume including a torso; and an arbiter for receiving the plurality of computer models, scoring the computer models by one or more methodologies, and outputting at least one computer model estimated by the arbiter to best approximate the position of the user in the frame. - View Dependent Claims (11, 12, 13, 14, 15)
-
-
16. A computer-readable storage medium capable of programming a processor to perform a method tracking body parts of a user to determine a state estimate of the user'"'"'s position in a current frame of image data, comprising:
-
(a) generating a plurality of skeletal hypotheses, the plurality of skeletal hypotheses defining a probability distribution; (b) selecting one or more skeletal hypotheses as being the most probable state estimates based on the probability distribution, where the probability distribution indicates one or more skeletal hypotheses as the probable state estimate; and (c) indicating that no state estimate is determined for the frame of image data where the probability distribution does not indicate one or more skeletal hypotheses as being probable state estimates. - View Dependent Claims (17, 18, 19)
-
Specification