Head pose estimation using RGBD camera
First Claim
1. A method comprising:
- capturing a series of images with depth data of a head of a subject;
obtaining a reference pose for the head of the subject from one image in the series of images and using the reference pose to define a reference coordinate frame for the series of images;
determining a rotation matrix and a translation vector associated with a pose of the head in each image in the series of images relative to the reference coordinate frame using the depth data;
extracting a face on the head from a background in the series of images using the depth data, wherein extracting the face from the background comprises;
calculating a depth of the face in each image using the depth data; and
segmenting out the face from the background using a threshold and the depth of the face; and
tracking the face in the series of images after extracting the face from the background.
1 Assignment
0 Petitions
Accused Products
Abstract
A three-dimensional pose of the head of a subject is determined based on depth data captured in multiple images. The multiple images of the head are captured, e.g., by an RGBD camera. A rotation matrix and translation vector of the pose of the head relative to a reference pose is determined using the depth data. For example, arbitrary feature points on the head may be extracted in each of the multiple images and provided along with corresponding depth data to an Extended Kalman filter with states including a rotation matrix and a translation vector associated with the reference pose for the head and a current orientation and a current position. The three-dimensional pose of the head with respect to the reference pose is then determined based on the rotation matrix and the translation vector.
-
Citations
16 Claims
-
1. A method comprising:
-
capturing a series of images with depth data of a head of a subject; obtaining a reference pose for the head of the subject from one image in the series of images and using the reference pose to define a reference coordinate frame for the series of images; determining a rotation matrix and a translation vector associated with a pose of the head in each image in the series of images relative to the reference coordinate frame using the depth data; extracting a face on the head from a background in the series of images using the depth data, wherein extracting the face from the background comprises; calculating a depth of the face in each image using the depth data; and segmenting out the face from the background using a threshold and the depth of the face; and tracking the face in the series of images after extracting the face from the background. - View Dependent Claims (2, 3, 4)
-
-
5. An apparatus comprising:
-
a Red, Green, Blue, Distance (RGBD) camera to capture images with depth data of a head of a subject; and a processor coupled to the RGBD camera to receive a series of images with the depth data of the head of the subject, the processor being configured to obtain a reference pose for the head of the subject from one image in the series of images and use the reference pose to define a reference coordinate frame for the series of images;
determine a rotation matrix and a translation vector associated with a pose of the head in each image in the series of images relative to the reference coordinate frame using the depth data, wherein the processor is further configured to extract a face on the head from a background in the series of images using the depth data; and
track the face in the series of images after extracting the face from the background, wherein the processor is configured to extract the face from the background by being configured to calculate a depth of the face in each image using the depth data; and
segment out the face from the background with a threshold and the depth of the face. - View Dependent Claims (6, 7, 8)
-
-
9. An apparatus comprising:
-
means for capturing a series of images with depth data of a head of a subject; means for obtaining a reference pose for the head of the subject from one image in the series of images and using the reference pose to define a reference coordinate frame for the series of images; means for determining a rotation matrix and a translation vector associated with a pose of the head in each image in the series of images relative to the reference coordinate frame using the depth data; means for extracting a face on the head from a background in the series of images using the depth data, wherein the means for extracting the face from the background comprises; means for calculating a depth of the face in each image using the depth data; and means for segmenting out the face from the background using a threshold and the depth of the face; and means for tracking the face in the series of images after extracting the face from the background. - View Dependent Claims (10, 11, 12)
-
-
13. A non-transitory computer-readable medium including program code executable by one or more processors stored thereon, comprising:
-
program code to receive a series of images with depth data of a head of a subject; program code to obtain a reference pose for the head of the subject from one image in the series of images and use the reference pose to define a reference coordinate frame for the series of images; program code to determine a rotation matrix and a translation vector associated with a pose of the head in each image in the series of images relative to the reference coordinate frame using the depth data; program code to extract a face on the head from a background in the series of images using the depth data, wherein the program code to extract the face from the background comprises; program code to calculate a depth of the face in each image using the depth data; and program code to segment out the face from the background using a threshold and the depth of the face; and program code to track the face in the series of images after extracting the face from the background. - View Dependent Claims (14, 15, 16)
-
Specification