OBJECT ORIENTATION ESTIMATION

• US 20150348269A1
• Filed: 05/27/2014
• Published: 12/03/2015
• Est. Priority Date: 05/27/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: DEDHIA, Bhaven P., ESHET, Yaron, HULTEN, Geoffrey J.
• Granted Patent: US 9,727,776 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

The description relates to estimating object orientation. One example includes determining a first estimate of object orientation using a first technique and image data. In this example, a second estimate of the object orientation can be determined using a second technique and the image data. The first estimate can be corrected with the second estimate to generate a corrected object orientation estimate which can be output.

Claims

1. A method performed by a computing device, the method comprising:

determining a first estimate of object orientation using a first technique and image data;

determining a second estimate of the object orientation using a second technique and the image data;

correcting the first estimate using the second estimate to generate a corrected object orientation estimate; and

outputting the corrected object orientation estimate.

2. The method of claim 1, wherein the first technique includes determining a geometric model associated with the object.

3. The method of claim 1, wherein the first technique comprises determining landmarks and an up vector of the object.

4. The method of claim 1, wherein the first estimate comprises estimates of angles of rotation relative to multiple axes of rotation of the object.

5. The method of claim 4, wherein the multiple axes of rotation of the object correspond to pitch, roll, and yaw.

6. The method of claim 4, wherein the second estimate of the object orientation includes another estimate of one of the angles of rotation.

7. The method of claim 6, wherein the multiple axes of rotation of the object correspond to pitch, roll, and yaw and the second estimate corresponds to yaw.

8. The method of claim 1, wherein the image data include two-dimensional image data and depth data, and the determining the first estimate further comprises using the depth data.

9. The method of claim 8, wherein the depth data includes time-of-flight depth data.

10. The method of claim 8, further comprising assigning a weight to the second estimate according to a distance of the object from a camera that collects the depth data, wherein the weight is used to generate the corrected object orientation estimate.

11. The method of claim 1, wherein the correcting includes determining an extent to which the first estimate is consistent with the second estimate.

12. The method of claim 1, wherein the object is a body part of a user, a handheld object, a vehicle, or a computing device.

13. A computer-executable memory device or storage device storing computer-executable instructions that, when executed by one or more processing devices, cause the one or more processing devices to perform acts comprising:

determining landmarks of an object;

determining a geometric model associated with the object;

using the geometric model, determining a first rotation angle estimate of the object relative to an axis of rotation using a first technique;

using the landmarks, determining a second rotation angle estimate of the object relative to the axis of rotation using a second technique;

correcting the first rotation angle estimate using the second rotation angle estimate to generate a corrected rotation angle estimate;

determining other estimates of other rotation angles of the object relative to other axes of rotation of the object; and

determining an object orientation from the corrected rotation angle estimate and the other estimates of the other rotation angles.

14. The computer-executable memory device or storage device of claim 13, wherein the determining the geometric model further comprises determining a valid mask corresponding to the object.

15. The computer-executable memory device or storage device of claim 14, wherein the determining the valid mask comprises filtering occlusions.

16. The computer-executable memory device or storage device of claim 15, wherein the occlusions are filtered by constructing a histogram of depth values relative to the object.

17. The computer-executable memory device or storage device of claim 13, wherein the geometric model is a plane and at least one of the other rotation angle estimates of the object is determined from the plane and a normal vector to the plane.

18. The computer-executable memory device or storage device of claim 13, wherein the first rotation angle estimate of the object corresponds to yaw and the other rotation angle estimates correspond to pitch and roll.

19. A system, comprising:

a processing device; and

a storage device storing computer-executable instructions which, when executed by the processing device, cause the processing device to;

determine a facial plane and facial landmarks of a head of a user from an image of the user;

estimate one or more head rotation angles of the user using a first technique and the facial plane; and

correct at least one of the head rotation angles of the user using a second technique and the facial landmarks.

20. The system of claim 19, wherein the second technique is an iterative binary technique.

21. The system of claim 20, wherein the computer-executable instructions further cause the processing device to adjust a number of iterations of the iterative binary technique according to available computational resources of the system.