In-home depth camera calibration

• US 8,866,889 B2
• Filed: 11/03/2010
• Issued: 10/21/2014
• Est. Priority Date: 11/03/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Masalkar, Prafulla J., Stachniak, Szymon P., Leyvand, Tommer, Zhang, Zhengyou, Del Castillo, Leonardo, Mathe, Zsolt
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): Sun, Yulin
• Application Number:
  US12/939,038
• Publication Number:
  US 20120105585A1
• Time in Patent Office: 1,448 Days
• Field of Search: 348/46, 382/154
• US Class Current: 348/46
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/327 Calibration thereof

Abstract

A system and method are disclosed for calibrating a depth camera in a natural user interface. The system in general obtains an objective measurement of true distance between a capture device and one or more objects in a scene. The system then compares the true depth measurement to the depth measurement provided by the depth camera at one or more points and determines an error function describing an error in the depth camera measurement. The depth camera may then be recalibrated to correct for the error. The objective measurement of distance to one or more objects in a scene may be accomplished by a variety of systems and methods.

Claims

1. A method of recalibrating a depth image sensor, the depth sensor capturing image data from a 3D space, the method comprising:

a) measuring a depth value of a person in the 3-D space using the depth sensor;

b) determining a number of pixels in the image of the person along an axis;

c) calculating a length of the person along the axis based on the depth value measured in said step (a) and the number of pixels determined in said step (b);

d) receiving input from the person regarding a true length of the person along the axis;

e) determining a true depth value of the person independently of the measurement of said step (a) by comparing the length calculated in said step (c) with the true length received in said step (d);

f) modeling a depth error resulting from a deviation in the true depth determined in said step e and the depth measured in said step (a); and

g) correcting the depth measurements made by the depth sensor using the depth error modeled in said step (f).

2. The method of claim 1, said step (b) comprising the step of measuring a height, head to feet, of the person.

3. The method of claim 1, said step (b) comprising the step of measuring an arm span of the person.

4. The method of claim 1, wherein the size of the person is unknown to a computing device performing said step (b).

5. The method of claim 4, said step (c) comprising the steps of defining a first line between a set of points along the axis measured at a first location in the 3-D space, defining a second line between the same set of points measured at a second location in the 3-D space, defining an equation where the first and second lines are proportional to each other by an error coefficient, and solving for the error coefficient.

6. The method of claim 1, wherein a depth error which is linear at different points in the 3-d space is modeled using two sample points along the axis.

7. The method of claim 1, wherein a depth error which is non-linear at different points in the 3-d space is modeled using more than two sample points along the axis.

8. A method of recalibrating a depth sensor of a natural user interface, the depth sensor capturing image data from a 3-D space, the method comprising:

a) measuring a depth of an object within the 3-D space using the depth sensor;

b) receiving a user indication of an x-axis or y-axis dimensional size of an object;

c) correlating the user-indicated size of the object to a true depth of the object from the depth sensor based on a known correlation between the size of the object and number of pixels depicting that object at distances between the object and the depth sensor;

d) modeling a depth error resulting from a deviation in the true depth determined in said step (b) and the depth measured in said step (a); and

e) compensating depth measurements made by the depth sensor using the depth error modeled in said step (d).

9. The method of claim 8, where, for linear depth errors, said step (d) comprises the step of modeling the error in the form of measured depth of said step (a) =α

(true depth of said step (c)) 31 β

, where the β

component represents the depth at which the sensor and the true depth are equal to each other, and α

represents the slope of a line defined between the true depth and depth measured by the depth image sensor.