ORIENTING THE POSITION OF A SENSOR

• US 20120146902A1
• Filed: 12/08/2010
• Published: 06/14/2012
• Est. Priority Date: 12/08/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Adermann, Stanley W., Peeper, Craig, Stachniak, Szymon, Kline, David C., Plagge, Mark
• Granted Patent: US 8,553,934 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

Techniques are provided for re-orienting a field of view of a depth camera having one or more sensors. The depth camera may have one or more sensors for generating a depth image and may also have an RGB camera. In some embodiments, the field of view is re-oriented based on the depth image. The position of the sensor(s) may be altered to change the field of view automatically based on an analysis of objects in the depth image. The re-orientation process may be repeated until a desired orientation of the sensor is determined. Input from the RGB camera might be used to validate a final orientation of the depth camera, but is not required to during the process of determining new possible orientation of the field of view.

Claims

1. A method of orienting a sensor, comprising:

generating a depth image from a sensor having a field of view;

determining one or more potential targets based on the depth image;

selecting one of the potential targets as a candidate object to track with the sensor;

determining whether to re-orient the sensor based on the present orientation of the sensor and a position of the candidate object in the field of view;

re-orienting the sensor if it is determined to do so; and

repeating the generating depth information, the determining one or more potential targets, the selecting, the determining whether to re-orient the sensor, and the re-orienting the sensor until it is determined not to re-orient the sensor.

2. The method of claim 1, wherein the depth image is divided into a plurality of regions having depth values, and wherein the determining one or more potential targets based on the depth image includes:

grouping regions of the depth image that have a depth value that is within a pre-determined amount of the depth value of an adjacent region.

3. The method of claim 1, wherein the re-orienting the sensor includes moving the sensor to include more of the candidate object within the field of view of the sensor.

4. The method of claim 1, wherein the re-orienting the sensor includes moving the sensor to position the candidate object in the field of view of the sensor in accordance with directions from an application that a user is able to interact with through input from the sensor.

5. The method of claim 1, further comprising:

sending a list of the one or more potential targets to a software application that reacts to user actions that are identified based on input from the sensor, wherein the selecting one of the potential targets as a candidate object to track with the sensor is based on input from the application.

6. The method of claim 1, further comprising:

instructing a user to move to a different location; and

repeating the generating depth information, the determining one or more potential targets, the selecting, and the determining whether to re-orient the sensor for at least one additional sensor orientation after instructing the user to move.

7. The method of claim 1, wherein the selecting one of the potential targets as a candidate object is based on one or more of height, width or depth of the potential targets.

8. The method of claim 1, further comprising determining a present physical position of the sensor based on a depth image that was acquired with the sensor in a previous orientation and accelerometer data associated with the present sensor position.

9. An apparatus, comprising:

a depth camera having one or more sensors, at least one of the one or more sensors is able to collect depth information and having a field of view; and

logic coupled to the depth camera, the logic generates a depth image from the depth information;

the logic determines one or more potential targets based on the depth image;

the logic selects one of the potential targets as a candidate object to track with the sensor;

the logic determines whether to re-orient the one or more sensors based on the present orientation of the one or more sensors and a position of the candidate object in the field of view;

the logic re-orients the one or more sensors if it is determined to do so; and

the logic repeats the generating depth information, the determining one or more potential targets, the selecting, the determining whether to re-orient the one or more sensors, and the re-orienting the one or more sensors until it is either determined that a current candidate object is properly within the field of view or that there are no potential targets.

10. The apparatus of claim 9, wherein the depth image is divided into a plurality of regions having depth values, and wherein the logic groups regions of the depth image that have a depth value that is within a pre-determined amount of the depth value of an adjacent region as a part of the determining one or more potential targets based on the depth image.

11. The apparatus of claim 9, wherein the logic moves the one or more sensors to include more of the candidate object within a field of view of the one or more sensors in order to re-orient the one or more sensors.

12. The apparatus of claim 9, wherein the logic receives directions from an application that a user is able to interact with through depth camera input, the logic uses the directions to determine how to re-orient the one or more sensors.

13. The apparatus of claim 9, wherein the logic selects the candidate object to track based on input from an application with which a user is able to interact with based on depth information collected by the depth camera.

14. The apparatus of claim 9, wherein the logic selects the candidate object to based on one or more of height, width or depth of potential targets.

15. The apparatus of claim 9, wherein the logic includes:

a processor; and

a computer readable medium coupled to the computer processor, the computer readable medium includes instructions which when executed on the processor cause the processor to implement the logic.

16. The apparatus of claim 9, wherein the one or more sensors include an RGB camera.

17. A method of orienting a depth camera, the method comprising:

a) generating a depth image from the depth camera having one or more sensors, the depth camera having a field of view;

b) determining zero or more potential targets based on the depth image;

c) determining whether to re-orient the field of view of the depth camera if zero potential targets were determined;

d) re-orienting the field of view of the depth camera if it is determined to re-orient when zero potential targets were determined;

e) selecting one of the potential targets as a candidate object to track with the one or more sensors;

f) determining whether to re-orient the field of view of the depth camera based on the present orientation of the sensor and the candidate object;

g) re-orienting the field of view of the depth camera if it is determined to do so; and

repeating said a) through g) until it is determined that the field of view of the depth camera should not be re-oriented.

18. The method of claim 17, further comprising positioning the field of view of the depth camera in a way to capture a possible user if it is determined to not re-orient when zero potential targets were determined.

19. The method of claim 17, further comprising determining voxels from the depth image, the potential targets are determined based on identifying groups of voxels that are near each other in 3D-space.

20. The method of claim 17, further comprising:

determining a location for a user based on the depth image;

instructing the user to move to the location; and

performing said a) through g) after instructing the user to move to the location.