Systems And Methods For Processing An Image For Target Tracking

• US 20110102438A1
• Filed: 11/05/2009
• Published: 05/05/2011
• Est. Priority Date: 11/05/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Peeper, Craig, Marais, Charles Claudius, Mathe, Zsolt, Bertolami, Joe, Geiss, Ryan Michael
• Granted Patent: US 8,988,432 B2
• Time in Patent Office: Days
• US Class Current: 345/426
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. The image may then be processed. For example, the image may be downsampled, a shadow, noise, and/or a missing potion in the image may be determined, pixels in the image that may be outside a range defined by a capture device associated with the image may be determined, a portion of the image associated with a floor may be detected. Additionally, a target in the image may be determined and scanned. A refined image may then be rendered based on the processed image. The refined image may then be processed to, for example, track a user.

Claims

1. A method for processing an image of a scene, the method comprising:

receiving the image;

processing the image to determine one or more characteristics of the scene; and

rendering a refined image based the one or more characteristics of the scene.

2. The method of claim 1, wherein processing the image to determine the one or more characteristics of the scene comprises downsampling the image.

3. The method of claim 1, wherein processing the image to determine the one or more characteristics of the scene comprises determining whether the image includes a shadow;

and estimating one or more values for pixels included in the image associated with the shadow.

4. The method of claim 1, wherein processing the image to determine the one or more characteristics of the scene comprises determining whether the image includes noise;

and removing the noise in the image.

5. The method of claim 1, wherein processing the image to determine the one or more characteristics of the scene comprises detecting whether one or more pixels in the image may be outside a range defined by a capture device, wherein the capture device captures the image.

6. The method of claim 1, wherein processing the image to determine the one or more characteristics of the scene comprises determining whether the image includes a missing portion;

and estimating one or more values for invalid pixels associated with the missing portion.

7. The method of claim 1, wherein processing the image to determine the one or more characteristics of the scene comprises detecting a portion of the image associated with a floor in the scene.

8. The method of claim 1, wherein processing the image to determine the one or more characteristics of the scene comprises:

determining whether the image includes a target; and

scanning the target if, based on the determination, the image includes the target.

9. The method of claim 8, wherein processing the image to determine the one or more characteristics of the target comprises segmenting the target from the image.

10. A computer-readable storage medium having stored thereon computer executable instructions for processing an image, the computer executable instructions comprising instructions for:

receiving an image, wherein the image comprises a plurality of pixels;

processing the image, wherein processing the image comprises at least one of the following downsampling the image, determining whether the image includes one of the following a shadow, noise, and a missing portion, determining whether the image includes a target, and scanning the target if, based on the determination, the image includes the target; and

rendering a refined image based on the processed depth image.

11. The computer-readable storage medium of claim 10, wherein the instructions for processing the image further comprises instructions for detecting whether one or more pixels in the image are outside a range defined by a capture device associated with the image.

12. The computer-readable storage medium of claim 10, wherein the instructions for determining whether one or more pixels are outside the range defined by the capture device associated with the image comprises instructions for providing an image array, wherein the image array has a number of pixels equal to a total number of pixels in the image;

incrementing a count associated with the pixels in the image array if a value associated with the corresponding pixels in the image is invalid;

analyzing the count associated with the pixels in the image array; and

marking the pixels in the image with an infinity value if, based on the analysis, the count associated with the corresponding pixels in the image array is greater than a range threshold.

13. The computer-readable storage medium of claim 10, wherein the instructions for processing the image further comprises instructions for detecting a portion of the image associated with a floor in the scene.

14. The computer-readable storage medium of claim 10, wherein the instructions for detecting the portion of the image associated with the floor in the scene further comprises instructions for identifying a lower region of the image;

and analyzing the lower region to fit a floor plane associated with the floor thereto.

15. The computer-readable storage medium of claim 10, wherein the instructions for processing the image further comprises instructions for segmenting the target from the image.

16. The computer-readable storage medium of claim 15, wherein the instructions for segmenting the target from the image comprises instructions for creating a binary mask of the target;

and discarding an environment included in the image.

17. The computer-readable storage medium of claim 10, wherein the image comprises a plurality of pixels, and wherein the instructions for downsampling the depth image comprises instructions for reducing a number of the plurality of pixels and filtering one or more invalid pixels in the plurality of pixels.

18. The computer-readable storage medium of claim 10, wherein the instructions for determining whether the image includes one of the following a shadow, noise, and a missing portion further comprises instructions for estimating one or more values for pixels in the image associated with one of the following the shadow, the noise, and the missing portion.

19. The computer-readable storage medium of claim 10, wherein the instructions for scanning the target comprises instructions for determining one or more measurements of the target.

20. A system for processing an image of a scene, the system comprising:

a capture device, wherein the capture device comprises a camera component that receives a depth image of the scene, wherein the depth image comprises pixels; and

a processor in operative communication with the capture device, wherein the processor downsamples the image;

determines whether the image includes one of the following a shadow, noise, and a missing portion;

detects whether one or more of the pixels in the image are outside a range defined by the capture device;

detects a portion of the image associated with a planar surface in the scene;

determines whether the image includes a target;

scans the target, if based on the determination, the depth image includes the target; and

segments the target from the image.