Depth camera compatibility

• US 8,619,122 B2
• Filed: 02/02/2010
• Issued: 12/31/2013
• Est. Priority Date: 02/02/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gilboa, Guy, Adler, Avishai, Katz, Sagi
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): Aghevli, Reza
• Application Number:
  US12/698,944
• Publication Number:
  US 20110187820A1
• Time in Patent Office: 1,428 Days
• Field of Search: None
• US Class Current: 348/43
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size

Abstract

Compatibility between a depth image consumer and a depth image producer is provided by receiving a native depth image having an unsupported type that is not supported by a depth image consumer, and processing the native depth image into an emulation depth image having a supported type that is supported by the depth image consumer. This emulation depth image is then output to the depth image consumer.

Claims

1. A method of providing depth camera compatibility, the method comprising:

receiving a native depth image having an unsupported type that is not supported by a depth image consumer;

processing the native depth image into an emulation depth image of reduced or modified content relative to the native depth image, the emulation depth image usable by the depth image consumer and having a supported type that is supported by the depth image consumer; and

outputting the emulation depth image having the supported type.

2. The method of claim 1, where receiving the native depth image includes receiving the native depth image from a time of flight depth camera.

3. The method of claim 2, where processing the native depth image includes processing the native depth image from the time of flight depth camera into the emulation depth image, where the emulation depth image emulates a depth image produced by a structured light depth camera.

4. The method of claim 3, where processing the native depth image includes applying an edge preserving filter to the native depth image.

5. The method of claim 3, where processing the native depth image further includes building a confidence map by passing a median filter on an illumination image.

6. The method of claim 5, where building the confidence map includes applying the median filter and a soft threshold function to the illumination image.

7. The method of claim 5, where processing the native depth image further includes building an oblique surface map from the native depth image.

8. The method of claim 7, where processing the native depth image further includes unifying the confidence map and the oblique surface map into an invalidation testing map.

9. The method of claim 8, further comprising invalidating pixels of the invalidation testing map that are outside a predetermined range.

10. The method of claim 3, where processing the native depth image further includes enhancing edges between adjacent pixel regions with different depth values in the native depth image.

11. The method of claim 3, where processing the native depth image further includes quantizing depth values.

12. The method of claim 3, where processing the native depth image further includes assigning deeper depth values to pixels belonging to objects below a threshold size.

13. The method of claim 3, where processing the native depth image further includes assigning shadow pixel values to pixels that are virtually occluded from a virtual vantage point of a virtual sensor virtually spaced away from the time of flight depth camera.

14. The method of claim 1, further comprising receiving from the depth image consumer supported virtual depth camera parameters for the depth image consumer.

15. The method of claim 14, where receiving supported virtual depth camera parameters includes receiving an indication of a model depth camera that is compatible with the depth image consumer.

16. The method of claim 15, where processing the native depth image into the emulation depth image includes modifying the native depth image to emulate a depth image produced by the model depth camera.

17. A machine-readable storage device holding instructions executable by a logic device to:

receive a native depth image having an unsupported type that is not supported by a depth image consumer;

process the native depth image into an emulation depth image of reduced or modified content relative to the native depth image, the emulation depth image usable by the depth image consumer and having a supported type that is supported by the depth image consumer; and

output the emulation depth image having the supported type.

18. The machine-readable storage device of claim 17, further holding instructions executable by the logic device to:

receive from the depth image consumer an indication of a model depth camera that is compatible with the depth image consumer.

19. The machine-readable storage device of claim 18, further holding instructions executable by the logic device to:

modify the native depth image to emulate a depth image produced by the model depth camera.

20. A depth camera interchangeability system for converting a native depth image from a time of flight depth camera to an emulation depth image emulating a depth image produced by a structured light depth camera, comprising:

a capture module to receive the native depth image from the time of flight camera;

a noise suppression module configured to preserve depth edges between adjacent pixel regions with different depth values in the native depth image;

an edge enhancement module configured to enhance depth edges between adjacent pixel regions with different depth values in the native depth image;

an invalidation module configured to invalidate pixels having a combined illumination and obliqueness outside a predetermined range;

a depth quantization module configured to quantize depth values;

a small object correction model configured to assign deeper depth values to pixels belonging to objects below a threshold size; and

a shadow simulation module configured to assign shadow pixel values to pixels that are virtually occluded from a virtual vantage point of a virtual sensor virtually spaced away from the time of flight depth camera.