Identification of objects in a 3D video using non/over reflective clothing

• US 8,831,289 B2
• Filed: 01/17/2012
• Issued: 09/09/2014
• Est. Priority Date: 03/09/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft International Holdings BV (Microsoft Corporation)
• Inventors: Katz, Sagi, Yahav, Giora
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US13/351,680
• Publication Number:
  US 20120114180A1
• Time in Patent Office: 966 Days
• Field of Search: 382/106, 382/103, 382/209
• US Class Current: 382/106
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence

Abstract

A computing system generates a depth map from at least one image, detects objects in the depth map, and identifies anomalies in the objects from the depth map. Another computing system identifies at least one anomaly in an object in a depth map, and uses the anomaly to identify future occurrences of the object. A system includes a three dimensional (3D) imaging system to generate a depth map from at least one image, an object detector to detect objects within the depth map, and an anomaly detector to detect anomalies in the detected objects, wherein the anomalies are logical gaps and/or logical protrusions in the depth map.

Claims

1. A computing system for processing images, comprising:

a 3-D imaging system, the 3-D imaging system generates at least one depth map from at least one image of a first person in an application during a calibration process of the application;

an object detector, the object detector identifies a body part of the first person in the at least one depth map;

an anomaly detector, the anomaly detector, responsive to the object detector, identifies, from the at least one depth map, an anomaly in the body part of the first person, the anomaly in the body part of the first person comprising;

(a) a logical gap anomaly which represents a first object of the body part of the first person appearing to be further from the 3-D imaging system than the first object actually is due to a lower reflective property of the first object, where the first object is identified by pixel depths which are less than pixel depths of an adjacent region of the first object or (b) a logical protrusion anomaly which represents the first object appearing to be closer to the 3-D imaging system than the first object actually is due to a higher reflective property of the first object, where the first object is identified by pixel depths which are greater than pixel depths of an adjacent region of the first object; and

an anomaly marker, the anomaly marker, responsive to the anomaly detector, generates a marker which identifies the anomaly in the body part of the first person, the anomaly marker forwards the marker which identifies the anomaly in the body part of the first person to the object detector and the object detector, in response to the anomaly marker, subsequently uses the marker which identifies the anomaly in the body part of the first person as a template to identify a future occurrence of the body part of the first person in a series of depth maps after the calibration process.

2. The computing system of claim 1, wherein:

the anomaly marker marks the anomaly in the body part of the first person by an indication of a size and shape of the anomaly in the body part of the first person.

3. The computing system of claim 1, wherein:

the application comprises a multiplayer application; and

the calibration process occurs prior to a start of the multiplayer application.

4. The computing system of claim 1, wherein:

the first object comprises hair of the body part of the first person and the body part comprises a head of the first person.

5. The computing system of claim 1, wherein:

the first object comprises an object worn on the body part of the first person and the body part comprises an arm or wrist of the first person.

6. The computing system of claim 1, wherein:

the anomaly in the body part of the first person is used to distinguish between left and right paired objects, and the first object is one object of a pair of objects.

7. The computing system of claim 1, wherein:

the first object comprises at least one of;

clothing, a clothing accessory, a jewelry or a medical artifact worn on the body part.

8. The computing system of claim 1, wherein the anomaly detector, to identify the anomaly in the body part of the first person:

calculates a depth difference by a comparison of depths of at least two regions in the body part of the first person; and

determines that an anomaly exists where an absolute value of the depth difference exceeds a threshold.

9. The computing system of claim 1, further comprising:

an anomaly compensator, the anomaly compensator compensates for the anomaly in the body part of the first person by a replacement of pixels associated with the anomaly in the body part of first person with pixels of depth similar to that of a region adjacent to the anomaly in the body part of first person.

10. The computing system of claim 1, wherein:

the at least one image comprises a first image of a scene generated using continuous radiation;

a second image of the scene is generated using pulsed radiation; and

the anomaly detector, to identify the anomaly in the body part of the first person;

finds associated pixels of the first and second images which have intensities below a threshold;

calculates differences between the intensities of the associated pixels in the first and second images; and

determines that an anomaly exists in response to a determination that the calculated differences are less than a pixel intensity threshold.

11. The computing system of claim 10, wherein the anomaly detector, to identify the anomaly in the body part of the first person:

derives compensated depth values by a division of the intensities of the first image by the intensities of the second image;

determines that the compensated depth values exceed a depth threshold; and

determines that an anomaly exists responsive to the determination that the compensated depth values exceed the depth threshold.

12. The computing system of claim 1, wherein:

the at least one image comprises a second person in the application during the calibration process of the application;

the object detector identifies a body part of the second person in the at least one depth map;

the anomaly detector, responsive to the object detector, identifies, from the at least one depth map, an anomaly in the body part of the second person, the anomaly in the body part of the second person comprising;

(a) a logical gap anomaly which represents a second object of the body part of the second person appearing to be further from a 3-D imaging system than the second object actually is due to a lower reflective property of the second object, where the second object is identified by pixel depths which are less than pixel depths of an adjacent region of the second object or (b) a logical protrusion anomaly which represents the second object appearing to be closer to the 3-D imaging system than the second object actually is due to a higher reflective property of the second object, where the second object is identified by pixel depths which are greater than pixel depths of an adjacent region of the second object; and

the anomaly marker, responsive to the anomaly detector, provides a marker which identifies the anomaly in the body part of the second person, the object detector uses the marker which identifies the anomaly in the body part of the second person as a template to identify a future occurrence of the body part of the second person in the series of depth maps and to differentiate between the first and second persons.

13. A method for processing images, comprising:

performing a calibration process prior to a start of an application, the performing the calibration process comprises;

generating at least one depth map from at least one image of a first person;

detecting a body part of the first person in the at least one depth map;

identifying, from the at least one depth map, responsive to the detecting, an anomaly in the body part of the first person, the anomaly in the body part of the first person comprising;

(a) a logical gap anomaly which represents a first object of the body part of the first person appearing to be further from the 3-D imaging system than the first object actually is due to a lower reflective property of the first object, where the first object is identified by pixel depths which are less than pixel depths of an adjacent region of the first object or (b) a logical protrusion anomaly which represents the first object appearing to be closer to the 3-D imaging system than the first object actually is due to a higher reflective property of the first object, where the first object is identified by pixel depths which are greater than pixel depths of an adjacent region of the first object;

responsive to the identifying, generating a marker which identifies the anomaly in the body part of the first person; and

forwarding the marker which identifies the anomaly in the body part of the first person to an object detector; and

after the performing the calibration process, during the application and at the object detector;

using the marker which identifies the anomaly in the body part of the first person as template, identifying a future occurrence of the body part of the first person in a series of depth maps.

14. The method of claim 13, wherein:

the marker which identifies the anomaly in the body part of the first person comprises an indication of a size and shape of the anomaly in the body part of the first person.

15. The method of claim 13, wherein:

the application comprises a multiplayer application.

16. The method of claim 13, wherein:

the performing the calibration process comprises prompting the first person to stand in a specific pose or to display the body part.

17. The method of claim 13, wherein:

the performing the calibration process further comprises;

detecting a body part of a second person in the at least one depth map;

identifying, from the at least one depth map, an anomaly in the body part of the second person, the anomaly in the body part of the second person comprising;

(a) a logical gap anomaly which represents a second object of the body part of the second person appearing to be further from the 3-D imaging system than the second object actually is due to a lower reflective property of the second object, where the second object is identified by pixel depths which are less than pixel depths of an adjacent region of the second object or (b) a logical protrusion anomaly which represents the second object appearing to be closer to the 3-D imaging system than the second object actually is due to a higher reflective property of the second object, where the second object is identified by pixel depths which are greater than pixel depths of an adjacent region of the second object; and

responsive to the identifying of the anomaly in the body part of the second person, providing a marker which identifies the anomaly in the body part of the second person; and

after the performing the calibration process, during the application;

using the marker which identifies the anomaly in the body part of the second person as a template, identifying a future occurrence of the body part of the second person and differentiating between the body part of the first person and the body part of the second person, in the series of depth maps.

18. The method of claim 17, further comprising:

the performing the calibration process comprises prompting the first and second persons to stand in specific poses in which their projections as perceived by the 3-D imaging system do not overlap.

19. A computing system for processing images, comprising:

a 3-D imaging system, the 3-D imaging system generates at least one depth map from at least one image of first and second persons in a multiplayer application during a calibration process of the multiplayer application;

an object detector, the object detector identifies a body part of the first person and a body part of the second person in the at least one depth map;

an anomaly detector, the anomaly detector, responsive to the object detector, identifies, from the at least one depth map, an anomaly in the body part of the first person and an anomaly in the body part of the second person, the anomaly in the body part of the first person comprising a logical gap anomaly or a logical protrusion anomaly, and the anomaly in the body part of the second person comprising the logical gap anomaly or the logical protrusion anomaly; and

an anomaly marker, the anomaly marker, responsive to the anomaly detector, generates a marker which identifies the anomaly in the body part of the first person and a marker which identifies the anomaly in the body part of the second person, the object detector uses the marker which identifies the anomaly in the body part of the first person and the marker which identifies the anomaly in the body part of the second person to differentiate between the body part of the first person and the body part of the second person in future occurrences of the body part of the first person and the body part of the second person in a series of depth maps, after the calibration process.

20. The computing system of claim 19, wherein:

the anomaly in the body part of the first person represents a first object of the body part of the first person appearing to be further from the 3-D imaging system than the first object actually is due to a lower reflective property of the first object, where the first object is identified by pixel depths which are less than pixel depths of an adjacent region of the first object or the first object appearing to be closer to the 3-D imaging system than the first object actually is due to a higher reflective property of the first object, where the first object is identified by pixel depths which are greater than pixel depths of an adjacent region of the first object; and

the anomaly in the body part of the second person represents a second object of the body part of the second person appearing to be further from the 3-D imaging system than the second object actually is due to a lower reflective property of the second object, where the second object is identified by pixel depths which are less than pixel depths of an adjacent region of the second object or the second object appearing to be closer to the 3-D imaging system than the second object actually is due to a higher reflective property of the second object, where the second object is identified by pixel depths which are greater than pixel depths of an adjacent region of the second object.