IDENTIFICATION OF OBJECTS IN A 3D VIDEO USING NON/OVER REFLECTIVE CLOTHING

• US 20090226079A1
• Filed: 03/09/2008
• Published: 09/10/2009
• Est. Priority Date: 03/09/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft International Holdings BV (Microsoft Corporation)
• Inventors: Katz, Sagi, Yahav, Giora
• Granted Patent: US 8,121,351 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence

Abstract

A method includes generating a depth map from at least one image, detecting objects in the depth map, and identifying anomalies in the objects from the depth map. Another method includes identifying at least one anomaly in an object in a depth map, and using the anomaly to identify future occurrences of the object. A system includes a three dimensional (3D) imaging system to generate a depth map from at least one image, an object detector to detect objects within the depth map, and an anomaly detector to detect anomalies in the detected objects, wherein the anomalies are logical gaps and/or logical protrusions in the depth map.

Claims

1. A method comprising:

generating a depth map from at least one image;

detecting objects in said depth map; and

identifying anomalies in said objects from said depth map.

2. The method according to claim 1 and wherein said identifying comprises:

calculating a depth difference by comparing depths of at least two regions in said objects; and

determining that an anomaly exists where an absolute value of said depth difference exceeds a threshold.

3. The method according to claim 1 wherein said at least one image is a first image generated using continuous radiation and a second image of the same scene generated using pulsed radiation, and wherein said identifying comprises:

defining a depth threshold and a pixel intensity threshold;

finding associated pixels of said first and second images which have intensities below a threshold in both images; and

determining that an anomaly exists where a calculated difference between the intensity of said associated pixels is less than said pixel intensity threshold and a derived depth exceeds said depth threshold.

4. The method according to claim 1 and wherein said anomalies are logical gaps in said objects whose pixel depths are less than those of pixels in at least one adjacent region of said object.

5. The method according to claim 1 and wherein said anomalies are logical protrusions in said objects whose pixel depths are greater than those of pixels in at least one adjacent region of said object.

6. The method according to claim 1 and also comprising:

compensating for said anomalies by replacing pixels associated with said anomalies with pixels of depth similar to that of at least one region adjacent to said anomaly, wherein said region is a part of said object.

7. The method according to claim 1 and also comprising:

marking said detected anomalies; and

using said marked anomalies to identify future occurrences of said detected objects.

8. A method comprising:

identifying at least one anomaly in an object in a depth map; and

using said at least one anomaly to identify future occurrences of said object.

9. The method according to claim 8 and wherein said anomalies are logical gaps in said objects whose pixel depths are less than those of pixels in at least one adjacent region of said object.

10. The method according to claim 8 and wherein said anomalies are caused by materials with lower reflective properties than those of other materials represented in said depth map.

11. The method according to claim 8 and wherein said anomalies are logical protrusions in said objects whose pixel depths are greater than those of pixels in at least one adjacent region of said object.

12. The method according to claim 8 and wherein said anomalies are caused by materials with higher reflective properties than those of other materials represented in said depth map.

13. The method according to claim 8 and also comprising performing said identifying as part of a calibration process prior to operation.

14. The method according to claim 8 and wherein said object is a part of a subject'"'"'s body.

15. The method according to claim 8 and wherein said using comprises:

distinguishing between said object and a second similar object.

16. The method according to claim 15 and wherein said object and said similar object are a pair of objects, wherein one of said pair of objects is identified as a left object and one of said pair of objects is a right object.

17. The method according to claim 14 and wherein said at least one anomaly is caused by a contrast in reflectivity between at least two parts of said subject'"'"'s body.

18. The method according to claim 8 and also comprising:

marking said object as a specific individual.

19. The method according to claim 18 and wherein said individual is a player in a multiplayer game.

20. The method according to claim 8 and also comprising:

compensating for said anomalies by replacing pixels associated with said anomalies with pixels of depth similar to that of at least one region adjacent to said anomaly, wherein said region is a part of said object.

21. A system comprising:

a three dimensional (3D) imaging system to generate a depth map from at least one image;

an object detector to detect objects within said depth map; and

an anomaly detector to detect anomalies in said detected objects, wherein said anomalies are at least one of a logical gap and a logical protrusion in said depth map.

22. The system according to claim 21 and wherein:

said 3D imaging system comprises means to process images generated from both pulsed and continuous radiation; and

said anomaly detector comprises means to compare pixel intensities from associated regions of associated said images to detect anomalies when a depth of said pixels is closer than a threshold.

23. The system according to claim 21 and wherein said anomaly detector comprises means to compare a difference in pixel depth between at least two regions of said detected objects.

24. The system according to claim 21 and also comprising:

an anomaly compensator to generate a modified version of said depth map without said detected anomalies.

25. The system according to claim 21 and also comprising:

an anomaly marker to mark said detected anomalies and associate them with said detected objects.

26. The system according to claim 25 and wherein said object detector also comprises means to use said marked anomalies to detect and identify said associated objects.

27. The system according to claim 25 and also wherein said anomaly marker comprises means to associate said detected anomaly as representing a specific individual subject.

28. The system according to claim 27 and wherein said individual subject is a participant in a multiplayer application.

29. The system according to claim 25 and wherein said object detector also comprises means to use said marked anomalies to identify an individual subject in an application with multiple participants.

30. A method comprising:

analyzing reflective properties of at least one object represented in a depth map, wherein said at least one object is at least one of a part of a subject'"'"'s body and an object on said subject'"'"'s body;

marking anomalies caused by differences in said reflective properties; and

identifying future occurrences of said at least one object based on said marked anomalies.

31. The method according to claim 30 and also comprising:

using said marked anomalies to distinguish between left and right paired objects, wherein said at least one object is one of said paired objects.

32. The method according to claim 31 and also comprising:

using said marked anomalies to distinguish between players of a multiplayer game.

33. The method according to claim 31 and wherein said at least one object is at least one of:

clothing, a clothing accessory, a part of said subject'"'"'s body, jewelry and a medical artifact.