Identification of objects in a 3D video using non/over reflective clothing

• US 8,121,351 B2
• Filed: 03/09/2008
• Issued: 02/21/2012
• Est. Priority Date: 03/09/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft International Holdings BV (Microsoft Corporation)
• Inventors: Katz, Sagi, Yahav, Giora
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US12/044,981
• Publication Number:
  US 20090226079A1
• Time in Patent Office: 1,444 Days
• Field of Search: 382/106, 382/154, 382/209, 382/217, 345/419, 345/422, 345/584
• US Class Current: 382/106
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence

Abstract

A method includes generating a depth map from at least one image, detecting objects in the depth map, and identifying anomalies in the objects from the depth map. Another method includes identifying at least one anomaly in an object in a depth map, and using the anomaly to identify future occurrences of the object. A system includes a three dimensional (3D) imaging system to generate a depth map from at least one image, an object detector to detect objects within the depth map, and an anomaly detector to detect anomalies in the detected objects, wherein the anomalies are logical gaps and/or logical protrusions in the depth map.

Claims

1. A computer-implemented method for processing images, comprising:

generating a depth map from at least one image;

detecting at least one object in said depth map; and

identifying anomalies in said at least one object from said depth map, said identifying anomalies comprises identifying at least one logical gap anomaly and at least one logical protrusion anomaly in said at least one object which are caused by differing reflective properties of the at least one object, said identifying the at least one logical gap anomaly comprises identifying pixel depths which are less than those of pixels in at least one adjacent region of said at least one object, and said identifying the at least one logical protrusion anomaly comprises identifying pixel depths which are greater than those of pixels in at least one adjacent region of said at least one object.

2. The computer-implemented method according to claim 1, wherein said identifying anomalies comprises:

calculating a depth difference by comparing depths of at least two regions in said at least one object; and

determining that said anomalies exist where an absolute value of said depth difference exceeds a threshold.

3. The computer-implemented method according to claim 1 wherein said at least one image is a first image generated using continuous radiation and a second image of the same scene generated using pulsed radiation, and wherein said identifying anomalies comprises:

finding associated pixels of said first and second images which have intensities below a threshold;

calculating differences between the intensities of said associated pixels in said first and second images; and

determining that the anomalies exist in response to determining that the calculated differences are less than a pixel intensity threshold.

4. The computer-implemented method according to claim 3, wherein said identifying anomalies comprises:

deriving compensated depth values by dividing the intensities of the first image by the intensities of the second image; and

determining if the compensated depth values exceed a depth threshold, said determining that said anomalies exist is responsive to the determining that the compensated depth values exceed the depth threshold.

5. The computer-implemented method according to claim 1, further comprising:

compensating for said anomalies by replacing pixels associated with said anomalies with pixels of depth similar to that of at least one region adjacent to said anomalies, wherein said region is a part of said at least one object.

6. The computer-implemented method according to claim 1, wherein said depth map is in a series of depth maps, further comprising:

marking said detected anomalies; and

using said marked anomalies as a template, identifying future occurrences of said detected at least one object in said series of depth maps.

7. A computer-implemented method for processing images, comprising:

identifying anomalies in an object in a depth map in a series of depth maps, said identifying anomalies comprises identifying at least one logical gap anomaly and at least one logical protrusion anomaly in said object which are caused by differing reflective properties of the object, said identifying the at least one logical gap anomaly comprises identifying pixel depths which are less than those of pixels in at least one adjacent region of said at least one object, and said identifying the at least one logical protrusion anomaly comprises identifying pixel depths which are greater than those of pixels in at least one adjacent region of said at least one object; and

using said anomalies as a template, identifying future occurrences of said object in the series of depth maps.

8. The computer-implemented method according to claim 7, wherein said anomalies are caused by materials with lower or higher reflective properties than those of other materials represented in said depth map.

9. The computer-implemented method according to claim 7, further comprising performing said identifying anomalies as part of a calibration process prior to operation.

10. The computer-implemented method according to claim 7, wherein said object is part of a subject'"'"'s body.

11. The computer-implemented method according to claim 10, wherein said anomalies are caused by a contrast in reflectivity between at least two parts of said subject'"'"'s body.

12. The computer-implemented method according to claim 7, wherein said using comprises:

distinguishing between said object and a second similar object.

13. The computer-implemented method according to claim 12, wherein said object and said similar object are a pair of objects, and one of said pair of objects is identified as a left object and one of said pair of objects is a right object.

14. The computer-implemented method according to claim 7, further comprising:

marking said object as a specific individual.

15. The computer-implemented method according to claim 14, wherein said individual is a player in a multiplayer game.

16. The computer-implemented method according to claim 7, further comprising:

compensating for said anomalies by replacing pixels associated with said anomalies with pixels of depth similar to that of at least one region adjacent to said anomalies, wherein said at least one region is a part of said object.

17. A computer-implemented method for processing images, comprising:

generating a depth map from at least one image;

detecting objects within said depth map; and

detecting anomalies in said detected objects, wherein said anomalies are caused by differing reflective properties of the object, and comprise at least one logical gap anomaly and at least one logical protrusion anomaly in said depth map, the detecting anomalies comprises detecting the at least one logical gap anomaly by identifying pixel depths which are less than those of pixels in at least one adjacent region of said object, and detecting the at least one logical protrusion anomaly by identifying pixel depths which are greater than those of pixels in at least one adjacent region of said object.

18. The computer-implemented method according to claim 17, wherein:

the at least one image is obtained by processing images generated from both pulsed and continuous radiation; and

said detecting anomalies comprises comparing pixel intensities of pixels from associated regions of said images, and responsive to said comparing, detecting anomalies when a depth of said pixels from said associated regions of said images is closer than a threshold.

19. The computer-implemented method according to claim 17, wherein said detecting anomalies comprises comparing a difference in pixel depth between at least two regions of said detected objects.

20. The computer-implemented method according to claim 17, further comprising:

generating a modified version of said depth map without said detected anomalies.

21. The computer-implemented method according to claim 17, further comprising:

marking said detected anomalies and associating said detected anomalies with said detected objects.

22. The computer-implemented method according to claim 21, wherein said detecting objects comprises detecting and identifying said associated objects using said marked anomalies.

23. The computer-implemented method according to claim 21, wherein said marking comprises associating said detected anomalies as representing a specific individual subject.

24. The computer-implemented method according to claim 23, wherein said individual subject is a participant in a multiplayer application.

25. The computer-implemented method according to claim 21, wherein said detecting comprises using said marked anomalies, identifying an individual subject in an application with multiple participants.

26. A computer-implemented method for processing images, comprising:

analyzing reflective properties of at least one object represented in a depth map in a series of depth maps;

detecting anomalies based on said analyzing by;

identifying at least one logical gap anomaly and at least one logical protrusion anomaly in said at least one object which are caused by differences in the reflective properties of said at least one object, said identifying the at least one logical gap anomaly comprises identifying pixel depths which are less than those of pixels in at least one adjacent region of said at least one object, and said identifying the at least one logical protrusion anomaly comprises identifying pixel depths which are greater than those of pixels in at least one adjacent region of said at least one object;

wherein said at least one object is at least one of a part of a subject'"'"'s body and an object on said subject'"'"'s body;

marking said anomalies caused by differences in said reflective properties; and

using said marked anomalies as a template, identifying future occurrences of said at least one object in said series of depth maps.

27. The computer-implemented method according to claim 26, further comprising:

using said marked anomalies, distinguishing between left and right paired objects, wherein said at least one object is one of said paired objects.

28. The computer-implemented method according to claim 27, further comprising:

using said marked anomalies, distinguishing between players of a multiplayer game.

29. The computer-implemented method according to claim 27, wherein said at least one object is at least one of:

clothing, a clothing accessory, a part of said subject'"'"'s body, jewelry and a medical artifact.