STEREOSCOPIC OBJECT DETECTION LEVERAGING ASSUMED DISTANCE

• US 20140376770A1
• Filed: 06/25/2013
• Published: 12/25/2014
• Est. Priority Date: 06/25/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Nister, David, Dollar, Piotr, Kienzle, Wolf, Ashman, Matthew S., Stojiljkovic, Ivan, Vukosavljevic, Magdalena, Radojevic, Mladen
• Granted Patent: US 9,934,451 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A method of object detection includes receiving a first image taken by a first stereo camera, receiving a second image taken by a second stereo camera, and offsetting the first image relative to the second image by an offset distance selected such that each corresponding pixel of offset first and second images depict a same object locus if the object locus is at an assumed distance from the first and second stereo cameras. The method further includes locating a target object in the offset first and second images.

Claims

1. A method of object detection, the method comprising;

receiving a first image taken by a first stereo camera;

receiving a second image taken by a second stereo camera;

offsetting the first image relative to the second image by an offset distance selected such that each corresponding pixel of offset first and second images depict a same object locus if the object locus is at an assumed distance from the first and second stereo cameras; and

locating a target object in the offset first and second images.

2. The method of claim 1, wherein locating includes applying a machine-learning trained classifier to the offset first and second images, the machine-learning trained classifier correlating a confidence of object detection to each pixel in the offset first and second images.

3. The method of claim 2, further comprising smoothing the correlated confidences for each of the offset first and second images.

4. The method of claim 1, wherein locating includes applying a plurality of different machine-learning trained classifiers to the offset first and second images, each of the plurality of different machine-learning trained classifiers determining a confidence of object detection for each pixel in the offset first and second images, and where a highest confidence for each pixel is correlated to that pixel.

5. The method of claim 4, wherein each of the plurality of different machine-learning trained classifiers are trained for detecting an object at an orientation different than other of the plurality of different machine-learning trained classifiers.

6. The method of claim 4, wherein each of the plurality of different machine-learning trained classifiers are trained for detecting an object of a different size than other of the plurality of different machine-learning trained classifiers.

7. The method of claim 1, wherein locating includes applying a machine-learning trained classifier to a set of offset first and second images, each offset first and second images of the set scaled to a different resolution.

8. The method of claim 7, further comprising correlating a confidence of object detection to each region of the offset first and second images, the confidence derived from the scaled offset first and second images of the resolution having a highest confidence of object detection for that region.

9. The method of claim 7, further comprising smoothing a correlated confidence across different resolutions.

10. The method of claim 1, wherein locating includes applying a machine-learning trained classifier to a set of offset first and second images, each member of the set rotated to a different orientation.

11. The method of claim 1, wherein locating includes combining a correlated confidence of object detection in the first image and a correlated confidence of object detection in the second image into a combined confidence of object detection for each corresponding pixel in the offset first and second images.

12. The method of claim 11, wherein combining includes averaging the correlated confidence of object detection in the first image and the correlated confidence of object detection in the second image.

13. The method of claim 11, further comprising discarding one or more combined confidences that are less than a threshold confidence.

14. The method of claim 11, further comprising discarding one or more combined confidences if the difference between correlated confidences of object detection in the first image and the second image is greater than a threshold.

15. A head mounted display, the head mounted display comprising:

a see-through display;

a first stereo camera configured to capture a first image;

a second stereo camera configured to capture a second image;

a logic machine; and

a storage machine including instructions executable by the logic machine to receive the first image taken by the first stereo camera, receive the second image taken by the second stereo camera, offset the first image relative to the second image by an offset distance selected such that each corresponding pixel of offset first and second images depict a same object locus if the object locus is at an assumed distance from the first and second stereo cameras, and locate a target object in the offset first and second images.

16. The head mounted display of claim 15, wherein the storage machine includes further instructions executable by the logic machine to apply a machine-learning trained classifier to the offset first and second images, the machine-learning trained classifier correlating a confidence of object detection to each pixel in the offset first and second images.

17. A method of finger detection, the method comprising;

receiving a first image taken by a first stereo camera;

receiving a second image taken by a second stereo camera;

offsetting the first image relative to the second image by an offset distance selected such that each corresponding pixel of offset first and second images depict a same finger locus if the finger locus is at an assumed distance from the first and second stereo cameras; and

applying a machine-learning trained classifier to the offset first and second images, the machine-learning trained classifier correlating a confidence of finger detection to each region in the offset first and second images.

18. The method of claim 17, wherein the machine-learning trained classifier is one of a plurality of machine-learning trained classifiers, each of the plurality of machine-learning trained classifiers trained for detecting a finger at a different orientation, and wherein each of the plurality of machine-learning trained classifiers are applied to each offset first and second images of a set of offset first and second images, each offset first and second images of the set scaled to a different resolution.

19. The method of claim 17, further comprising correlating a highest confidence of finger detection from the different orientations and resolutions to each region of the offset first and second images.

20. The method of claim 17, further comprising using the confidence of finger detection at each region to identify one or more candidate regions in either the first image or the second image, and recognizing a finger location based on the confidence of finger detection from that image.