Enhanced contrast for object detection and characterization by optical imaging based on differences between images

• US 9,652,668 B2
• Filed: 11/11/2016
• Issued: 05/16/2017
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David S., Yang, Hua
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US15/349,864
• Publication Number:
  US 20170061205A1
• Time in Patent Office: 186 Days
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/37 using transform domain methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet

Abstract

Enhanced contrast between an object of interest and background surfaces visible in an image is provided using controlled lighting directed at the object. Exploiting the falloff of light intensity with distance, a light source (or multiple light sources), such as an infrared light source, can be positioned near one or more cameras to shine light onto the object while the camera(s) capture images. The captured images can be analyzed to distinguish object pixels from background pixels.

Claims

1. A method identifying a hand by analyzing captured images, the method comprising:

activating at least one light source to illuminate a field of view containing a background and the hand including fingers;

operating a camera to capture a plurality of digital images including a first digital image captured at a time when the at least one light source is illuminating the field of view;

identifying pixels of the plurality of digital images that correspond to fingers of the hand that are visible in the plurality of digital images rather than to the background, the identifying including;

detecting, from the plurality of digital images, Gaussian brightness falloff patterns indicative of the fingers of the hand that are visible;

identifying a centerline axis of the visible fingers of the hand based on the detected Gaussian brightness falloff patterns indicative of the fingers without identifying edges of the visible fingers; and

identifying the pixels that correspond to the visible fingers based on the identified centerline axis of the visible fingers;

tracking motion of the identified pixels that correspond to the visible fingers through the plurality of digital images; and

constructing a model of the visible fingers based on the identified pixels that correspond to the visible fingers and the tracked motion of the identified pixels that correspond to the visible fingers.

2. The method of claim 1, further comprising:

comparing the tracked motion to a library of gestures;

identifying a particular gesture from the library of gestures that corresponds to the tracked motion; and

enabling the particular gesture that corresponds to the tracked motion.

3. The method of claim 1, wherein:

the constructing of the model includes constructing a 3D model of the visible fingers, including a position and a shape of the visible fingers, to geometrically determine whether the visible fingers correspond to an object of interest; and

the at least one light source is positioned such that objects of interest are located within a proximal zone of the field of view, the proximal zone extending from the camera to a distance less than twice an expected maximum distance between the objects of interest and the camera.

4. The method of claim 1, wherein:

the constructing of the model includes constructing a 3D model of the visible fingers, including a position and a shape of the visible fingers, to geometrically determine whether the visible fingers correspond to an object of interest; and

the at least one light source is positioned such that objects of interest are located within a proximal zone of the field of view, the proximal zone extending from the camera to a distance that is half or less than a distance to the background.

5. The method of claim 1, wherein edges of at least one of the visible fingers is not visible in at least one digital image of the plurality of digital images.

6. The method of claim 1, wherein the Gaussian brightness falloff patterns are characterized by a negative second derivative.

7. The method of claim 1, wherein the Gaussian brightness falloff patterns include an asymmetric curve.

8. The method of claim 1, wherein the identifying of the pixels that correspond to the visible fingers includes:

comparing pixels in successive digital images of the plurality of digital images to distinguish pixels corresponding to the visible fingers from pixels corresponding to the background.

9. The method of claim 1, wherein the identifying of the pixels that correspond to the visible fingers includes:

generating a reduced representation of each digital image of the plurality of digital images in which identified pixels corresponding to the background are zeroed out.

10. An image capture and analysis system for identifying a hand by analyzing captured images, the image capture and analysis system comprising:

at least one camera oriented toward a field of view containing a background and the hand including fingers;

at least one light source disposed on a same side of the field of view as the camera and oriented to illuminate the field of view; and

an image analyzer coupled to the at least one camera and the light source and configured to;

activate the at least one light source to illuminate the field of view;

operate the at least one camera to capture a plurality of digital images including a first digital image captured at a time when the at least one light source is illuminating the field of view;

identify pixels of the plurality of digital images that correspond to fingers of the hand that are visible in the plurality of digital images rather than to the background, the image analyzer identifying the pixels by;

detecting, from the plurality of digital images, Gaussian brightness falloff patterns indicative of the fingers of the hand that are visible;

identifying a centerline axis of the visible fingers of the hand based on the detected Gaussian brightness falloff patterns indicative of the fingers without identifying edges of the visible fingers; and

identifying the pixels that correspond to the visible fingers based on the identified centerline axis of the visible fingers; and

track motion of the identified pixels that correspond to the visible fingers through the plurality of digital images; and

construct a model of the visible fingers based on the identified pixels that correspond to the visible fingers and the tracked motion of the identified pixels that correspond to the visible fingers.

11. The image capture and analysis system of claim 10, wherein image analyzer is further configured to:

compare the tracked motion to a library of gestures;

identify a particular gesture from the library of gestures that corresponds to the tracked motion; and

enable the particular gesture that corresponds to the tracked motion.

12. The image capture and analysis system of claim 10, wherein:

the image analyzer constructs the model by constructing a 3D model of the visible fingers, including a position and a shape of the visible fingers, to geometrically determine whether the visible fingers correspond to an object of interest; and

the at least one light source is positioned such that objects of interest are located within a proximal zone of the field of view, the proximal zone extending from the camera to a distance less than twice an expected maximum distance between the objects of interest and the camera.

13. The image capture and analysis system of claim 10, wherein:

the image analyzer constructs the model by constructing a 3D model of the visible fingers, including a position and a shape of the visible fingers, to geometrically determine whether the visible fingers correspond to an object of interest; and

the at least one light source is positioned such that objects of interest are located within a proximal zone of the field of view, the proximal zone extending from the camera to a distance that is half or less than a distance to the background.

14. The image capture and analysis system of claim 10, wherein edges of at least one of the visible fingers is not visible in at least one digital image of the plurality of digital images.

15. The image capture and analysis system of claim 10, wherein the Gaussian brightness falloff patterns are characterized by a negative second derivative.

16. The image capture and analysis system of claim 10, wherein the Gaussian brightness falloff patterns include an asymmetric curve.

17. The image capture and analysis system of claim 10, wherein the image analyzer identifies the pixels that correspond to the visible fingers by:

comparing pixels in successive digital images of the plurality of digital images to distinguish pixels corresponding to the visible fingers from pixels corresponding to the background.

18. The image capture and analysis system of claim 10, wherein the image analyzer identifies the pixels that correspond to the visible fingers by:

generating a reduced representation of each digital image of the plurality of digital images in which identified pixels corresponding to the background are zeroed out.

19. A wearable goggle, comprising:

at least one camera oriented toward a field of view containing a background and a hand including fingers;

at least one light source disposed on a same side of the field of view as the camera and oriented to illuminate the field of view; and

a processor coupled to the at least one camera and the light source and configured to;

activate the at least one light source to illuminate the field of view;

operate the at least one camera to capture a plurality of digital images including a first digital image captured at a time when the at least one light source is illuminating the field of view;

identify pixels of the plurality of digital images that correspond to fingers of the hand that are visible in the plurality of digital images rather than to the background, the processor identifying the pixels by;

detecting, from the plurality of digital images, Gaussian brightness falloff patterns indicative of the fingers of the hand that are visible;

identifying a centerline axis of the visible fingers of the hand based on the detected Gaussian brightness falloff patterns indicative of the fingers without identifying edges of the visible fingers; and

identifying the pixels that correspond to the visible fingers based on the identified centerline axis of the visible fingers; and

track motion of the identified pixels that correspond to the visible fingers through the plurality of digital images; and

construct a model of the visible fingers based on the identified pixels that correspond to the visible fingers and the tracked motion of the identified pixels that correspond to the visible fingers.