Enhanced contrast for object detection and characterization by optical imaging using formed difference images

• US 9,495,613 B2
• Filed: 12/04/2015
• Issued: 11/15/2016
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David, Yang, Hua
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US14/959,880
• Publication Number:
  US 20160086055A1
• Time in Patent Office: 347 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/37 using transform domain methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet

Abstract

Enhanced contrast between an object of interest and background surfaces visible in an image is provided using controlled lighting directed at the object. Exploiting the falloff of light intensity with distance, a light source (or multiple light sources), such as an infrared light source, can be positioned near one or more cameras to shine light onto the object while the camera(s) capture images. The captured images can be analyzed to distinguish object pixels from background pixels.

Claims

1. A method for analyzing captured images, the method comprising:

capturing a first plurality of digital images of a field of view using a camera and at least one light source;

identifying pixels of the first plurality of digital images that correspond to an object rather than to a background based on differences between first, second and third digital images captured by the capturing, the identifying including;

forming a first difference image based upon a difference between the first digital image captured by the capturing and the second digital image captured by the capturing;

forming a second difference image based upon the difference between the second digital image captured by the capturing and the third digital image captured by the capturing; and

determining a third difference image, wherein each of one or more pixels of the third difference image comprises a pixel of a smaller pixel value between a corresponding pixel in the first difference image and a corresponding pixel in the second difference image, wherein the third difference image identifies the pixels that correspond to the object; and

constructing a model of the object based on the identified pixels, the model including a position and a shape of the object.

2. The method of claim 1, wherein the pixel value corresponds to a brightness level, such that each of one or more pixels of the third difference image comprises one pixel from the corresponding pixels of the first difference image and the second difference image having a smaller brightness level.

3. The method of claim 2, further comprising using a threshold to distinguish foreground pixels and background pixels.

4. The method of claim 1, wherein the capturing of the first plurality of digital images comprises:

pulsing the at least one light source from a first illumination power to a second illumination power.

5. The method of claim 4, wherein the pulsing comprises:

pulsing the at least one light source at regular intervals while the camera captures the first plurality of digital images, such that a light intensity is stronger for object pixels than for background pixels.

6. The method of claim 1, wherein the first, second and third digital images are successive digital images.

7. The method of claim 1, wherein the identifying of the pixels of the first plurality of digital images comprises:

identifying pixels that correspond to a hand rather than to a ceiling.

8. The method of claim 1, wherein the identifying of the pixels of the first plurality of digital images comprises:

sending a reduced representation of each digital image of the first plurality of digital images in which identified background pixels are zeroed out.

9. The method of claim 1, wherein the constructing of the model comprises:

constructing a 3D model.

10. The method of claim 1, wherein the capturing of the first plurality of digital images further comprises:

filtering light outside a band around a peak frequency of the at least one light source.

11. The method of claim 1, wherein the identifying comprises:

detecting in at least one digital image, of the first plurality of digital images, a Gaussian brightness falloff pattern indicative of a rounded object.

12. The method of claim 1, wherein the constructing of the model further comprises:

reconstructing a position of the object from an outline of a shape of the object, as seen from a particular vantage point, and defined by the identified pixels of the first plurality of digital images, the reconstruction of the position including;

defining a plurality of lines tangent to the object in the first plurality of digital images from the particular vantage point in planes, referred to herein as “

slices”

, from the particular vantage point to the object in a given slice; and

determining the position and an approximate cross-section of the object in the slice using one or more simple closed curves.

13. A wearable goggle, comprising:

at least one camera oriented toward a field of view;

at least one light source disposed on a same side of the field of view as the at least one camera and oriented to illuminate the field of view; and

a processor coupled to the at least one camera and the at least one light source and configured to;

capture a first plurality of digital images of the field of view using the at least one camera and the at least one light source;

identify pixels of the first plurality of digital images that correspond to an object rather than to a background based on differences between first, second and third digital images captured by the capturing, the identifying including;

forming a first difference image based upon a difference between the first digital image captured by the capturing and the second digital image captured by the capturing;

forming a second difference image based upon the difference between the second digital image captured by the capturing and the third digital image captured by the capturing; and

determining a third difference image, wherein each of one or more pixels of the third difference image comprises a pixel of a smaller pixel value between a corresponding pixel in the first difference image and a corresponding pixel in the second difference image, wherein the third difference image identifies the pixels that correspond to the object; and

construct a model of the object based on the identified pixels, the model including a position and a shape of the object.

14. The wearable goggle of claim 13, wherein the pixel value corresponds to a brightness level, such that each of one or more pixels of the third difference image comprises one pixel from the corresponding pixels of the first difference image and the second difference image having a smaller brightness level.

15. A computer system comprising:

at least one camera oriented toward a field of view;

at least one light source disposed on a same side of the field of view as the at least one camera and oriented to illuminate the field of view; and

a processor coupled to the at least one camera and the at least one light source and configured to;

capture a first plurality of digital images of the field of view using the at least one camera and the at least one light source;

identify pixels of the first plurality of digital images that correspond to an object rather than to a background based on differences between first, second and third digital images captured by the capturing, the identifying including;

forming a first difference image based upon a difference between the first digital image captured by the capturing and the second digital image captured by the capturing;

forming a second difference image based upon the difference between the second digital image captured by the capturing and the third digital image captured by the capturing; and

determining a third difference image, wherein each of one or more pixels of the third difference image comprises a pixel of a smaller pixel value between a corresponding pixel in the first difference image and a corresponding pixel in the second difference image, wherein the third difference image identifies the pixels that correspond to the object; and

construct a model of the object based on the identified pixels, the model including a position and a shape of the object.

16. The computer system of claim 15, wherein the pixel value corresponds to a brightness level, such that each of one or more pixels of the third difference image comprises one pixel from the corresponding pixels of the first difference image and the second difference image having a smaller brightness level.

17. A tablet computer system comprising:

at least one camera having a field of view and oriented toward a front surface so as to capture a motion of a user;

at least one light source disposed on a same side of the field of view as the at least one camera and oriented to illuminate the field of view; and

a processor coupled to the at least one camera and the at least one light source and configured to;

capture a first plurality of digital images of the field of view using the at least one camera and the at least one light source;

identify pixels of the first plurality of digital images that correspond to an object rather than to a background based on differences between first, second and third digital images captured by the capturing, the identifying including;

forming a first difference image based upon a difference between the first digital image captured by the capturing and the second digital image captured by the capturing;

forming a second difference image based upon the difference between the second digital image captured by the capturing and the third digital image captured by the capturing; and

determining a third difference image, wherein each of one or more pixels of the third difference image comprises a pixel of a smaller pixel value between a corresponding pixel in the first difference image and a corresponding pixel in the second difference image, wherein the third difference image identifies the pixels that correspond to the object; and

construct a model of the object based on the identified pixels, the model including a position and a shape of the object.

18. The tablet computer system of claim 17, wherein the pixel value corresponds to a brightness level, such that each of one or more pixels of the third difference image comprises one pixel from the corresponding pixels of the first difference image and the second difference image having a smaller brightness level.