Gesture recognition interface system with a light-diffusive screen

• US 8,345,920 B2
• Filed: 06/20/2008
• Issued: 01/01/2013
• Est. Priority Date: 06/20/2008
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Ferren, Bran, Nishihara, H. Keith
• Primary Examiner(s): Werner, Brian P
• Assistant Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US12/143,529
• Publication Number:
  US 20090316952A1
• Time in Patent Office: 1,656 Days
• Field of Search: 382/100, 382/103, 715/863
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

One embodiment of the invention includes a gesture recognition interface system. The interface system may comprise at least one light source positioned to illuminate a first side of a light-diffusive screen. The interface system may also comprise at least one camera positioned on a second side of the light-diffusive screen, the second side being opposite the first side, and configured to receive a plurality of images based on a brightness contrast difference between the light-diffusive screen and an input object. The interface system may further comprise a controller configured to determine a given input gesture based on changes in relative locations of the input object in the plurality of images. The controller may further be configured to initiate a device input associated with the given input gesture.

Claims

1. A gesture recognition interface system comprising:

at least one light source positioned to illuminate a first side of a light-diffusive screen configured to display visual content;

at least one camera positioned on a second side of the light-diffusive screen, the second side being opposite the first side, and configured to receive a plurality of images performed in a foreground of the second side based on a brightness contrast difference between diffused illumination from the at least one light source via the light-diffusive screen and an input object;

a projector configured to provide the visual content;

a beamsplitter configured to combine light emitted from the at least one light source and the visual content substantially on-axis on the first side of the light-diffusive screen; and

a controller configured to determine a given input gesture that is interactive with the visual content based on changes in relative locations of the input object in the plurality of images, and being further configured to initiate a device input associated with the given input gesture.

2. The interface system of claim 1, wherein the controller is configured to implement an object detection algorithm that is configured to detect a two-dimensional location of at least one feature associated with the input object in each of the plurality of images.

3. The interface system claim 1, wherein the projector is further configured as the at least one light source based on projecting the visual content and substantially unfiltered infrared (IR) light to the first side of the light-diffusive screen.

4. The interface system of claim 1, wherein the light-diffusive screen is configured as a substantially vertical visual content display surface.

5. The interface system of claim 1, wherein the at least one light source is configured to emit infrared (IR) light, and wherein each of the at least one camera comprises an IR filter, such that the plurality of images are configured as silhouette images of the input object relative to diffused IR light emitted from the second side of the light-diffusive screen.

6. The interface system of claim 1, wherein the at least one camera comprises a plurality of cameras that are each configured to substantially concurrently receive a silhouette image of the input object, the silhouette image of the input object received by each of the plurality of cameras comprising a matched set of images of the input object, and wherein the controller determines a three-dimensional physical location of the input object relative to the light-diffusive screen based on an amount of parallax separation of the input object in the matched set of images.

7. The interface system of claim 6, wherein the plurality of cameras comprises a plurality of camera pairs, each of the plurality of camera pairs being configured to concurrently receive images of different respective portions of the light-diffusive screen.

8. The interface system of claim 1, further comprising at least one Lambertian surface configured to diffusely reflect the illumination from the at least one light source to the first side of the light-diffusive screen.

9. A method of providing device inputs, the method comprising:

illuminating a first side of a light-diffusive screen with at least one light source;

providing simulated inputs in a foreground of a second side of the light-diffusive screen via gestures associated with an input object, the second side being opposite the first side of the light-diffusive screen;

generating a first plurality of images and a second plurality of images associated with the input object substantially concurrently via a first camera and a second camera, respectively, based on a brightness contrast between the input object and the second side of the light-diffusive screen;

determining a plurality of three-dimensional physical locations of the input object based on a relative separation of the input object in the first plurality of images relative to the second plurality of images;

determining if changes in the plurality of three-dimensional physical locations of the input object correspond to any of a plurality of predefined gestures; and

providing at least one device input based on a given one of the plurality of predefined gestures upon determining that the changes in the plurality of three-dimensional physical locations of the input object correspond to the given one of the plurality of predefined gestures.

10. The method of claim 9, wherein illuminating the first side of the light-diffusive screen comprises illuminating the first side of the light-diffusive screen with infrared (IR) light, and wherein generating the first and second plurality of images comprises generating a first and second plurality of silhouette images of the input object relative to diffused IR light emitted from the second side of the light-diffusive screen.

11. The method of claim 9, wherein generating the first and second plurality of images associated with the input object comprises generating a matched pair of images associated with the input object, and wherein determining the plurality of three-dimensional physical locations of the input object comprises determining a plurality of three-dimensional physical locations of at least one end-point associated with the input object.

12. The method of claim 9, further comprising displaying visual content on the light-diffusive screen, wherein providing simulated inputs comprises providing the gestures in the foreground of the second side of the light-diffusive screen as interaction with the visual content.

13. The method of claim 12, wherein displaying the visual content comprises projecting the visual content onto the first side of the light-diffusive screen.

14. The method of claim 13, wherein projecting the visual content further comprises concurrently emitting substantially unfiltered infrared (IR) light from a projector to the first side of the light-diffusive screen.

15. The method of claim 13, further comprising substantially combining light emitted from the at least one light source and the visual content substantially on-axis via a beamsplitter.

16. A gesture recognition interface system comprising:

means for projecting visual content;

means for displaying the visual content and for substantially diffusing infrared (IR) illumination provided from the means for projecting;

means for generating a first plurality of silhouette images and a second plurality of silhouette images of the input object relative to the diffused illumination, the first and second plurality of images forming a plurality of matched pairs of silhouette images of the input object;

means for generating two-dimensional location information associated with at least one feature of the input object for each of the first plurality of images and the second plurality of images;

means for interpolating three-dimensional location information associated with the input object based on the two-dimensional location information associated with the at least one feature of the input object for each of the plurality of matched pairs of images of the input object;

means for translating changes in the three-dimensional location information associated with at least one feature of the input object to a given input gesture; and

means for providing device inputs based on matching the given input gesture with one of a plurality of predefined gestures.

17. The interface system of claim 16, further comprising means for displaying visual content on the means for diffusing, the given input gesture being interactive with the visual content.