Gesture recognition light and video image projector

• US 9,377,874 B2
• Filed: 11/02/2007
• Issued: 06/28/2016
• Est. Priority Date: 11/02/2007
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Aull, Kenneth W., Nishihara, H Keith, Hsu, Shi-Ping
• Primary Examiner(s): Boddie, William
• Assistant Examiner(s): Shapiro, Leonid
• Application Number:
  US11/934,580
• Publication Number:
  US 20090115721A1
• Time in Patent Office: 3,161 Days
• Field of Search: 345/156, 345/173, 345/179
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A system and method is provided for a gesture recognition interface system. The system comprises a projector configured to project colorless light and visible images onto a background surface. The projection of the colorless light can be interleaved with the projection of the visible images. The system also comprises at least one camera configured to receive a plurality of images based on a reflected light contrast difference between the background surface and a sensorless input object during projection of the colorless light. The system further comprises a controller configured to determine a given input gesture based on changes in relative locations of the sensorless input object in the plurality of images, and being further configured to initiate a device input associated with the given input gesture.

Claims

1. A gesture recognition interface system comprising:

a projector configured to project invisible light and visible images onto a background surface, the projection of the invisible light being interleaved with the projection of the visible images;

at least one camera configured to receive a plurality of images based on a reflected light contrast difference between the background surface and a sensorless input object during projection of the invisible light; and

a controller configured to determine a given input gesture based on changes in relative locations of the sensorless input object in the plurality of images, and being further configured to initiate a device input associated with the given input gesture.

2. The interface system of claim 1, wherein the invisible light is infrared (IR) light.

3. The interface system of claim 2, wherein the projector is configured as a digital light projection (DLP) projector comprising a color wheel that includes a plurality of visible light lenses and an IR lens.

4. The interface system of claim 1, further comprising an automated calibration component configured to provide a calibration pattern via the projector onto the background surface during projection of the invisible light, the calibration pattern being projected in the invisible light, the controller being configured to associate the calibration pattern with known pixel locations.

5. The interface system of claim 4, wherein the automated calibration component is further configured to calibrate the gesture recognition interface system based on a user input.

6. The interface system of claim 4, wherein the automated calibration component is further configured to calibrate the gesture recognition interface system at each of periodic time intervals.

7. The interface system of claim 1, wherein the sensorless input object is a user'"'"'s hand.

8. The interface system of claim 1, wherein the controller is further configured to determine a three-dimensional physical location of the sensorless input object, and wherein the given input gesture is associated with changes in the three-dimensional physical location of the sensorless input object.

9. The interface system of claim 1, wherein the projector is configured as a holograph projector, such that the visible images are three-dimensional holographic images.

10. The interface system of claim 1, wherein the background surface is configured as a display device for the visible images, such that the background surface operates as a touchscreen input/output device with respect to a given gesture input performed in a three-dimensional gesture recognition environment in a foreground of the background surface.

11. The interface system of claim 1, wherein the at least one camera comprises at least one pair of cameras, and wherein the controller determines a three-dimensional physical location of the sensorless input object in a three-dimensional gesture recognition environment in a foreground of the background surface based on an amount of parallax separation between matched pairs of images of the sensorless input object.

12. A method of providing device inputs, the method comprising:

projecting invisible light interleaved with visible images onto a background surface;

providing simulated inputs over the background surface via gestures associated with a sensorless input object;

generating a plurality of images associated with the sensorless input object based on a reflected light contrast between the sensorless input object and the invisible light;

determining a plurality of three-dimensional physical locations of the sensorless input object relative to the background surface;

determining if changes in the plurality of three-dimensional physical locations of the sensorless input object correspond to a given one of a plurality of predefined gestures; and

providing at least one device input based on determining that the changes in the plurality of three-dimensional physical locations of the sensorless input object correspond to the given one of the plurality of predefined gestures.

13. The method of claim 12, wherein projecting the invisible light interleaved with visible images comprises projecting infrared (IR) light and the visible images from a DLP projector comprising a color wheel that includes a plurality of visible light lenses and an IR lens.

14. The method of claim 12, wherein projecting the invisible light comprises:

projecting a calibration pattern onto the background surface during projection of the invisible light the calibration pattern being projected in the invisible light; and

correlating pixels associated with the calibration pattern with physical space relative to the background surface.

15. The method of claim 14, wherein projecting the calibration pattern comprises projecting the calibration pattern in response to an elapsed periodic time.

16. The method of claim 12, wherein generating a plurality of images comprises generating a first plurality of images associated with a first camera and generating a second plurality of images associated with a second camera, and wherein determining the plurality of three-dimensional physical locations of the sensorless input object comprises determining a height of the sensorless input object relative to the background surface based on an amount of parallax separation between the first plurality of images and the second plurality of images of the sensorless input object.

17. A gesture recognition interface system comprising:

means for projecting invisible light interleaved with visible images onto a background surface;

means for generating a plurality of images of a sensorless input object based on a brightness contrast between the background surface and the sensorless input object;

means for generating three-dimensional location information associated with the sensorless input object based on the plurality of images of the sensorless input object;

means for translating changes in the three-dimensional location information associated with the sensorless input object to a given input gesture; and

means for providing device inputs based on matching the given input gesture with one of a plurality of predefined gestures.

18. The interface system of claim 17, wherein the means for projecting invisible light comprises means for projecting a calibration pattern onto the background surface while projecting the invisible light the calibration pattern being projected in the invisible light, the system further comprising means for calibrating the gesture recognition interface system based on the calibration pattern.

19. The interface system of claim 18, wherein the means for projecting the calibration pattern is configured to project the calibration pattern at expiration of a predefined elapsed time.

20. The interface system of claim 17, wherein the invisible light is infrared (IR) light.