Calibration of a Gesture Recognition Interface System

• US 20090116742A1
• Filed: 11/01/2007
• Published: 05/07/2009
• Est. Priority Date: 11/01/2007
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Nishihara, H Keith
• Granted Patent: US 8,139,110 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

One embodiment of the invention includes a gesture recognition interface system that determines input gestures based on changes in relative locations of an input object to initiate device inputs associated with the input gestures. The system comprises at least one light source positioned to illuminate a background surface in a first light spectrum to generate a light contrast difference between the input object and the background surface. The system also comprises a display device configured to display at least one calibration pattern on the background surface in a second light spectrum in response to a calibration command and at least one camera configured to receive images of the background surface in both the first light spectrum and the second light spectrum. The system further comprises an automated calibration component configured to issue the calibration command and to associate features of the at least one calibration pattern in the received images with known physical locations.

Claims

1. A gesture recognition interface system that determines input gestures based on changes in relative locations of an input object to initiate device inputs associated with the input gestures, the system comprising:

at least one light source positioned to illuminate a background surface in a first light spectrum to generate a light contrast difference between the input object and the background surface;

a display device configured to display at least one calibration pattern on the background surface in a second light spectrum in response to a calibration command;

at least one camera configured to receive images of the background surface in both the first light spectrum and the second light spectrum; and

an automated calibration component configured to issue the calibration command and to associate features of the at least one calibration pattern in the received images with known physical locations.

2. The system of claim 1, wherein the first light spectrum is non-visible light and the second light spectrum is visible light.

3. The system of claim 2, wherein the first light spectrum is infrared (IR) light.

4. The system of claim 1, wherein the at least one camera comprises a filter configured to pass a first amount of the first light spectrum and a second amount of the second light spectrum, the first amount being substantially greater than the second amount.

5. The system of claim 1, wherein the automated calibration component is further configured to deactivate the at least one light source upon displaying the at least one calibration pattern on the background surface.

6. The system of claim 1, wherein the at least one calibration pattern comprises a first calibration pattern and a second calibration pattern, the display device being configured to display the first calibration pattern to define a display area associated with the background surface and to display the second calibration pattern to associate the relative locations of the input object with known physical locations in three-dimensional space.

7. The system of claim 1, wherein the display device is further configured to display image data with which the input object interacts on the background surface subsequent to a calibration procedure.

8. The system of claim 1, wherein the background surface is retroreflective, and wherein each of the at least one light source is substantially collocated with each of the respective at least one camera.

9. The system of claim 1, wherein the automated calibration component is configured to adjust an exposure of the at least one camera during a calibration procedure to increase a light contrast between the features of the at least one calibration pattern and the background surface.

10. The system of claim 1, wherein the at least one camera comprises a plurality of cameras, the automated calibration component being configured to associate the at least one calibration pattern with the known physical locations based on a parallax separation of a respective plurality of images of the features of the at least one calibration pattern.

11. A method for calibrating a gesture recognition interface system, the method comprising:

providing at least one light source configured to illuminate a background surface in a first light spectrum to generate a reflected light contrast between an input object and the illuminated background surface;

providing at least one camera configured to receive images of the illuminated background surface to a gesture recognition controller that determines input gestures based on the reflected light contrast, the at least one camera comprising a filter configured to pass a first amount of the first light spectrum and a second amount of a second light spectrum;

confirming that the at least one light source is deactivated;

adjusting an exposure of at least one camera;

displaying at least one calibration pattern on the background surface in the second light spectrum; and

associating features of the at least one calibration pattern with known physical locations based on the received images.

12. The method of claim 11, wherein the first light spectrum is infrared (IR) light and the second light spectrum is visible light.

13. The method of claim 11, further comprising:

reactivating the at least one light source subsequent to a calibration procedure;

simulating inputs over the background surface using the input object; and

translating the simulated inputs into device inputs based on the received images.

14. The method of claim 13, further comprising displaying image data with which the input object interacts on the background surface upon reactivating the at least one light source.

15. The method of claim 11, wherein displaying the at least one calibration pattern comprises:

displaying a first calibration pattern to define a display area associated with the background surface; and

displaying a second calibration pattern to associate the relative locations of the input object with known physical locations in three-dimensional space.

16. The method of claim 11, wherein the at least one camera comprises a plurality of cameras, and wherein associating the features of the at least one calibration pattern comprises:

determining a parallax separation of concurrent received images of the features of the at least one calibration pattern; and

calculating a two-dimensional location of the features of the at least one calibration pattern based on the parallax separation of the concurrent received images.

17. A gesture recognition interface system comprising:

means for receiving images of a background surface;

means for generating infrared (IR) illumination to provide a brightness contrast between the background surface and an input object;

means for generating three-dimensional location information associated with the input object based on the received images;

means for translating changes in the three-dimensional location information of the input object into a given input gesture;

means for displaying at least one calibration pattern in visible light on the background surface; and

means for calibrating the means for generating the three-dimensional location information based on features of the at least one calibration pattern in the received images.

18. The system of claim 17, wherein the means for generating the respective plurality of images each comprise a filter configured to pass a first amount of the first light spectrum and a second amount of the second light spectrum, the first amount being substantially greater than the second amount.

19. The system of claim 17, wherein the means for receiving images concurrently receives first images and second images, and wherein the means for calibrating is configured to calibrate based on a parallax separation of the features of the at least one calibration pattern in the first images relative to the second images.

20. The system of claim 17, wherein the means for displaying the at least one calibration pattern is further configured to display image data with which the input object interacts on the background surface.