Gesture-Based Human Machine Interface

• US 20110304650A1
• Filed: 05/31/2011
• Published: 12/15/2011
• Est. Priority Date: 06/09/2010
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Campillo, David Esteban, Jimenez, David Luis Scarlatti, Ortiz, Nicolas Pena, Tapia, Pablo Soriano
• Granted Patent: US 9,569,010 B2
• Time in Patent Office: Days
• US Class Current: 345/661
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A gesture-based human machine interface, for example, a user interface for controlling a program executing on a computer, and related method are provided. Gestures of the user are monitored and a response is provided that is based upon the detected gestures. An object is used to point to information displayed on a screen. The information displayed on the screen is modified in response to a determination of the position on the screen to which the object is pointing and in response to the distance of the object from the screen.

Claims

1. A method of using a computer system through a gesture-based human machine interface, the method comprising:

pointing to a position on a screen of the computer system using an object to point;

capturing images of a space in front of the screen with at least two cameras, the space including the object;

analyzing the images using a processor to identify the object, to determine the position on the screen to which the object is pointing, and to determine a distance of the object from the screen; and

modifying information displayed on the screen in response to the determination of both the position on the screen to which the object is pointing and the distance of the object from the screen.

2. The method of claim 1, wherein determining the position on the screen to which the object is pointing comprises tracking movement of the object.

3. The method of claim 1, wherein determining the position on the screen to which the object is pointing comprises determining an extension of the object toward the screen.

4. The method of claim 1, wherein modifying the information displayed on the screen comprises magnifying the information displayed on a portion of the screen to which the object is pointing with a magnification that is dependent upon the determined distance of the object from the screen.

5. The method of claim 1 further comprising:

analyzing the images using the processor to determine speeds of movement of the object; and

modifying the information displayed on the screen differently according to different speeds of movement of the object.

6. The method of claim 5 further comprising:

analyzing the images using the processor to determine a speed of movement of the object towards or away from the screen; and

modifying the information displayed on the screen in response to the determination of the speed of movement of the object towards or away from the screen.

7. The method of claim 5 further comprising:

analyzing the images using the processor to determine a speed of movement of the object laterally in front of the screen; and

modifying the information displayed on the screen in response to the determination of the speed of movement of the object.

8. The method of claim 1 further comprising:

pointing to positions on the screen of the computer system using two objects to point;

analyzing the images using a processor to identify the two objects, to determine the positions on the screen to which the two objects are pointing, and to determine distances of the two objects from the screen; and

modifying the information displayed on the screen in response to the determination of both the positions on the screen to which the two objects are pointing and the distances of the two objects from the screen.

9. The method of claim 1, wherein the object is an extended finger of a user'"'"'s hand.

10. An apparatus comprising:

a screen operable to display information;

at least two cameras configured to capture images of a space in front of the screen; and

a processor configured to

receive the images,

analyze the images to identify an object pointing at a position on the screen, to determine the position on the screen to which the object is pointing, and to determine a distance of the object from the screen, and

modify the information displayed on the screen in response to the determination of both the position on the screen to which the object is pointing and the distance of the object from the screen.

11. The apparatus of claim 10, wherein the processor is configured to analyze the images to track movement of the object to determine the position on the screen to which the object is pointing.

12. The apparatus of claim 10, wherein the processor is configured to analyze the images to determine an extension of the object toward the screen to determine the position on the screen to which the object is pointing.

13. The apparatus of claim 10, wherein the processor is further configured to determine a speed of movement of the object and to modify the information displayed on the screen in response to the determination of the speed of movement.

14. The apparatus of claim 10, wherein the processor is configured to magnify the information displayed on a portion of the screen to which the object is pointing with a magnification that is dependent upon the determined distance of the object from the screen.

15. The apparatus of claim 10, wherein the object is an extended finger of a user'"'"'s hand.

16. A gesture-based human interface of a computer system, the interface comprising:

a screen operable to display information;

at least two cameras configured to capture images of a space in front of the screen; and

a processor configured to

receive the images,

analyze the images to identify an object pointing at a position on the screen, to determine the position on the screen to which the object is pointing, to determine a distance of the object from the screen, and to determine a speed of movement of the object, and

modify the information displayed on the screen in response to the determination of the position on the screen to which the object is pointing, the distance of the object from the screen, and the speed of movement.

17. The interface of claim 16, wherein the processor is configured to magnify the information displayed on a portion of the screen to which the object is pointing with a magnification that is dependent upon the determined distance of the object from the screen.

18. The interface of claim 16, wherein the object is an extended finger of a user'"'"'s hand.

19. The interface of claim 16, wherein the processor is configured to analyze the images to determine an extension of the object toward the screen to determine the position on the screen to which the object is pointing.

20. The interface of claim 16, wherein the processor is configured to analyze the images to track movement of the object to determine the position on the screen to which the object is pointing.