Gesture-based human machine interface
First Claim
Patent Images
1. A method of using a computer system through a gesture-based human machine interface, the method comprising:
- pointing to a position on a screen of the computer system using an object to point, wherein pointing includes maintaining a distance between the object and the screen, the distance being a positive value, and wherein the screen comprises outer corners and a center;
capturing images of a space in front of the screen with at least four cameras configured to attach at the outer corners of the screen and point toward the center of the screen, wherein capturing images with the at least four cameras allows a three-dimensional map of the space in front of the screens to be constructed, the space including the object;
analyzing the images using a processor to identify the object, to determine the position of the object in three-dimensions, to determine the position on the screen to which the object is pointing, to determine the distance between the object and the screen, and to determine speeds of movement of the object; and
modifying information displayed on the screen in response to the determination of both the position on the screen to which the object is pointing and the distance between the object and the screen;
wherein modifying the information displayed on the screen comprises magnifying the information displayed on a portion of the screen with a magnification that is dependent upon the determined distance between the object and the screen, wherein a zoom button is positioned at the position on the screen to which the object is pointing;
wherein the determined distance is between a maximum value and a minimum value, wherein the magnification scaling varies exponentially as the determined distance varies between the maximum value and the minimum value; and
wherein a gradual speed of movement of the object towards a menu option presented on the screen is interpreted as a single click and a rapid speed of movement of the object towards the menu option presented on the screen is interpreted as a double click.
1 Assignment
0 Petitions
Accused Products
Abstract
A gesture-based human machine interface, for example, a user interface for controlling a program executing on a computer, and related method are provided. Gestures of the user are monitored and a response is provided that is based upon the detected gestures. An object is used to point to information displayed on a screen. The information displayed on the screen is modified in response to a determination of the position on the screen to which the object is pointing and in response to the distance of the object from the screen.
15 Citations
20 Claims
-
1. A method of using a computer system through a gesture-based human machine interface, the method comprising:
-
pointing to a position on a screen of the computer system using an object to point, wherein pointing includes maintaining a distance between the object and the screen, the distance being a positive value, and wherein the screen comprises outer corners and a center; capturing images of a space in front of the screen with at least four cameras configured to attach at the outer corners of the screen and point toward the center of the screen, wherein capturing images with the at least four cameras allows a three-dimensional map of the space in front of the screens to be constructed, the space including the object; analyzing the images using a processor to identify the object, to determine the position of the object in three-dimensions, to determine the position on the screen to which the object is pointing, to determine the distance between the object and the screen, and to determine speeds of movement of the object; and modifying information displayed on the screen in response to the determination of both the position on the screen to which the object is pointing and the distance between the object and the screen; wherein modifying the information displayed on the screen comprises magnifying the information displayed on a portion of the screen with a magnification that is dependent upon the determined distance between the object and the screen, wherein a zoom button is positioned at the position on the screen to which the object is pointing;
wherein the determined distance is between a maximum value and a minimum value, wherein the magnification scaling varies exponentially as the determined distance varies between the maximum value and the minimum value; and
wherein a gradual speed of movement of the object towards a menu option presented on the screen is interpreted as a single click and a rapid speed of movement of the object towards the menu option presented on the screen is interpreted as a double click. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method of using a computer system through a gesture-based human machine interface, the method comprising:
-
pointing to a position on a screen of the computer system using an object to point, wherein pointing includes maintaining a distance between the object and the screen, the distance being a positive value, and wherein the screen comprises outer corners and a center; capturing images of a space in front of the screen with at least four cameras configured to attach at the outer corners of the screen and point toward the center of the screen, wherein capturing images with the at least four cameras allows a three-dimensional map of the space in front of the screens to be constructed, the space including the object; analyzing the images using a processor to identify the object, to determine the position of the object in three-dimensions, to determine the position on the screen to which the object is pointing, and to determine the distance between the object and the screen; modifying information displayed on the screen in response to the determination of both the position on the screen to which the object is pointing and the distance between the object and the screen; analyzing the images using the processor to determine speeds of movement of the object; and modifying the information displayed on the screen differently according to different speeds of movement of the object; wherein a gradual speed of movement of the object towards a menu option presented on the screen is interpreted as a single click and a rapid speed of movement of the object towards menu option presented on the screen is interpreted as a double click; wherein when movement of the object is a slow lateral movement causes an object displayed on the screen to be moved around within the screen; wherein when movement of the object is a rapid movement causes the object displayed on the screen to be removed from the screen; and wherein modifying the information displayed on the screen further comprises magnifying the information displayed on a portion of the screen to which the object is pointing with a magnification that is dependent upon the determined distance between the object and the screen, wherein the determined distance is between a maximum value and a minimum value, and wherein the magnification scaling varies exponentially as the determined distance varies between the maximum value and the minimum value. - View Dependent Claims (8, 9)
-
-
10. An apparatus comprising:
-
a screen operable to display information, wherein the screen comprises outer corners and a center; at least four cameras configured to attach at the outer corners of the screen and point toward the center of the screen to capture images of a space in front of the screen, wherein capturing images with the at least four cameras allows a three-dimensional map of the space in front of the screens to be constructed, and a processor configured to; receive the images, analyze the images to identify an object pointing at a position on the screen, to determine a screen position relative to the object and the at least four cameras, the screen being a reference plane for determining the position on the screen to which the object is pointing, to determine a distance of the object from the screen, the distance being a positive value, and to determine speeds of movement of the object, and modify the information displayed on the screen in response to the determination of both the position on the screen to which the object is pointing and the distance of the object from the screen; wherein modifying the information displayed on the screen comprises magnifying the information displayed on a portion of the screen with a magnification that is dependent upon the determined distance between the object and the screen, wherein a zoom button is positioned at the position on the screen to which the object is pointing;
wherein the determined distance is between a maximum value and a minimum value, wherein the magnification scaling varies exponentially as the determined distance varies between the maximum value and the minimum value; and
wherein a gradual speed of movement of the object towards a menu option presented on the screen is interpreted as a single click and a rapid speed of movement of the object towards the menu option presented on the screen is interpreted as a double click. - View Dependent Claims (11, 12, 13, 14)
-
-
15. An apparatus comprising:
-
a screen operable to display information, wherein the screen comprises outer corners and a center; at least four cameras configured to attach at the outer corners of the screen and point toward the center of the screen to capture images of a space in front of the screen, wherein capturing images with the at least four cameras allows a three-dimensional map of the space in front of the screens to be constructed, and a processor configured to; receive the images, analyze the images to identify an object pointing at a position on the screen, to determine a screen position relative to the object and the at least four cameras, the screen being a reference plane for determining the position on the screen to which the object is pointing, and to determine a distance of the object from the screen, the distance being a positive value, and modify the information displayed on the screen in response to the determination of both the position on the screen to which the object is pointing and the distance of the object from the screen, and determine a speed of movement of the object and to modify the information displayed on the screen in response to the determination of the speed of movement, wherein a gradual speed of movement of the object towards a menu option presented on the screen is interpreted as a single click and a rapid speed of movement of the object towards the menu option presented on the screen is interpreted as a double click; wherein modifying the information displayed on the screen comprises magnifying the information displayed on a portion of the screen to which the object is pointing with a magnification that is dependent upon the determined distance between the object and the screen, wherein the determined distance is between a maximum value and a minimum value, and wherein the magnification scaling varies exponentially as the determined distance varies between the maximum value and the minimum value.
-
-
16. A gesture-based human interface of a computer system, the interface being a computer program product stored on a non-transitory storage medium in the computer system, and further comprising:
-
a first computer program product module configured to enable a screen operable to display information, wherein the screen comprises outer corners and a center; a second computer program product module configured to instruct at least four cameras configured to attach at the outer corners of the screen and point toward the center of the screen to capture images of a space in front of the screen; and a third computer program product module configured to control a processor configured to; receive the images, analyze the images to identify an object pointing at a position on the screen, to determine a screen position relative to the object and the at least four cameras, the screen being a reference plane for determining the position on the screen to which the object is pointing and for determining a distance of the object from the screen, the distance being a positive value, and to determine a speed of movement of the object, and modify the information displayed on the screen in response to the determination of the position on the screen to which the object is pointing, the distance of the object from the screen, and the speed of movement; wherein modifying the information displayed on the screen comprises magnifying the information displayed on a portion of the screen to which the object is pointing with a magnification that is dependent upon the determined distance between the object and the screen, wherein the determined distance is between a maximum value and a minimum value, and wherein the magnification scaling varies exponentially as the determined distance varies between the maximum value and the minimum value; and wherein a gradual speed of movement of the object towards a menu option presented on the screen is interpreted as a single click and a rapid speed of movement of the object towards the menu option presented on the screen is interpreted as a double click. - View Dependent Claims (17, 18, 19, 20)
-
Specification