MULTI-MODAL INTERACTION ON MULTI-TOUCH DISPLAY
First Claim
1. On a computing system comprising a multi-touch display, a method of multi-modal interaction, the method comprising:
- recognizing a hand posture of a user'"'"'s first hand directed at the display;
displaying on the display a modal region based on the hand posture, the modal region defining an area on the display;
applying a mode to the modal region, the mode indicating a family of functionalities to be associated with the modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode;
while the modal region remains displayed, recognizing a touch gesture from a user'"'"'s second hand directed at the display within the modal region on the display; and
performing an action on the display based upon a mapping of the touch gesture from the user'"'"'s second hand to the action.
2 Assignments
0 Petitions
Accused Products
Abstract
Embodiments are disclosed herein that relate to multi-modal interaction on a computing device comprising a multi-touch display. One disclosed embodiment comprises a method of multi-modal interaction including recognizing a hand posture of a user'"'"'s first hand directed at the display and displaying a modal region based on the hand posture, wherein the modal region defines an area on the display. The method further includes receiving an input selecting a mode to be applied to the modal region, wherein the mode indicates functionalities to be associated with the modal region and defines a mapping of touch gestures to actions associated with the mode. The method further includes, while the modal region remains displayed, recognizing a touch gesture from a user'"'"'s second hand directed at the display within the modal region and performing an action on the display based upon a mapping of the touch gesture.
108 Citations
20 Claims
-
1. On a computing system comprising a multi-touch display, a method of multi-modal interaction, the method comprising:
-
recognizing a hand posture of a user'"'"'s first hand directed at the display; displaying on the display a modal region based on the hand posture, the modal region defining an area on the display; applying a mode to the modal region, the mode indicating a family of functionalities to be associated with the modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode; while the modal region remains displayed, recognizing a touch gesture from a user'"'"'s second hand directed at the display within the modal region on the display; and performing an action on the display based upon a mapping of the touch gesture from the user'"'"'s second hand to the action. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. In a display device comprising a multi-touch display, a computing device comprising:
-
a processor; and memory comprising instructions stored thereon that are executable by the processor to; recognize a hand posture of a user'"'"'s first hand touching the display; display on the display a modal region based on the hand posture, the modal region defining an area on the display and being displayed at a location on the display based on a location of the hand posture; receive an input selecting a mode to be applied to the modal region, the mode indicating a family of functionalities to be associated with the modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode; detect a change in the location of the hand posture and in response, adjust the location of the modal region; while the modal region remains displayed, recognize a touch gesture from a second hand directed at the display within the modal region; perform an action on the display based upon a mapping of the touch gesture from the second hand to the action; and recognize a removal of the hand posture from the display and in response, remove display of the modal region from the display. - View Dependent Claims (12, 13, 14, 15, 16)
-
-
17. In a computing system comprising a multi-touch display, a method of interacting with a plurality of users, the method comprising:
-
recognizing a hand posture of a first hand of a first user directed at the display; in response to recognizing the hand posture of the first hand of the first user, displaying on the display a first modal region based on the hand posture of the first hand of the first user, the first modal region defining an area on the display; receiving an input selecting a mode to be applied to the first modal region, the mode indicating a family of functionalities to be associated with the first modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode; while the first modal region remains displayed, recognizing a touch gesture from a second hand of the first user directed at the display within the first modal region, and in response, performing a first action on the display based upon a mapping of the touch gesture from the second hand of the first user to the first action; while the first modal region remains displayed, recognizing a hand posture of a first hand of a second user directed at the display; in response to recognizing the hand posture of the first hand of the second user, displaying on the display a second modal region; receiving an input selecting a mode to be applied to the second modal region; and while the second modal region remains displayed, recognizing a touch gesture from a second hand of the second user directed at the display within the second modal region, and in response, performing a second action on the display based upon a mapping of the touch gesture from the second hand of the second user to the second action. - View Dependent Claims (18, 19, 20)
-
Specification