Multi-modal interaction on multi-touch display
First Claim
1. On a computing system comprising a multi-touch display, a method of multi-modal interaction, the method comprising:
- displaying on the display a first content item and a second content item;
recognizing a hand posture of a user'"'"'s first hand directed at the display;
displaying on the display a modal region based on the hand posture, the modal region defining an area on the display at a first location on the display based upon a location of the hand posture;
applying a mode to the modal region, the mode indicating a family of functionalities to be associated with the modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode;
while the modal region remains displayed, recognizing a first touch gesture interacting with the first content item within the modal region at the first location on the display;
performing a first action on the display based upon a mapping of the first touch gesture to the first action;
,detecting a change in the location of the hand posture;
in response, displaying a movement of the modal region to a second location on the display such that the modal region moves over the second content item;
recognizing a second touch gesture interacting with the second content item within the modal region at the second location on the display; and
performing a second action on the display based upon a mapping of the second touch gesture to the second action.
2 Assignments
0 Petitions
Accused Products
Abstract
Embodiments are disclosed herein that relate to multi-modal interaction on a computing device comprising a multi-touch display. One disclosed embodiment comprises a method of multi-modal interaction including recognizing a hand posture of a user'"'"'s first hand directed at the display and displaying a modal region based on the hand posture, wherein the modal region defines an area on the display. The method further includes receiving an input selecting a mode to be applied to the modal region, wherein the mode indicates functionalities to be associated with the modal region and defines a mapping of touch gestures to actions associated with the mode. The method further includes, while the modal region remains displayed, recognizing a touch gesture from a user'"'"'s second hand directed at the display within the modal region and performing an action on the display based upon a mapping of the touch gesture.
-
Citations
18 Claims
-
1. On a computing system comprising a multi-touch display, a method of multi-modal interaction, the method comprising:
-
displaying on the display a first content item and a second content item; recognizing a hand posture of a user'"'"'s first hand directed at the display; displaying on the display a modal region based on the hand posture, the modal region defining an area on the display at a first location on the display based upon a location of the hand posture; applying a mode to the modal region, the mode indicating a family of functionalities to be associated with the modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode; while the modal region remains displayed, recognizing a first touch gesture interacting with the first content item within the modal region at the first location on the display; performing a first action on the display based upon a mapping of the first touch gesture to the first action;
,detecting a change in the location of the hand posture; in response, displaying a movement of the modal region to a second location on the display such that the modal region moves over the second content item; recognizing a second touch gesture interacting with the second content item within the modal region at the second location on the display; and performing a second action on the display based upon a mapping of the second touch gesture to the second action. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. In a display device comprising a multi-touch display, a computing device comprising:
-
a processor; and memory comprising instructions stored thereon that are executable by the processor to; display on the display a first content item and a second content item; recognize a hand posture of a hand touching the display; display on the display a modal region based on the hand posture, the modal region defining an area on the display and being displayed at a first location on the display based on a location of the hand posture; receive an input selecting a mode to be applied to the modal region, the mode indicating a family of functionalities to be associated with the modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode; recognize a first touch gesture interacting with the first content item within the modal region at the first location on the display; perform a first action on the display based upon a mapping of the first touch gesture to the first action; detect a change in the location of the hand posture and in response, adjust the location of the modal region to a second location on the display such that the modal region moves over the second content item; while the modal region remains displayed, recognize a second touch gesture from a second hand directed at the second content item within the modal region at the second location on the display; perform a second action on the display based upon the mapping of the second touch gesture to the second action; and recognize a removal of the hand posture from the display and in response, remove display of the modal region from the display. - View Dependent Claims (10, 11, 12, 13, 14)
-
-
15. In a computing system comprising a multi-touch display, a method of interacting with a plurality of users, the method comprising:
-
displaying a first content item and a second content item on the display; recognizing a hand posture of a first hand of a first user directed at the display; in response to recognizing the hand posture of the first hand of the first user, displaying on the display a first modal region based on the hand posture of the first hand of the first user, the first modal region defining an area on the display and being displayed at a first location on the display based upon a location of the hand posture; receiving an input selecting a mode to be applied to the first modal region, the mode indicating a family of functionalities to be associated with the first modal region and defining a mapping of one or more touch gestures to one or more corresponding actions associated with the mode; while the first modal region remains displayed, recognizing a touch gesture from a second hand of the first user directed at the first content item within the first modal region, and in response, performing a first action on the display based upon a mapping of the touch gesture from the second hand of the first user to the first action; detecting a change in the location of the hand posture, and in response adjusting the location of the first modal region to a second location on the display such that the first modal region moves over the second content item; while the first modal region remains displayed, recognizing a second touch gesture from a second hand directed at the second content item within the first modal region at the second location on the display; perform a second action on the display based upon the mapping of the second touch gesture to the second action; while the first modal region remains displayed, recognizing a hand posture of a first hand of a second user directed at the display; in response to recognizing the hand posture of the first hand of the second user, displaying on the display a second modal region; receiving an input selecting a mode to be applied to the second modal region; and while the second modal region remains displayed, recognizing a third touch gesture directed at the display within the second modal region, and in response, performing a third action on the display based upon a mapping of the third touch gesture to the third action. - View Dependent Claims (16, 17, 18)
-
Specification