MULTI-TOUCH INTERFACE GESTURES FOR KEYBOARD AND/OR MOUSE INPUTS

• US 20120127206A1
• Filed: 01/31/2012
• Published: 05/24/2012
• Est. Priority Date: 08/30/2010
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: THOMPSON, Robert Lee, HUA, Xueliang, ZHANG, Zheng, CASARES, Juan, SHAO, Wenlong, ZHANG, Li-Zhu
• Granted Patent: US 9,639,186 B2
• Time in Patent Office: Days
• US Class Current: 345/661
• CPC Class Codes: G06F 2203/0383 Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A mouse-and-keyboard based user interface is updated based on gestures made on a touch screen that is displaying the mouse-and-keyboard based user interface. The user interface update process includes the steps of receiving one or more touch events in response to a gesture made on the touch screen, translating the touch events to a mouse-and-keyboard based command, transmitting the mouse-and-keyboard based command to an operating system, and receiving an updated display in response thereto.

Claims

1. A method of interpreting gestures made on a touch screen that is displaying a mouse-and-keyboard based user interface (UI), comprising:

receiving a press-and-hold event in combination with a tap event in response to gestures made on the touch screen;

translating the received events to a mouse right-click command;

transmitting the mouse right-click command to an operating system; and

receiving from the guest operating system, in response to the transmitted mouse right-click command, an updated mouse-and-keyboard based UI.

2. The method of claim 1, wherein the operating system is executing on a remote machine.

3. The method of claim 1, wherein the operating system is a guest operating system executing in a virtual machine.

4. The method of claim 1, wherein a location of the tap event is greater than a threshold distance from the location of press-and-hold event.

5. The method of claim 1, wherein the press-and-hold event is received in response to a user placing and holding a finger in contact with the touch screen.

6. The method of claim 1, wherein the updated mouse-and-keyboard based UI includes a display of a context menu at a location of the press-and-hold event.

7. A method of interpreting gestures made on a touch screen that is displaying a mouse-and-keyboard based user interface (UI), comprising:

receiving a first press-and-hold event for a time period that is longer than a predetermined time period to cause a loupe that provides an enlarged view of a portion of the UI within the loupe to be displayed and, at the end of the time period, to cause a drag handle to be displayed in place of the loupe;

receiving a second press-and-hold event at a location of the drag handle; and

transmitting mouse down commands with locations that correspond to updated locations of the second press-and-hold event.

8. The method of claim 7, further comprising:

determining that the second press-and-hold event is terminated; and

causing the drag handle to no longer be displayed.

9. The method of claim 7, wherein the drag handle overlies an icon included in the UI when the second press-and-hold event is received.

10. The method of claim 9, wherein a location of the icon in the UI is changed as the location of the second press-and-hold event is updated.

11. The method of claim 7, wherein the drag handle overlies a boundary of a window included in the UI when the second press-and-hold event is received.

12. The method of claim 11, wherein the window included in the UI is resized as the location of the second press-and-hold event is updated.

13. A method of interpreting gestures made on a touch screen that is displaying a mouse-and-keyboard based user interface (UI), comprising:

receiving a simultaneous multi-finger tap event to cause a virtual keyboard to be displayed;

determining that the virtual keyboard obscures a portion of the UI on which the multi-finger tap event was made; and

in response to said determining, causing the UI to be panned such that the portion of the UI is visible to the user.

14. The method of claim 13, where the multi-finger tap event is received in response to a user tapping three fingers on the touch screen.

15. A computing device comprising:

a touch screen displaying a user interface (UI) having a cursor, and a virtual touch pad overlaid on the UI; and

a processing unit programmed to recognize gestures made on the touch screen and execute, in response thereto, a process associated with the gestures, wherein the gestures made on top of the virtual touch pad cause an updated UI with a new position of the cursor to be displayed.

16. The computing device of claim 15, wherein the virtual touch pad is semi-transparent so that portions of the UI beneath the virtual touch pad are visible.

17. The computing device of claim 15, wherein the virtual touch pad is movable to overlie different portions of the UI.

18. The computing device of claim 15, wherein the virtual touch pad includes a mouse navigation portion, a left click button, and a right click button.