System for and method of translating motion-based user input between a client device and an application host computer

• US 9,395,826 B1
• Filed: 05/25/2012
• Issued: 07/19/2016
• Est. Priority Date: 05/25/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Hopto Inc.
• Inventors: Cronin, John
• Primary Examiner(s): Choudhury, Azizul
• Application Number:
  US13/481,748
• Time in Patent Office: 1,516 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A system that translates motion-based user input between a client device and an application host computer. In various embodiments, the system and methods enable an application host computer, application server, or cloud server that supports a first type of input/output (I/O) devices (e.g., mouse, keyboard, and display) to interpret I/O commands from a client device that supports a second type of I/O devices (e.g., touchscreen display), wherein the I/O commands of the client device support motion-based user input. The operations of the interpreted I/O commands are then returned from the application host computer, application server, or cloud server to the client device and rendered on the display thereof. The system and methods enable cross-platform application execution by translating I/O operations that are native to the user'"'"'s remote device to I/O operations that are native to the application host computer, application server, or cloud server.

Claims

1. A method for translating a motion-based finger gesture user input between a client device and an application host device, the method comprising:

receiving, by a host device from a client device an I/O command corresponding to an application being published by the host device, the host device supporting an I/O device of a first type and the client device supporting an I/O device of a different second type, wherein the received I/O command is associated with the motion-based finger gesture user input received via the I/O device of the second type;

executing instructions stored in memory, wherein execution of the instructions by a processor of the host device;

includes executing instructions for a functionality associated with a first type of operating system, and a processor at the client device executes instructions out of a client memory that are associated with a corresponding functionality of a second type of operating system,

interprets the I/O command received from the client device as a corresponding to a mouse point and click I/O operation being generated on the host device that makes a selection in a dropdown menu at the host device, the dropdown menu associated with the published application, the interpretation based on a table stored in memory associating each of a plurality of host operations with a corresponding motion-based client I/O command, and

performs the corresponding I/O operation in the application to generate a result of the I/O operation; and

sending, by the host device, the results of the I/O operation to the client device, wherein the client device renders a display corresponding to the result of the I/O operation.

2. The method of claim 1, wherein the I/O device of the first type comprises keyboard, scroll bar, mouse, mouse wheel, or hot keys.

3. The method of claim 1, wherein the I/O device of the second type is a touchscreen and wherein the motion-based finger gesture user input comprises gestures.

4. The method of claim 3, wherein the motion-based finger gesture user input includes multi-touch gestures.

5. The method of claim 1, wherein the interpretation comprises determining whether the I/O command associated with the motion-based finger gesture user input is native to the host computer.

6. The method of claim 1, wherein the motion-based finger gesture user input is associated with a location on the I/O device.

7. The method of claim 6, wherein the I/O command is identified based on the location of the motion-based finger gesture user input on the I/O device.

8. An apparatus for translating a motion-based finger gesture user input between a client device and an application host device, the apparatus comprising:

memory for storing a table associating each of a plurality of host operations by a host device supporting an I/O device of a first type with a corresponding motion-based I/O command on a client device supporting an I/O device of a different second type;

a communication network interface for receiving from the client device, an I/O command corresponding to an application being published by the host device, and wherein the received I/O command is associated with the motion-based finger gesture user input received via the I/O device of the second type; and

a processor for executing instructions stored in memory, wherein execution of the instructions by the processor;

includes executing instructions for a functionality associated with a first type of operating system, and a processor at the client device executes instructions out of a client memory that are associated with a corresponding functionality of a second type of operating system,

interprets the I/O command received from the client device as a corresponding to a mouse point and click I/O operation being generated on the host device that makes a selection in a dropdown menu at the host device, the dropdown menu associated with the published application, the interpretation based on a table stored in memory associating each of a plurality of host operations with a corresponding motion-based client I/O command, and

performs the corresponding I/O operation in the application to generate a result of the I/O operation, and

wherein the communication network interface sends the result of the I/O operation to the client device, wherein the client device renders a display corresponding to the result of the I/O operation.

9. The apparatus of claim 8, wherein the I/O device of the second type is a touchscreen and wherein the motion-based finger gesture user input comprises gestures.

10. The apparatus of claim 9, wherein the motion-based finger gesture user input includes multi-touch gestures.

11. The apparatus of claim 8, wherein the interpretation comprises determining whether the I/O command associated with the motion-based finger gesture user input is native to the host computer.

12. The apparatus of claim 8, wherein the motion-based finger gesture user input is associated with a location on the I/O device.

13. The apparatus of claim 8, wherein the I/O command is identified based on the location of the motion-based finger gesture user input on the I/O device.

14. A non-transitory computer-readable storage medium, having embodied thereon a program executable by a processor to perform a method for translating motion-based finger gesture user input between a client device and an application host device, the method comprising:

receiving, by a host device from a client device, an I/O command corresponding to an application being published by the host device, the host device supporting an I/O device of a first type and the client device supporting an I/O device of a different second type, wherein the received I/O command is associated with the motion-based finger gesture user input received via the I/O device of the second type;

executing instructions stored in memory, wherein execution of the instructions by a processor of the host device;

includes executing instructions for a functionality associated with a first type of operating system, and a processor at the client device executes instructions out of a client memory that are associated with a corresponding functionality of a second type of operating system,

interprets the I/O command received from the client device as a corresponding to a mouse point and click I/O operation being generated on the host device that makes a selection in a dropdown menu at the host device, the dropdown menu associated with the published application, the interpretation based on a table stored in memory associating each of a plurality of host operations with a corresponding motion-based client I/O command, and

performs the corresponding I/O operation in the application to generate a result of the I/O operation; and

sending, by the host device, the results of the I/O operation to the client device, wherein the client device renders a display corresponding to the result of the I/O operation.

15. The non-transitory computer-readable storage medium of claim 14, wherein the I/O device of the first type comprises keyboard, scroll bar, mouse, mouse wheel, or hot keys.

16. The non-transitory computer-readable storage medium of claim 14, wherein the I/O device of the second type is a touchscreen and wherein the motion-based finger gesture user input comprises gestures.

17. The non-transitory computer-readable storage medium of claim 16, wherein the motion-based finger gesture user input includes multi-touch gestures.

18. The non-transitory computer-readable storage medium of claim 14, wherein the interpretation comprises determining whether the I/O command associated with the motion-based finger gesture user input is native to the host computer.

19. The non-transitory computer-readable storage medium of claim 14, wherein the motion-based finger gesture user input is associated with a location on the I/O device.