Systems and methods for gesture interaction with cloud-based applications

• US 9,606,629 B2
• Filed: 09/10/2012
• Issued: 03/28/2017
• Est. Priority Date: 09/09/2011
• Status: Active Grant

• Current Assignee: CloudOn Ltd. (Dropbox, Inc.)
• Original Assignee: CloudOn Ltd. (Dropbox, Inc.)
• Inventors: Binyamin, Shuki, Zaveri, Jay, Kamil, Assaf, Glik, Tzahi, Yaacoby, Doron, Morgenstern, Meir
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): Shrewsbury, Nathan
• Application Number:
  US13/608,971
• Publication Number:
  US 20130132856A1
• Time in Patent Office: 1,660 Days
• Field of Search: 715/740
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Various systems and methods described herein relate to server-based computing, where the systems and methods provide a client with access to an application executing remotely from the client device and having access to data (e.g., one or more files) residing on a cloud-based storage (e.g., provided by a third-party cloud-based storage service, such as Dropbox or Box). For some systems and methods, the application may be remotely executed and provided to the client such that the application has in-application/embedded access (hereafter, referred to as “native access”) to the cloud-based storage and files residing on the cloud-based storage. Additionally, some systems and methods may enable a remotely executed application to receive a gesture user input, even when the remotely executed application is not natively configured to receive such user input.

Claims

1. A system comprising:

a virtualization module configured to execute a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application being presented at a first client device in network communication with the first cloud-based server;

a gesture input module configured to receive, from the first client device, gesture data about a gesture input signal received at the first client device, the gesture input signal being recognizable by software on the first client device and received from a user of the first client device, the gesture input signal being associated with a request for a desired function affecting performance or behavior of the first virtual instance of the first application;

a gesture mapping module configured to map the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein to map the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and

a gesture emulation module configured to use the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to execute, in the virtual computing environment, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application.

2. The system of claim 1, wherein executing the set of input events in the virtual computing environment comprises identifying a graphical user interface element of the first virtual instance associated with at least one input event in the set of input events and executing the at least one input event with respect to the graphical user interface element.

3. The system of claim 2, wherein executing the set of input events in the virtual computing environment further comprises moving an input cursor to the graphical user interface element before executing the at least one input event with respect to the graphical user interface element.

4. The system of claim 1, wherein the gesture data comprises a gesture type, a gesture phase, gesture position, gesture velocity, an input pressure, or a gesture timestamp.

5. The system of claim 1, wherein the first gesture input comprises a finger-dragging gesture, a pinch gesture, a swipe gesture, a tap gesture, or a flicking gesture.

6. The system of claim 1, wherein the set of input events comprises a mouse-based input event, a keyboard-based input event, a gesture-based input event, an audio-based input event, or a video input event.

7. The system of claim 1, wherein the set of input events comprises a vertical scroll event, a horizontal scroll event, or a scroll-control-click event.

8. The system of claim 1, wherein the set of input events comprises a zoom command, a pan command, or a rotate command.

9. The system of claim 1, wherein executing the set of input events in the virtual computing environment comprises generating an event handling message operable in the virtual computing environment.

10. The system of claim 1, wherein mapping the first gesture input to the second input is based on a movement physics model.

11. The system of claim 10, wherein the movement physics model comprises a flick model or a polynomial movement model.

12. The system of claim 1, further comprising a storage module configured to establish access to a cloud-based storage provided by a second cloud-based server over a network, the cloud-based storage comprising a file, and the first virtualized instance having access to the file on the cloud-based storage.

13. A method comprising:

executing a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application presented at a first client device in network communication with the first cloud-based server;

receiving, from the first client device, gesture data about a gesture input signal received at the first client device, gesture input signal being recognizable by software on the first client device and received from a user of the first client device to request for a desired function affecting performance or behavior of the first virtual instance of the first application;

mapping the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein mapping the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and

executing, in the virtual computing environment, using the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application.

14. The method of claim 13, wherein executing the set of input events comprises issuing a user input message in the virtual computing environment.

15. The method of claim 13, wherein executing the set of input events in the virtual computing environment comprises identifying a graphical user interface element of the first virtual instance of the first application, the graphical user interface element associated with at least one input event in the set of input events, and executing the at least one input event with respect to the graphical user interface element.

16. The method of claim 15, wherein executing the set of input events in the virtual computing environment further comprises moving an input cursor to the graphical user interface element before executing the at least one input event with respect to the graphical user interface element.

17. The method of claim 13, wherein the gesture data comprises a gesture type, a gesture phase, gesture position, gesture velocity, an input pressure, or a gesture timestamp.

18. The method of claim 13, wherein the first gesture input comprises a finger-dragging gesture, a pinch gesture, a swipe gesture, a tap gesture, or a flicking gesture.

19. The method of claim 13, wherein the set of input events comprises a mouse-based input event, a keyboard-based input event, a gesture-based input event, an audio-based input event, or a video input event.

20. The method of claim 14, wherein the set of input events comprises a vertical scroll event, a horizontal scroll event, or a scroll-control-click event.

21. The method of claim 13, wherein the set of input events comprises a zoom command, a pan command, or a rotate command.

22. The method of claim 13, wherein executing the set of input events in the virtual computing environment comprises generating an event handling message operable in the virtual computing environment.

23. The method of claim 13, wherein mapping the first gesture input to the second input is based on a movement physics model.

24. The method of claim 23, wherein the movement physics model comprises a flick model or a polynomial movement model.

25. The method of claim 13, further comprising establishing access to a cloud-based storage provided by a second cloud-based server over a network, the cloud-based storage comprising a file, and the first virtual instance having access to the file on the cloud-based storage.

26. A non-transitory computer readable medium comprising executable instructions, the instructions executable by a processor to perform a method, the method comprising:

executing a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application presented at a first client device in network communication with the first cloud-based server;

receiving, from the first client device, gesture data about a gesture input signal received at the first client device, gesture input signal being recognizable by software on the first client device and received from a user of the first client device to request for a desired function affecting performance or behavior of the first virtual instance of the first application;

mapping the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein mapping the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and

executing, in the virtual computing environment, using the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application.

27. A system comprising:

means for executing a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application presented at a first client device in network communication with the first cloud-based server;

means for receiving, from the first client device, gesture data about a gesture input signal received at the first client device, gesture input signal being recognizable by software on the first client device and received from a user of the first client device to request for a desired function affecting performance or behavior of the first virtual instance of the first application;

means for mapping the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein mapping the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and

means for executing, in the virtual computing environment, using the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application.