Systems and Methods for Gesture Interaction with Cloud-Based Applications

• US 20130132856A1
• Filed: 09/10/2012
• Published: 05/23/2013
• Est. Priority Date: 09/09/2011
• Status: Active Grant

• Current Assignee: CloudOn Ltd. (Dropbox, Inc.)
• Original Assignee: CloudOn Ltd. (Dropbox, Inc.)
• Inventors: Binyamin, Shuki, Zaveri, Jay, Kamil, Assaf, Glik, Tzahi, Yaacoby, Doron, Morgenstern, Meir
• Granted Patent: US 9,606,629 B2
• Time in Patent Office: Days
• US Class Current: 715/740
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Various systems and methods described herein relate to server-based computing, where the systems and methods provide a client with access to an application executing remotely from the client device and having access to data (e.g., one or more files) residing on a cloud-based storage (e.g., provided by a third-party cloud-based storage service, such as Dropbox or Box). For some systems and methods, the application may be remotely executed and provided to the client such that the application has in-application/embedded access (hereafter, referred to as “native access”) to the cloud-based storage and files residing on the cloud-based storage. Additionally, some systems and methods may enable a remotely executed application to receive a gesture user input, even when the remotely executed application is not natively configured to receive such user input.

Claims

1. A system comprising:

a virtualization module configured to execute a virtualized application instance based on a version of the application in a virtual computing environment on a first cloud-based server;

a gesture input module configured to provide gesture data for a gesture input received at a remote client device;

a gesture mapping module configured to map the gesture input to a set of input events based on the gesture data; and

a gesture emulation module configured to execute the set of input events in the virtual computing environment such that the set of input events affects the virtualized application instance.

2. The system of claim 1, wherein executing the set of input events in the virtual computing environment comprises identifying a graphical user interface element of the virtualized application instance associated with at least one input event in the set of input events and executing the at least one input event with respect to the identified graphical user interface element.

3. The system of claim 2, wherein executing the set of input events in the virtual computing environment further comprises moving an input cursor to the graphical user interface element before executing the at least one input event with respect to the identified graphical user interface.

4. The system of claim 1, wherein the gesture data comprises a gesture type, a gesture phase, gesture position, gesture velocity, an input pressure, or a gesture timestamp.

5. The system of claim 1, wherein providing gesture data for the gesture input comprises receiving the gesture input and generating gesture data based on the gesture input.

6. The system of claim 1, wherein the gesture input comprises a finger-dragging gesture, a pinch gesture, a swipe gesture, a tap gesture, and a flicking gesture.

7. The system of claim 1, wherein the set of input events comprises a mouse-based input event, keyboard-based input event, a gesture-based input event, an audio-based input event, or a video input event.

8. The system of claim 1, wherein the set of input events comprises a vertical scroll event, a horizontal scroll event, or a scroll-control-click event.

9. The system of claim 1, wherein the set of input events comprises a zoom command, a pan command, or a rotate command.

10. The system of claim 1, wherein executing the set of input events in the virtual computing environment comprises generating an event handling message operable in the virtual computing environment.

11. The system of claim 1, wherein mapping the gesture input to the set of input events is further based on a movement physics model.

12. The system of claim 11, wherein the movement physics model comprises a flick model or a polynomial movement model.

13. The system of claim 1, further comprising a storage module configured to establish access to a cloud-based storage provided by a second cloud-based server over a network, the cloud-based storage comprising a file, and the virtualized application instance having access to the file on the cloud-based storage.

14. A method comprising:

executing a virtualized application instance based on a version of the application in a virtual computing environment on a first cloud-based server;

providing gesture data for a gesture input received at a client device;

mapping the gesture input to a set of input events based on the gesture data; and

executing the set of input events in the virtual computing environment such that the set of input events affects the virtualized application instance.

15. The method of claim 14, wherein executing the set of input events comprises issuing a user input message in the virtual computing environment.

16. The method of claim 14, wherein executing the set of input events in the virtual computing environment comprises identifying a graphical user interface element of the virtualized application instance associated with at least one input event in the set of input events and executing the at least one input event with respect to the identified graphical user interface element.

17. The method of claim 16, wherein executing the set of input events in the virtual computing environment further comprises moving an input cursor to the graphical user interface element before executing the at least one input event with respect to the identified graphical user interface.

18. The method of claim 14, wherein the gesture data comprises a gesture type, a gesture phase, gesture position, gesture velocity, an input pressure, or a gesture timestamp.

19. The method of claim 14, wherein providing gesture data for the gesture input comprises receiving the gesture input and generating gesture data based on the gesture input.

20. The method of claim 14, wherein the gesture input comprises a finger-dragging gesture, a pinch gesture, a swipe gesture, a tap gesture, and a flicking gesture.

21. The method of claim 14, wherein the set of input events comprises a mouse-based input event, keyboard-based input event, a gesture-based input event, an audio-based input event, or a video input event.

22. The method of claim 15, wherein the set of input events comprises a vertical scroll event, a horizontal scroll event, or a scroll-control-click event.

23. The method of claim 14, wherein the set of input events comprises a zoom command, a pan command, or a rotate command.

24. The method of claim 14, wherein executing the set of input events in the virtual computing environment comprises generating an event handling message operable in the virtual computing environment.

25. The method of claim 14, wherein mapping the gesture input to the set of input events is further based on a movement physics model.

26. The method of claim 25, wherein the movement physics model comprises a flick model or a polynomial movement model.

27. The method of claim 14, further comprising establishing access to a cloud-based storage provided by a second cloud-based server over a network, the cloud-based storage comprising a file, and the virtualized application instance having access to the file on the cloud-based storage.

28. A non-transitory computer readable medium comprising executable instructions, the instructions executable by a processor to perform a method, the method comprising:

executing a virtualized application instance based on a version of the application in a virtual computing environment on a first cloud-based server;

providing gesture data for a gesture input received at a remote client device;

mapping the gesture input to a set of input events based on the gesture data; and

executing the set of input events in the virtual computing environment such that the set of input events affects the virtualized application instance.

29. A system comprising:

means for executing a virtualized application instance based on a version of the application in a virtual computing environment on a first cloud-based server;

means for providing gesture data for a gesture input received at a remote client device;

means for mapping the gesture input to a set of input events based on the gesture data; and

means for executing the set of input events in the virtual computing environment such that the set of input events affects the virtualized application instance.