Mechanism to enhance user experience of mobile devices through complex inputs from external displays

• US 10,338,684 B2
• Filed: 03/26/2014
• Issued: 07/02/2019
• Est. Priority Date: 03/26/2014
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Veeramani, Karthik, Paidipathi, Ujwal, Joshi, Ajit Prakash
• Primary Examiner(s): Sadio, Insa
• Application Number:
  US14/226,008
• Publication Number:
  US 20150277568A1
• Time in Patent Office: 1,924 Days
• Field of Search: 345173- 17, 725 14
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A method of mapping user movements captured by a capture device external to a computing device, to inputs events on the computing device, may comprise executing an application on the computing device, using at least one processor of the computing device. The computing device may transmit video data of the application to a receiver device. The computing device may receive gesture data associated with the application, the gesture data based on movements of a user captured from a capture device communicatively coupled to the receiver device. The gesture data may be mapped to an input event on the computing device and data simulating the input event may be provided to a sensor on the computing device.

Claims

1. A computing device for mapping user movements captured by a capture device external to the computing device, to input events on the computing device, the computing device comprising:

a display to display video data of an application executing on the computing device;

at least one processor;

a storage device comprising instructions, which when executed by the at least one processor, configure the at least one processor to;

establish a video channel between the computing device and a receiver device external to the computing device;

establish an input channel between the computing device and the receiver device;

transmit, over the video channel, the video data of the application to the receiver device; and

receive gesture data associated with the application, over the input channel, the gesture data based on movements of a user captured from a capture device, external to the computing device, communicatively coupled to the receiver device;

map the gesture data into an input event on the computing device; and

provide data simulating the input event to a sensor of the computing device.

2. The computing device of claim 1, wherein to map the gesture data into an input event on the computing device, the at least one processor is configured to:

retrieve a type of movement of the user from the gesture data; and

determine the input event based on the type of movement.

3. The computing device of claim 2, wherein when the type of movement of the user is a swipe movement the gesture data is mapped to a touch input event and data simulating the touch input event is provided to a touch sensor of the computing device.

4. The computing device of claim 2, wherein when the type of movement of the user is a lean movement the gesture data is mapped to a rotate input event and data simulating the rotate input event is provided to a gyroscope of the computing device.

5. The computing device of claim 1, wherein to transmit video data of the application to the receiver device, the at least one processor is configured to:

stream video displayed on the display of the computing device to the receiver device via the video channel.

6. The computing device of claim 1, wherein the at least one processor is further configured to transmit updated video data to the receiver device based on the application updating in response to the input event.

7. The computing device of claim 1, wherein the computing device is a mobile phone.

8. The computing device of claim 1, wherein the receiver device is a television.

9. A method of mapping user movements captured by a capture device external to a computing device, to input events on the computing device, the method comprising:

executing an application on the computing device, using at least one processor of the computing device;

establishing a video channel between the computing device and a receiver device external to the computing device;

transmitting, from the computing device, video data of the application to the receiver device external to the computing device;

establishing an input channel between the computing device and the receiver device;

receiving, via the input channel at the computing device from the receiver device, gesture data associated with the application, the gesture data based on movements of a user captured from a capture device, external to the computing device, communicatively coupled to the receiver device;

mapping the gesture data to an input event on the computing device; and

providing data simulating the input event to a sensor on the computing device.

10. The method of claim 9, wherein mapping the gesture data into an input event on the computing device comprises:

retrieving a type of movement of the user from the gesture data; and

determining the input event based on the type of movement.

11. The method of claim 10, wherein mapping the gesture data into an input event further comprises:

mapping the gesture data to a touch input event when the type of movement is a swipe movement.

12. The method of claim 11, wherein providing the data simulating the input event to the sensor on the computing device comprises:

providing data simulating the touch input event to a touch sensor on the computing device.

13. The method of claim 10, wherein mapping the gesture data into an input event further comprises:

mapping the gesture data to a rotate input event when the type of movement is a lean movement.

14. The method of claim 13, wherein providing the data simulating the input event to the sensor on the computing device comprises:

providing data simulating the rotate input event to a gyroscope sensor on the computing device.

15. The method of claim 9, wherein transmitting video data of the application to a receiver device comprises:

streaming video displayed on a display of the computing device to the receiver device via the video channel.

16. The method of claim 15, wherein streaming video displayed on a display of the computing device to the receiver device comprises:

wirelessly streaming the video to the receiver device.

17. The method of claim 9, further comprising:

transmitting updated video data to the receiver device based on the application updating in response to the input event.

18. At least one non-transitory machine-readable storage medium including instructions for mapping user movements captured by a capture device external to a computing device to input events on the computing device, which when executed by a machine, cause the machine to perform operations comprising:

executing an application on the computing device;

establishing a video channel between the computing device and a receiver device external to the computing device;

transmitting, from the computing device, video data of the application to the receiver device;

establishing an input channel between the computing device and the receiver device;

receiving, via the input channel at the computing device from the receiver device, gesture data associated with the application, the gesture data based on movements of a user captured from a capture device communicatively coupled to the receiver device;

mapping the gesture data to an input event on the computing device; and

providing data simulating the input event to a sensor on the computing device.

19. The at least one machine-readable storage medium of claim 18, wherein mapping the gesture data into an input event on the computing device comprises:

retrieving a type of movement of the user from the gesture data; and

determining the input event based on the type of movement.

20. A capturing system to map user movements captured by a capture device to input events on a computing device, the system comprising:

a network device;

at least one processor;

a storage device comprising instructions, which when executed by the at least one processor, configure the at least one processor to;

receive video data of an application executing on a computing device external to the capturing system, the video data received over a video channel established between the computing device and the capturing system;

receive user movement data of a user; and

process the user movement data to determine a user gesture type and output gesture data;

map the gesture data into an input event on the computing device;

generate mapped input data simulating the input event; and

wherein the network device is to transmit the mapped input data to the computing device over an input channel established between the computing device and the capturing system.

21. The capturing system of claim 20, wherein to map the gesture data into an input event on the computing device, the at least one processor is configured to:

retrieve a type of movement of the user from the gesture data; and

determine the input event based on the type of movement.

22. The capturing system of claim 21, wherein when the type of movement of the user is a swipe movement the gesture data is mapped to a touch input event and wherein the mapped input data includes data simulating the touch input event fora touch sensor of the computing device.

23. The capturing system of claim 21, wherein when the type of movement of the user is a lean movement the gesture data is mapped to a rotate input event and wherein the mapped input include data simulating the rotate input event is provided fora gyroscope of the computing device.