Continued virtual links between gestures and user interface elements

• US 8,717,318 B2
• Filed: 03/29/2011
• Issued: 05/06/2014
• Est. Priority Date: 03/29/2011
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Anderson, Glen J., Palmer, Ryan
• Primary Examiner(s): SHENG, TOM V
• Application Number:
  US13/074,639
• Publication Number:
  US 20120249429A1
• Time in Patent Office: 1,134 Days
• Field of Search: 345/156, 345/173, 345/175, 178 1803- 1807, 178/18.09, 715/863, 715/864
• US Class Current: 345/173
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A device includes a processor to receive input data from an image detector, where the input data includes data obtained from tracking air movements of a user'"'"'s body part interacting with a virtual object of the electronic display, the processor to map the input data to a control input to move the virtual object beyond the display. The device could, for example, include a mobile device such as a smartphone or a laptop. The virtual object could for example move to another display or to a bezel of the device. A touch screen sensor may allow the virtual object to be pinched from the display, before being lifted beyond the display. The processor may map the input data to control input to create a virtual binding of the virtual object in order to create a visual rendering of a connection between the virtual object and the user'"'"'s body part.

Claims

1. A device, comprising:

a touch screen display, and

a processor to receive (i) touch input data from the touch screen display, the touch input data including data obtained from sensing interactions of a user with a virtual object of the touch screen display and (ii) air movement input data from an image detector, the air movement input data including data obtained from tracking air movements of a body part of the user interacting with the virtual object, the processor to map the touch and air movement input data to a control input to move the virtual object beyond the touch screen display.

2. The device of claim 1, further comprising the image detector.

3. The device of claim 1, wherein the body part is a hand or a part of a hand.

4. The device of claim 1, wherein:

the touch screen display is a first display; and

the processor is to map the touch and air movement input data to the control input to move the virtual object from the first display toward a second display.

5. The device of claim 4, wherein:

the device is a first device; and

the processor is to determine a proximity to the first device of a second device including the second display before mapping the input data to the control input.

6. The device of claim 1, wherein the interactions include a pinching motion of the user'"'"'s body part with respect to a location of the virtual object on the display screen of the first display, the processor to map the first input data to the control input to pinch the virtual object from the first display.

7. The device of claim 6, wherein the air movements include a releasing motion of the user'"'"'s body part with respect to target location of the virtual object on the second display, the processor to map the second input data to the control input to release the virtual object onto the second display.

8. The device of claim 1, wherein the processor is to map the first input data and the second input data to the control input to create a virtual binding of the virtual object.

9. The device of claim 8, wherein the user'"'"'s body part is the user'"'"'s first body part, the processor to receive a third input data from the image detector, the third input data including data on air movements by a second body part of the user interacting with the virtual binding on the first display, to processor to map the third input data to the control input to manipulate the virtual binding.

10. The device of claim 1, wherein the processor is to map the input data to the control input to move the virtual object from the touch screen display to a bezel of the device.

11. The device of claim 10, wherein the processor is to map the input data to the control input to indicate a position of the virtual object on the bezel on a map of the display.

12. A device, comprising:

a processor to receive (ii) touch input data from a touch screen display of a first device, the touch input data including data obtained from sensing interactions of a user with a virtual object of the touch screen display and (ii) air movement input data from an image detector of the first device, the air movement input data including data obtained from tracking air movements of a body part of the user interacting with the virtual object, the processor to map the touch and air input data to a control input to place the virtual object onto a second display of a second electronic device.

13. The device of claim 12, further including the image detector.

14. The device of claim 12, wherein the body part is a hand or a part of a hand.

15. The device of claim 12, wherein the processor is to determine a proximity to the second electronic device of the first electronic device including the touch screen display, and to map the input data to the control input to display an optional target location of the virtual object on the second display before moving the virtual object to the second display.

16. The device of claim 12, wherein the air movements include a releasing motion of the user'"'"'s body part with respect to a target location of the virtual object on the second display.

17. The device of claim 12, wherein the input data includes data on inertial properties of the releasing motion, the processor to map the input data to the control input to determine an inertial behavior of the virtual object once on the second display.

18. A method comprising:

receiving touch input data from a touch screen display, the touch input data including data obtained from sensing interactions of a user with a virtual object of the touch screen display;

receiving air movement input data from an image detector, the input data including data obtained from tracking air movements of a body part of the user interacting with the virtual object; and

mapping the touch screen and air movement input data to a control input to move the virtual object beyond the touch screen display.

19. The method of claim 18, wherein:

the touch screen display is a first display; and

mapping includes mapping the touch screen and air movement input data to the control input to move the virtual object from the first display toward a second display.

20. The method of claim 19, wherein the touch screen display is part of a first electronic device, the method further including determining a proximity to the first electronic device of a second electronic device including the second display before mapping.

21. The method of claim 19, wherein mapping includes mapping the input data to the control input to further place the virtual object onto the second display.

22. The method of claim 21, further including determining a proximity to the second electronic device of the touch screen display, wherein mapping includes mapping the input data to the control input to display an optional target location of the virtual object on the second display before moving the virtual object to the second display.

23. The method of claim 21, wherein the air movements include a releasing motion of the user'"'"'s body part with respect to a target location of the virtual object on the second display.

24. The method of claim 21, wherein:

the input data includes data on inertial properties of the releasing motion; and

mapping includes mapping the input data to the control input to determine an inertial behavior of the virtual object once on the second display.

25. The method of claim 18, wherein:

the interactions include a pinching motion of the user'"'"'s body part with respect to a location of the virtual object on a display screen of the first display; and

mapping includes mapping the first input data to the control input to pinch the virtual object from the first display.

26. The method of claim 25, wherein:

the air movements include a releasing motion of the user'"'"'s body part with respect to a target location of the virtual object on the second display; and

mapping including mapping the second input data to the control input to release the virtual object onto the second display.

27. The method of claim 18, wherein mapping includes mapping the first input data and the second input data to the control input to create a virtual binding of the virtual object.

28. The method of claim 18, wherein mapping includes mapping the input data to the control input to move the virtual object from the electronic display to a bezel of the touch screen display.

29. The method of claim 28, wherein mapping includes mapping the input data to the control input to indicate a position of the virtual object on the bezel on a map of the touch screen display.

30. A non-transitory computer readable medium encoded with computer executable instructions, which, when accessed, cause a machine to perform operations comprising:

receiving touch input data from a touch screen display, the touch input data including data obtained from sensing interactions of a user with a virtual object of the touch screen display;

receiving air movement input data from an image detector, the air movement input data including data obtained from tracking air movements of a body part of the user interacting with the virtual object; and

mapping the touch screen and air movement input data to a control input to move the virtual object beyond the touch screen display.

31. The computer readable medium of claim 30, wherein:

the touch screen display is a first display; and

mapping includes mapping the input data to the control input to move the virtual object from the first display toward a second display.