UTILIZING REAL WORLD OBJECTS FOR USER INPUT

• US 20150309629A1
• Filed: 04/28/2014
• Published: 10/29/2015
• Est. Priority Date: 04/28/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: AMARIUTEI, Alexandru, HILDRETH, Evan Robbert
• Granted Patent: US 10,013,083 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Methods, systems, apparatuses and computer-readable media for utilizing real world objects to interact with a user interface are presented. The method may comprise a device processing image data to identify an interactive surface area and an interacting object. Subsequently, the device may determine at least two regions of the interactive surface area. In addition, the device may map commands of a user interface to the at least two regions of the interactive surface area. Subsequently, the device may determine a selected region of the interactive surface area based on a proximity of the interacting object to the interactive surface area. In addition, the device may perform a mapped command of the user interface, wherein the mapped command is determined based on the selected region.

Claims

1. A method of a device, comprising:

processing image data to identify an interactive surface area and an interacting object;

determining at least two regions of the interactive surface area;

mapping commands of a user interface to the at least two regions of the interactive surface area;

determining a selected region of the interactive surface area based on a proximity of the interacting object to the interactive surface area; and

performing a mapped command of the user interface, on the device, wherein the mapped command is determined based on the selected region.

2. The method of claim 1, wherein the processing the image to identify the interactive surface area is based on a pose of the interactive surface area.

3. The method of claim 1, wherein the interactive surface area is a hand and the interacting object is a thumb attached to the hand.

4. The method of claim 1, wherein the interactive surface area is a hand and the interacting object is a second hand.

5. The method of claim 1, wherein the interactive surface area may be one or more objects.

6. The method of claim 1, wherein the interacting object may be one or more objects.

7. The method of claim 1, further comprising tracking each of the at least two regions of the interactive surface area.

8. The method of claim 7, wherein the mapping commands of a user interface to regions of the interactive surface area further comprises:

selecting a number of commands of the user interface based on the tracked regions of the interactive surface area; and

correlating each tracked region with a different command.

9. The method of claim 1, wherein determining the at least two regions of the interactive surface area is based on physical features of the interactive surface area.

10. The method of claim 1, wherein the proximity comprises direct contact between the interacting object and the interactive surface area.

11. The method of claim 1, further comprising tracking the interacting object.

12. The method of claim 2, wherein the processing the image to identify the interactive surface area is further based on number of objects in the image, a size of an object, a number of regions on the object, visibility of regions on an object, mobility of the object, the interacting object or any combination thereof.

13. The method of claim 1, wherein the interacting object is detected based on number of objects in the image, size of an object, visibility of an object, mobility of an object, the interactive surface area or any combination thereof.

14. The method of claim 7, further comprising:

determining visibility of each tracked region of the interactive surface area;

if a tracked region is hidden, determining priority of commands mapped on the hidden tracked region and on a visible tracked region;

remapping commands from the hidden tracked region to the visible tracked region based on the priority of commands.

15. The method of claim 14, wherein the priority of commands are based on a command'"'"'s likelihood of use, number of commands required, organization of commands, and regions previously associated with the command.

16. The method of claim 1, further comprising:

determining a motion of the interacting object and whether the motion corresponds to a gesture;

determining boundaries of each region of the interactive surface area that allows gestures;

determining a command correlated to the gesture; and

performing the command if the motion was performed within the boundaries.

17. The method of claim 16, wherein the motion was at least partially performed within the boundaries.

18. An apparatus, comprising:

at least one processor configured to;

process image data to identify an interactive surface area and an interacting object;

determine at least two regions of the interactive surface area;

map commands of a user interface to the at least two regions of the interactive surface area;

determine a selected region of the interactive surface area based on a proximity of the interacting object to the interactive surface area;

perform a mapped command, wherein the mapped command is determined based on the selected region; and

a memory coupled to the at least one processor.

19. The apparatus of claim 18, wherein the at least one processor further configured to track each of the at least two regions of the interactive surface area.

20. The apparatus of claim 19, wherein the at least one processor configured to map commands of a user interface to the at least two regions of the interactive surface area further configured to:

select a number of commands of the user interface based at least on the tracked regions of the interactive surface area; and

correlate each tracked region with a different command.

21. The apparatus of claim 18, wherein the at least one processor configured to determine the at least two regions of the interactive surface area is based on physical features of the interactive surface area.

22. The apparatus of claim 19, wherein the at least one processor further configured to:

determine visibility of each tracked region of the interactive surface area;

if a tracked region is hidden, determine priority of commands mapped on the hidden tracked region and on a visible tracked region;

remap commands from the hidden tracked region to the visible tracked region based on the priority of commands.

23. The apparatus of claim 18, wherein the at least one processor further configured to:

determine a motion of the interacting object and if the motion corresponds to a gesture;

determine boundaries of each region of the interactive surface area that allows gestures;

determine a command correlated to the gesture; and

perform the command if the motion was at performed within the boundaries.

24. An apparatus, comprising:

means for processing image data to identify an interactive surface area and an interacting object;

means for determining at least two regions of the interactive surface area;

means for mapping commands of a user interface to the at least two regions of the interactive surface area;

means for determining a selected region of the interactive surface area based on a proximity of the interacting object to the interactive surface area; and

means for performing a mapped command of the user interface, wherein the mapped command is determined based on the selected region.

25. The apparatus of claim 24, wherein the means for mapping commands of a user interface to regions of the interactive surface area further comprises:

means for tracking each of the at least two regions of the interactive surface area;

means for selecting a number of commands of the user interface based at least on the tracked regions of the interactive surface area; and

means for correlating each tracked region with a different command.

26. The apparatus of claim 24, wherein the means for determining the at least two regions of the interactive surface area is based on physical features of the interactive surface area.

27. The apparatus of claim 24, wherein the means for identifying the interactive surface area is further based on number of objects in the image, a size of an object, visibility of areas on the object, a number of regions on the object, mobility of the object, the interacting object or any combination thereof.

28. The apparatus of claim 24, wherein the means for identifying the interacting object is further based on number of objects in the image, size of an object, visibility of an object, mobility of an object, the interactive surface area or any combination thereof.

29. A computer program product, comprising:

a non-transitory computer-readable storage medium comprising;

code for processing image data to identify an interactive surface area and an interacting object;

code for determining at least two regions of the interactive surface area;

code for mapping commands of a user interface to the at least two regions of the interactive surface area;

code for determining a selected region of the interactive surface area based on a proximity of the interacting object to the interactive surface area; and

code for performing a mapped command of the user interface, wherein the mapped command is determined based on the selected region.

30. The computer program product of claim 29, wherein the code for mapping commands of a user interface to regions of the interactive surface area further comprises:

code for tracking each of the at least two regions of the interactive surface area;

code for selecting a number of commands of the user interface based at least on the tracked regions of the interactive surface area; and

code for correlating each tracked region with a different command.