Enhanced camera-based input

• US 10,268,339 B2
• Filed: 01/08/2014
• Issued: 04/23/2019
• Est. Priority Date: 07/27/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan
• Primary Examiner(s): Wills-Burns, Chineyere D
• Application Number:
  US14/150,562
• Publication Number:
  US 20140118249A1
• Time in Patent Office: 1,931 Days
• Field of Search: 345156-184
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

Enhanced camera-based input, in which a detection region surrounding a user is defined in an image of the user within a scene, and a position of an object (such as a hand) within the detection region is detected. Additionally, a control (such as a key of a virtual keyboard) in a user interface is interacted with based on the detected position of the object.

Claims

1. A computer-implemented method comprising:

obtaining an image of a control object in a scene, the image captured by a camera;

determining a location of a first object in a user interface on a display, based on the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

determining a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

determining a first set of items; and

dynamically positioning one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein;

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

2. The computer-implemented method of claim 1 wherein the user interface is shown on a display of a mobile device.

3. The computer-implemented method of claim 1 wherein the user interface is shown on a television display.

4. The computer-implemented method of claim 1 wherein the first object comprises a representation of a user.

5. The computer-implemented method of claim 1 further comprising:

determining a location of a second object in the user interface; and

repositioning the one or more guide lines such that the items do not overlap with the second object.

6. The computer-implemented method of claim 5 further, wherein the repositioning comprises changing one or more of:

a color of an item;

a line thickness of an item;

a size of an item;

or

a transparency of an item.

7. The computer-implemented method of claim 5 wherein the repositioning comprises repositioning the one or more guide lines such that items are outside a determined range of motion of a control portion of the second object.

8. The computer-implemented method of claim 1 farther comprising:

determining a new location of the first object in the user interface; and

repositioning the one or more guide lines based on the new location such that the items do not overlap with the non-control portion of the first object.

9. The computer-implemented method of claim 1 further comprising:

receiving a selection of an item; and

creating, based on the selection, a new guide line having a second set of items aligned therewith.

10. The computer-implemented method of claim 9 wherein:

a guide line component of the one or more guide lines is generally defined within the same X-Y plane as the user interface; and

creating the new guide line comprises creating the new guide line to be generally defined within a plane other than the X-Y plane.

11. The computer-implemented method of claim 10 wherein items aligned with the new guide line decrease in size from a first end of the new guide line to a second end of the new guide line.

12. A device comprising a processor configured to:

obtain an image of a control object in a scene, the image captured by a camera;

determine a location of a first object in a user interface on a display, based on a the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

determine a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

determine a first set of items; and

dynamically position one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein;

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

13. The device of claim 12, wherein the device is a mobile device.

14. The device of claim 12, wherein the processor is further configured to:

determine a location of a second object in the user interface; and

reposition the one or more guide lines such that the items do not overlap with the second object.

15. The device of claim 14, wherein the repositioning comprises changing one or more of:

a color of an item;

a line thickness of an item;

a size of an item;

or

a transparency of an item.

16. The device of claim 14, wherein the processor is configured to reposition the one or more guide lines such that items are outside a determined range of motion of a control portion of the second object.

17. The device of claim 12, wherein the processor is further configured to:

determine a new location of the first object in the user interface; and

reposition the one or more guide lines based on the new location such that the items do not overlap with the non-control portion of the first object.

18. The device of claim 12, wherein the processor is further configured to:

receive a selection of an item; and

create, based on the selection, a new guide line having a second set of items aligned therewith.

19. The device of claim 18, wherein the processor is further configured to:

cause a guide line component of the one or more guide lines to be generally defined within the same X-Y plane as the user interface; and

create the new guide line to be generally defined within a plane other than the X-Y plane.

20. The device of claim 18, wherein the processor is configured to cause items aligned with the new guide line to be displayed in decreasing size from a first end of the new guide line to a second end of the new guide line.

21. An apparatus comprising:

means for obtaining an image of a control object in a scene, the image captured by image capturing means;

means for determining a location of a first object in a user interface on a display, based on the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

means for determining a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

means for determining a first set of items; and

means for dynamically positioning one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein;

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

22. The apparatus of claim 21 further comprising:

means for determining a location of a second object in the user interface; and

means for repositioning the one or more guide lines such that the items do not overlap with the second object.

23. The apparatus of claim 22 the means for repositioning comprises means for changing one or more of:

a color of an item;

a line thickness of an item;

a size of an item;

or

a transparency of an item.

24. The apparatus of claim 22 wherein the means for repositioning comprises means for repositioning the one or more guide lines such that items are outside a determined range of motion of a control portion of the second object.

25. The apparatus of claim 21 further comprising:

means for determining a new location of the first object in the user interface; and

means for repositioning the one or more guide lines based on the new location such that the items do not overlap with the non-control portion of the first object.

26. The apparatus of claim 21 further comprising:

means for receiving a selection of an item; and

means for creating, based on the selection, a new guide line having a second set of items aligned therewith.

27. The apparatus of claim 26 wherein:

a guide line component of the one or more guide lines is generally defined within the same X-Y plane as the user interface; and

the means for creating the new guide line comprises means for creating the new guide line to be generally defined within a plane other than the X-Y plane.

28. The apparatus of claim 27 further comprising means for causing the items aligned with the new guide line to decrease in size from a first end of the new guide line to a second end of the new guide line.

29. A non-transitory computer readable medium encoded with instructions that, when executed, operate to cause a computer to perform operations comprising:

obtaining an image of a control object in a scene, the image captured by a camera;

determining a location of a first object in a user interface on a display, based on the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

determining a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

determining a first set of items; and

dynamically positioning one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein;

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

30. The computer readable medium of claim 29 further comprising instructions for causing the computer to:

determine a location of a second object in the user interface; and

reposition the one or more guide lines such that the items do not overlap with the second object.

31. The computer readable medium of claim 29 further comprising instructions for causing the computer to:

receive a selection of an item; and

create, based on the selection, a new guide line having a second set of items aligned therewith.