SYSTEMS AND TECHNIQUES FOR USER INTERFACE CONTROL

• US 20150324001A1
• Filed: 03/27/2014
• Published: 11/12/2015
• Est. Priority Date: 01/03/2014
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: YANAI, Yaron, ELHADAD, Eli, ROSENBERGER, Amir
• Granted Patent: US 9,395,821 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Embodiments of systems and techniques for user interface (UI) control are disclosed herein. In some embodiments, a UI control system may determine locations of landmarks on a body of a user of a computing system, determine a pointer based at least in part on the landmark locations, and identify a UI element of an UI of the computing system based at least in part on the pointer. Other embodiments may be described and/or claimed.

Claims

1. A user interface, UI, control system, comprising:

landmark logic to determine locations of a plurality of landmarks associated with a portion of a body of a user of a computing system;

pointer logic, coupled with the landmark logic, to determine a pointer based at least in part on the plurality of landmark locations;

identification logic, coupled with the pointer logic, to identify a UI element from a plurality of UI elements of a UI of the computing system, based at least in part on the pointer; and

display logic, coupled with the pointer logic, to provide a visual indicator of the identified UI element in a display of the computing system.

2. The UI control system of claim 1, wherein the landmark logic is to determine the locations of the plurality of landmarks on the portion of the user'"'"'s body based at least in part on an image of the user'"'"'s body captured by a depth camera.

3. The UI control system of claim 1, wherein the portion of the user'"'"'s body comprises the user'"'"'s hand.

4. The UI control system of claim 1, wherein the identification logic is to identify a UI element from a plurality of UI elements through:

identification of a plurality of areas in the UI corresponding to respective ones of the plurality of UI elements; and

identification of an amount of overlap between the area corresponding to the UI element and the pointer.

5. The UI control system of claim 4, wherein the amount of overlap between the area corresponding to the UI element and the pointer is based on a weight of a landmark.

6. The UI control system of claim 5, wherein the weight is based on a distance between a location of the landmark and a reference location on the portion of the user'"'"'s body.

7. The UI control system of claim 1, wherein the pointer comprises an area in the UI, and the area is based on a distance between a location of a landmark and a reference location on the portion of the user'"'"'s body.

8. The UI control system of claim 1, further comprising:

selection logic, coupled with the identification logic, to select the UI element, subsequent to identification of the UI element, based at least in part on gesture data indicative of a gesture of the user.

9. The UI control system of claim 8, further comprising display logic, coupled to the pointer logic, wherein:

the gesture data indicative of the gesture of the user includes first gesture data indicative of a hover over the UI element for a predetermined period of time;

the display logic is to display a confirmation UI element based on the first gesture data;

the gesture data indicative of a user gesture includes second gesture data indicative of a selection of the confirmation UI element subsequent to display of the confirmation UI element; and

the selection logic is to select the UI element based on the second gesture data.

10. A user interface, UI, control system, comprising:

landmark logic to determine a location of a lower portion of an index finger of a hand of a user of a computing system, the lower portion spaced away from a tip of the index finger;

pointer logic, coupled with the landmark logic, to determine a pointer based at least in part on the location of the lower portion of the index finger of the user'"'"'s hand;

identification logic, coupled with the pointer logic, to identify a UI element from a plurality of UI elements of a UI of the computing system, based at least in part on the pointer; and

display logic, coupled with the pointer logic, to provide a visual indicator of the pointer in a display of the computing system.

11. The UI control system of claim 10, wherein the lower portion of the index finger of the user'"'"'s hand is proximate to a base joint of the index finger of the user'"'"'s hand.

12. The UI control system of claim 10, wherein the pointer corresponds to a single point in the UI.

13. A user interface, UI, control system, comprising:

landmark logic to determine a location of a landmark associated with a portion of a body of a user of a computing system;

pointer logic, coupled with the landmark logic, to determine a pointer based at least in part on a relative position of the landmark location with respect to a virtual interaction region; and

identification logic, coupled with the pointer logic, to identify a UI element from a plurality of UI elements of a UI of the computing system, based at least in part on the pointer; and

display logic, coupled with the pointer logic, to provide a visual indicator of the pointer in a display of the computing system.

14. The UI control system of claim 13, wherein the visual indicator is indicative of a distance between the landmark location and the virtual interaction region, and wherein the visual indicator changes as the distance changes.

15. The UI control system of claim 14, wherein the visual indicator has a size that increases as the distance increases.

16. The UI control system of claim 13, wherein:

the landmark logic is to determine a location of a landmark through determination of a location of a first landmark proximate to a tip of an index finger of the user'"'"'s hand and determine a location of a second landmark between the first landmark and a base joint of the index finger of the user'"'"'s hand;

the pointer logic is to determine the pointer based on a relative position of the location of the first landmark with respect to the virtual interaction region when the first landmark is positioned on a first side of the virtual interaction region; and

the pointer logic is to determine the pointer based on a relative position of the location of the second landmark with respect to the virtual interaction region when the first landmark is positioned on a second side of the virtual interaction region different from the first side.

17-30. -30. (canceled)

31. One or more computer readable media comprising computer readable instructions which, when executed by one or more processing devices of a computing system, cause the computing system to:

determine locations of a plurality of landmarks associated with a portion of a body of a user of the computing system;

determine a pointer based at least in part on the plurality of landmark locations;

identify a UI element from a plurality of UI elements of a UI of the computing system, based at least in part on the pointer; and

provide a visual indicator of the pointer in a display of the computing system.

32. The one or more computer readable media of claim 31, wherein the locations of the plurality of landmarks on the portion of the user'"'"'s body are determined based on an image of the user'"'"'s body captured by a depth camera.

33. The one or more computer readable media of claim 31, further comprising computer readable instructions which, when executed by the one or more processing devices of the computing system, cause the computing system to:

increase the size of the UI element in a display of the computing system, in response to identification of the UI element.

34. The one or more computer readable media of claim 31, further comprising computer readable instructions which, when executed by the one or more processing devices of the computing system, cause the computing system to:

select the UI element, subsequent to identification of the UI element, based at least in part on gesture data indicative of a gesture of the user.

35. The one or more computer readable media of claim 34, wherein the gesture data indicative of the gesture of the user includes first gesture data indicative of a hover over the UI element for a predetermined period of time, and further comprising computer readable instructions which, when executed by the one or more processing devices of the computing system, cause the computing system to:

display a confirmation UI element based on the first gesture data;

receive second gesture data indicative of a selection of the confirmation UI element subsequent to display of the confirmation UI element; and

select the UI element based on the second gesture data.

36. The one or more computer readable media of claim 31, wherein identify a UI element from a plurality of UI elements comprises:

identify a plurality of areas in the UI corresponding to respective ones of the plurality of UI elements; and

identify an amount of overlap between the area corresponding to the UI element and the pointer.

37. The one or more computer readable media of claim 36, wherein the amount of overlap between the area corresponding to the UI element and the pointer is based on a weight of a landmark.

38. The one or more computer readable media of claim 37, wherein the weight is based on a distance between a location of the landmark and a reference location on the portion of the user'"'"'s body.