Resolution of directional ambiguity on touch-based interface gesture

• US 9,857,965 B1
• Filed: 01/06/2012
• Issued: 01/02/2018
• Est. Priority Date: 01/06/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Starner, Thad Eugene, Patel, Nirmal, Wong, Adrian, Johnson, Michael Patrick
• Primary Examiner(s): Awad, Amr
• Assistant Examiner(s): LUI, DONNA V
• Application Number:
  US13/344,643
• Time in Patent Office: 2,188 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2203/0339 Touch strips, e.g. orthogonal touch strips to control cursor movement or scrolling; single touch strip to adjust parameter or to implement a row of soft keys
• CPC Class Codes: G06F 2203/0383 Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Methods and systems involving resolution of directional ambiguity between a graphical display and a touch-based user-interface are disclosed herein. An example system may be configured to: (a) cause a visual depiction of a first reference marker on a graphical display; (b) receive first input data indicating an initial touch input on a touch-based user-interface, where the initial touch input corresponds to an input-direction path having a first end and a second end, and where the touch input corresponds to movement of the input-direction path; (c) associate movement of the first reference marker with subsequent touch inputs; (d) receive second input data indicating a subsequent touch input; and (e) cause a visual depiction of movement of a second reference marker.

Claims

1. A method comprising:

displaying on a graphical display of a wearable computing device, a first reference marker and indication to move the first reference marker in a predetermined direction on the graphical display to wake up the wearable computing device;

while the first reference marker is displayed, receiving first input data indicating an initial swipe gesture on a touch-based user-interface that is separate from and non-parallel to the graphical display;

in response to the receipt of the first input data indicating the initial swipe gesture;

(a) regardless of a direction of the initial swipe gesture on the touch-based user-interface, moving the first reference marker in the predetermined direction on the graphical display and waking up the wearable computing device;

(b) determining whether the initial swipe gesture is performed in either;

(i) a first direction along the touch-based user-interface, or (ii) a second direction along the touch-based user-interface, wherein the second direction is substantially opposite of the first direction;

(c) if the initial swipe gesture is performed in the first direction on the touch-based user-interface, then setting a touch-association parameter to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display; and

(d) otherwise, if the initial swipe gesture is performed in the second direction on the touch-based user-interface, then setting the touch-association parameter to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display;

receiving second input data indicating a subsequent touch input; and

causing movement of a second reference marker on the graphical display based on at least (i) the association indicated by the touch-association parameter and (ii) a direction of movement of the subsequent touch input on the touch-based user-interface.

2. The method of claim 1, wherein the first reference marker is a graphical object.

3. The method of claim 1, further comprising:

before receiving the initial swipe gesture, operating in a first mode in which no association exists between the movement of the first reference marker and the initial swipe gesture on the touch-based user-interface; and

after receiving the swipe gesture, operating in a second mode in which the touch-based user-interface is configured to receive second input data indicating subsequent touch inputs.

4. The method of claim 3, wherein the first mode is a locked mode in which the touch-based user-interface is configured to receive only the first input data indicating the initial touch input.

5. The method of claim 3, further comprising:

while operating in the second mode, detecting a transition event; and

responsive to detecting the transition event, transitioning to operating in the first mode.

6. The method of claim 5, wherein detecting the transition event comprises determining that a timer of the touch-based user-interface has expired.

7. The method of claim 1, carried out on a head-mounted display (HMD).

8. The method of claim 7, wherein the touch-based user-interface is attached to the HMD.

9. The method of claim 8, wherein the HMD comprises at least one lens element, and wherein the graphical display is integrated into the at least one lens element.

10. The method of claim 9, wherein the head-mounted display further comprises at least one extending side-arm, and wherein the touch-based user-interface is attached to one of the at least one extending side-arm.

11. The method of claim 9, wherein the graphical display is arranged substantially orthogonally to the touch-based user-interface.

12. A wearable head-mounted display (HMD) comprising:

means for displaying on a graphical display of a wearable computing device, a first reference marker and indication to move the first reference marker in a predetermined direction on the graphical display to wake up the wearable computing device;

means for while the first reference marker is displayed, receiving first input data indicating an initial swipe gesture on a touch-based user-interface that is separate from and non-parallel to the graphical display;

means for, in response to the receipt of the first input data indicating the initial swipe gesture;

(a) regardless of a direction of the initial swipe gesture on the touch-based user-interface, moving the first reference marker in the predetermined direction on the graphical display and waking up the wearable computing device;

(b) determining whether the initial swipe gesture is performed in either;

(i) a first direction along the touch-based user-interface, or (ii) a second direction along the touch-based user-interface, wherein the second direction is substantially opposite of the first direction;

(c) if the initial swipe gesture is performed in the first direction on the touch-based user-interface, then setting a touch-association parameter to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display; and

(d) otherwise, if the initial swipe gesture is performed in the second direction on the touch-based user-interface, then setting the touch-association parameter to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display;

means for receiving second input data indicating a subsequent touch input; and

means for causing movement of a second reference marker on the graphical display based on at least (i) the association indicated by the touch-association parameter and (ii) a direction of movement of the subsequent touch input on the touch-based user-interface.

13. The HMD of claim 12, wherein the touch-based user-interface is attached to the head-mounted display.

14. The HMD of claim 12, wherein the head-mounted display comprises at least one lens element, and wherein the graphical display is integrated into the at least one lens element.

15. The HMD of claim 14, wherein the head-mounted display further comprises at least one extending side-arm, and wherein the touch-based user-interface is attached to one of the at least one extending side-arm.

16. The HMD of claim 14, wherein the graphical display is arranged substantially orthogonally to the touch-based user-interface on a head-mounted display.

17. A non-transitory computer-readable medium having instructions stored thereon comprising:

displaying on a graphical display of a wearable computing device, a first reference marker and indication to move the first reference marker in a predetermined direction on the graphical display to wake up the wearable computing device;

while the first reference marker is displayed, receiving first input data an initial swipe gesture on a touch-based user-interface that is separate from and non-parallel to the graphical display;

in response to the receipt of the first input data indicating the initial swipe gesture;

(a) regardless of a direction of the initial swipe gesture on the touch-based user-interface, moving the first reference marker in the predetermined direction on the graphical display and waking up the wearable computing device;

(b) determining whether the initial swipe gesture is performed in either;

(i) a first direction along the touch-based user-interface, or (ii) a second direction along the touch-based user-interface, wherein the second direction is substantially opposite of the first direction;

(c) if the initial swipe gesture is performed in the first direction on the touch-based user-interface, then setting a touch-association parameter to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display; and

(d) otherwise, if the initial swipe gesture is performed in the second direction on the touch-based user-interface, then setting the touch-association parameter to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display;

receiving second input data indicating a subsequent touch input; and

causing movement of a second reference marker on the graphical display based on at least (i) the association indicated by the touch-association parameter and (ii) a direction of movement of the subsequent touch input on the touch-based user-interface.

18. The non-transitory computer readable medium of claim 17, wherein the first reference marker is a graphical object.

19. A system comprising:

a processor;

a non-transitory computer readable medium; and

program instructions stored on the non-transitory computer readable medium and executable by the processor to cause a computing device to;

display, on a graphical display of a computing device, a first reference marker and indication to move the first reference marker in a predetermined direction on the graphical display to wake up the computing device;

while the first reference marker is displayed, receive first input data indicating an initial swipe gesture on a touch-based user-interface that is separate from and non-parallel to the graphical display;

in response to the receipt of the first input data indicating the initial swipe gesture;

(a) regardless of a direction of the initial swipe gesture on the touch-based user-interface, move the first reference marker in the predetermined direction on the graphical display and wake up the wearable computing device;

(b) determine whether the initial swipe gesture is performed in either;

(i) a first direction along the touch-based user-interface, or (ii) a second direction along the touch-based user-interface, wherein the second direction is substantially opposite of the first direction;

(c) if the initial swipe gesture is performed in the first direction on the touch-based user-interface, then set a touch-association parameter to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display; and

(d) otherwise, if the initial swipe gesture is performed in the second direction on the touch-based user-interface, then set the touch-association parameter to associate subsequent touch input in the second direction on the touch-based user-interface with movement of graphics in the predetermined direction on the graphical display and to associate subsequent touch input in the first direction on the touch-based user-interface with movement of graphics in a direction opposite of the predetermined direction on the graphical display;

receive second input data indicating a subsequent touch input; and

cause movement of a second reference marker on the graphical display based on at least (i) the association indicated by the touch-association parameter and (ii) a direction of movement of the subsequent touch input on the touch-based user-interface.

20. The system of claim 19, wherein the system further comprises a head-mounted display (HMD).

21. The system of claim 20, wherein the touch-based user-interface is attached to the HMD.

22. The system of claim 21, wherein the HMD further comprises at least one lens element, and wherein the graphical display is integrated into the at least one lens element.

23. The system of claim 21, wherein the head-mounted display further comprises at least one extending side-arm, and wherein the touch-based user-interface is attached to one of the at least one extending side-arm.

24. The system of claim 21, wherein the graphical display is arranged substantially orthogonally to the touch-based user-interface.