Context based gesture delineation for user interaction in eyes-free mode

• US 9,971,495 B2
• Filed: 01/28/2013
• Issued: 05/15/2018
• Est. Priority Date: 01/28/2013
• Status: Active Grant

• Current Assignee: Nook Digital, LLC (f/k/a Barnesandnoble.com LLC) (Bahwan CyberTek Pvt Ltd.)
• Original Assignee: Nook Digital, LLC (f/k/a Barnesandnoble.com LLC) (Bahwan CyberTek Pvt Ltd.)
• Inventors: Shetty, Saj, Chattopadhyay, Krishanu, Klein, Douglas
• Primary Examiner(s): Vu, Kieu
• Assistant Examiner(s): Chung, Andrew
• Application Number:
  US13/751,951
• Publication Number:
  US 20140215340A1
• Time in Patent Office: 1,933 Days
• Field of Search: 715727-729, 51727-729
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

Techniques are disclosed for facilitating the use of an electronic device having a user interface that is sensitive to a user'"'"'s gestures. An “eyes-free” mode is provided in which the user can control the device without looking at the device display. Once the eyes-free mode is engaged, the user can control the device by performing gestures that are detected by the device, wherein a gesture is interpreted by the device without regard to a specific location where the gesture is made. The eyes-free mode can be used, for example, to look up a dictionary definition of a word in an e-book or to navigate through and select options from a hierarchical menu of settings on a tablet. The eyes-free mode advantageously allows a user to interact with the user interface in situations where the user has little or no ability to establish concentrated visual contact with the device display.

Claims

1. A device comprising:

a touch sensitive display for detecting gestures made by a user; and

a user interface including

(a) a reading overlay in which digital content is delivered to the user aurally, and in which the user interface is configured to respond to a forward transition gesture,

(b) an options overlay which is displayed in response to detection of the forward transition gesture in the reading overlay, and in which

(i) aural delivery of the digital content is paused,

(ii) a plurality of command options are arranged in a vertically-oriented list displayed on the touch sensitive display,

(iii) the options overlay responds to a vertical swipe menu navigation gesture that does not touch the vertically-oriented list and that identifies one of the command options by generating an audible announcement of the identified command option, wherein the vertical swipe menu navigation gesture must be performed multiple times to scroll through multiple items listed in the vertically-oriented list, and

(iv) the options overlay responds to the forward transition gesture by invoking a command option that was identified by the vertical swipe menu navigation gesture, and

(c) a control overlay which is displayed in response to detection of the forward transition gesture in the options overlay, and which displays a plurality of control options corresponding to the invoked command option,

wherein detection, at a location that is anywhere on the touch sensitive display, of the forward transition gesture in the options overlay results in a response that is independent of the location on the touch sensitive display where the forward transition gesture was detected, the location encompassing all portions of the touch sensitive display that are contacted by the forward transition gesture;

wherein the reading overlay is displayed in response to detection, at a location that is anywhere on the touch sensitive display, of at least one of a single-level backward transition gesture in the options overlay, and a multilevel backward transition gesture in the control overlay;

wherein the options overlay is also displayed in response to detection of the single-level backward transition gesture in the control overlay; and

wherein the same forward transition gesture is used to transition from the reading overlay to the options overlay, and from the options overlay to the control overlay; and

wherein the same single-level backward transition gesture is used to transition from the control overlay to the options overlay, and from the options overlay to the reading overlay.

2. The device of claim 1, wherein, when in the reading overlay, the user interface is further responsive to a reading mode transition gesture that is configured to toggle the user interface back-and-forth between a reading mode and a manual mode.

3. The device of claim 1, further comprising a speaker, wherein in the reading overlay the digital content is delivered to the user via the speaker.

4. The device of claim 1, wherein detection of a content navigation gesture in a manual mode of the reading overlay causes a segment of the digital content to be re-delivered to the user.

5. The device of claim 1, wherein:

detection of a content navigation gesture in a manual mode of the reading overlay causes a segment of the digital content to be re-delivered to the user; and

the re-delivered segment of digital content is selected from a group consisting of a word, a line, a sentence, and a paragraph.

6. The device of claim 1, wherein detection of a content navigation gesture in a manual mode of the reading overlay causes a segment of the digital content to be highlighted on the touch sensitive display.

7. The device of claim 1, further comprising a text-to-speech module, wherein in the reading overlay the digital content is converted to an audio signal using the text-to-speech module.

8. The device of claim 1, wherein:

the device is selected from a group consisting of an e-reader, a tablet computer and a smart phone.

9. A mobile electronic device comprising:

a touch sensitive display for displaying digital content and detecting gestures made by a user;

a speaker;

a text-to-speech module; and

a user interface including

(a) a reading overlay in which the text-to-speech module converts the displayed digital content into an audio signal that is played using the speaker, and in which the user interface is configured to respond to a forward transition gesture,

(b) an options overlay which is displayed in response to detection of the forward transition gesture in the reading overlay, and in which

(i) playing of the audio signal generated by the text-to-speech module is paused,

(ii) a plurality of command options are arranged in a vertically-oriented list displayed on the touch sensitive display,

(iii) the options overlay responds to a vertical swipe menu navigation gesture that does not touch the vertically-oriented list and that identifies one of the command options by generating a first audible announcement of the identified command option, wherein the vertical swipe menu navigation gesture must be repeated multiple times to scroll through multiple items listed in the vertically-oriented list, and

(iv) the options overlay responds to the forward transition gesture by invoking a command option that was identified by the vertical swipe menu navigation gesture, and

(c) a control overlay which is displayed in response to detection of the forward transition gesture in the options overlay, and which displays a plurality of control options corresponding to the invoked command option,

wherein detection, at a location that is anywhere on the touch sensitive display, of the forward transition gesture in the options overlay results in a response that is independent of the location on the touch sensitive display where the forward transition gesture was detected, the location encompassing all portions of the touch sensitive display that are contacted by the forward transition gesture;

wherein the reading overlay is displayed in response to detection, at a location that is anywhere on the touch sensitive display, of at least one of a single-level backward transition gesture in the options overlay, and a multilevel backward transition gesture in the control overlay;

wherein the options overlay is also displayed in response to detection of the single-level backward transition gesture in the control overlay; and

wherein the same forward transition gesture is used to transition from the reading overlay to the options overlay, and from the options overlay to the control overlay; and

wherein the same single-level backward transition gesture is used to transition from the control overlay to the options overlay, and from the options overlay to the reading overlay.

10. The mobile electronic device of claim 9, wherein the options overlay further responds to the forward transition gesture by using the speaker to play a second audible announcement of the invoked command option.

11. The mobile electronic device of claim 9, wherein detection of a content navigation gesture in a manual mode of the reading overlay causes a segment of the digital content to be read using the text-to-speech module and the speaker.

12. The mobile electronic device of claim 9, wherein detection of a content navigation gesture in a manual mode of the reading overlay causes a segment of the digital content to be copied to a virtual clipboard.

13. A non-transitory computer readable medium encoded with instructions that, when executed by at least one processor, cause an eyes-free control process to be carried out, the control process comprising:

providing a touch sensitive display that includes a user interface having

(a) a reading overlay in which an aural presentation of digital content is made, and in which the user interface is configured to respond to a forward transition gesture,

(b) an options overlay which is displayed in response to detection of the forward transition gesture in the reading overlay, and in which

(i) the aural presentation of the digital content is paused,

(ii) a plurality of command options are arranged in a vertically-oriented list displayed on the touch sensitive display,

(iii) the options overlay responds to a vertical swipe menu navigation gesture that does not touch the vertically-oriented list and that identifies one of the command options by generating a first audible announcement of the identified command option, wherein the vertical swipe menu navigation gesture must be performed multiple times to scroll through multiple items listed in the vertically-oriented list, and

(iv) the options overlay responds to the forward transition gesture by invoking a command option that was identified by the vertical swipe menu navigation gesture, and

(c) a control overlay which is displayed in response to detection of the forward transition gesture in the options overlay, and which displays a plurality of control options corresponding to the invoked command option,

wherein detection, at a location that is anywhere on the touch sensitive display, of the forward transition gesture in the options overlay results in a response that is independent of the location on the touch sensitive display where the forward transition gesture is detected, the location encompassing all portions of the touch sensitive display that are contacted by the forward transition gesture;

wherein the reading overlay is displayed in response to detection, at a location that is anywhere on the touch sensitive display, of at least one of a single-level backward transition gesture in the options overlay, and a multilevel backward transition gesture in the control overlay;

wherein the options overlay is also displayed in response to detection of the single-level backward transition gesture in the control overlay; and

wherein the same forward transition gesture is used to transition from the reading overlay to the options overlay, and from the options overlay to the control overlay; and

wherein the same single-level backward transition gesture is used to transition from the control overlay to the options overlay, and from the options overlay to the reading overlay.

14. The non-transitory computer readable medium of claim 13, wherein the options overlay further responds to the forward transition gesture by playing a second audible announcement of the invoked command option.