Device, method, and graphical user interface for integrating recognition of handwriting gestures with a screen reader

• US 8,881,269 B2
• Filed: 12/10/2012
• Issued: 11/04/2014
• Est. Priority Date: 03/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Fleizach, Christopher B., Minifie, Darren C., Hughes, Gregory F., Dour, Ryan N., Fisch, Ian M., Lopes Da Silva, Joel M., Pedersen, Michael M. II, Seymour, Eric T., Naik, Devang K., Dixon, Ryan S.
• Primary Examiner(s): Najjar, Saleh
• Assistant Examiner(s): Korsak, Oleg
• Application Number:
  US13/710,385
• Publication Number:
  US 20130263251A1
• Time in Patent Office: 694 Days
• Field of Search: 726/19, 345/175
• US Class Current: 726/19
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

While an electronic device with a display and a touch-sensitive surface is in a screen reader accessibility mode, the device displays an application launcher screen including a plurality of application icons. A respective application icon corresponds to a respective application stored in the device. The device detects a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters. A respective gesture that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character. The device determines whether the detected sequence of one or more gestures corresponds to a respective application icon of the plurality of application icons, and, in response to determining that the detected sequence of one or more gestures corresponds to the respective application icon, performs a predefined operation associated with the respective application icon.

Claims

1. A method, comprising:

while an electronic device with a display and a touch-sensitive surface is in a locked, passcode-protected state with a screen reader accessibility mode turned on;

displaying a passcode screen for receiving a passcode on the display, the passcode including a predefined sequence of characters, the passcode screen including a plurality of character icons;

detecting a sequence of gestures on the touch-sensitive surface that correspond to characters, wherein;

a respective gesture of the sequence of gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character;

determining whether the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

in response to determining that the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

unlocking the electronic device; and

maintaining the electronic device in the screen reader accessibility mode; and

,

in response to determining that the detected sequence of gestures does not correspond to the predefined sequence of characters in the passcode;

maintaining the electronic device in the locked, passcode-protected state with the screen reader accessibility mode turned on.

2. The method of claim 1, wherein:

the respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more character icons of the plurality of character icons without activating the one or more character icons.

3. The method of claim 1, wherein the respective path is in the shape of the respective character.

4. The method of claim 1, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a character icon of the plurality of character icons associated with the respective character.

5. The method of claim 1, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to any of the plurality of character icons.

6. The method of claim 1, including:

in response to determining that the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

ceasing to display the passcode screen.

7. The method of claim 1, including:

while detecting the sequence of gestures on the touch-sensitive surface that correspond to characters;

foregoing outputting audible accessibility information that corresponds to the characters.

8. The method of claim 1, including:

in response to determining that the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

ceasing to display the passcode screen; and

displaying an application launcher screen that includes a plurality of application icons.

9. The method of claim 8, including:

while displaying the passcode screen, foregoing outputting audible accessibility information associated with a respective character when the respective character is entered; and

while the device is unlocked and in the screen reader accessibility mode;

detecting a single finger gesture on the touch-sensitive surface at a location that corresponds to a respective application icon of the application launcher screen; and

,

in response to detecting the single finger gesture, outputting audible accessibility information associated with the respective application icon.

10. The method of claim 1, including:

while the electronic device is in the locked, passcode-protected state with the screen reader accessibility mode turned on, prior to detecting the sequence of gestures on the touch-sensitive surface that correspond to characters in the passcode;

detecting a predefined input; and

,

in response to detecting the predefined input, entering a handwriting gesture recognition mode within the screen reader accessibility mode.

11. The method of claim 1, including:

while the electronic device is in the locked, passcode-protected state with the screen reader accessibility mode turned on;

detecting a single finger gesture on the touch-sensitive surface at a location that corresponds to a character icon of the plurality of character icons; and

,

in response to detecting the single finger gesture on the touch-sensitive surface at the location that corresponds to the character icon;

entering a respective character that corresponds to the character icon in a received passcode; and

foregoing outputting audible accessibility information associated with the respective character.

12. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

while the electronic device is in a locked, passcode-protected state with a screen reader accessibility mode turned on;

displaying a passcode screen for receiving a passcode on the display, the passcode including a predefined sequence of characters, the passcode screen including a plurality of character icons;

detecting a sequence of gestures on the touch-sensitive surface that correspond to characters, wherein;

a respective gesture of the sequence of gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character;

determining whether the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

in response to determining that the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

unlocking the electronic device; and

maintaining the electronic device in the screen reader accessibility mode; and

,

in response to determining that the detected sequence of gestures does not correspond to the predefined sequence of characters in the passcode;

maintaining the electronic device in the locked, passcode-protected state with the screen reader accessibility mode turned on.

13. The electronic device of claim 12, wherein:

the respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more character icons of the plurality of character icons without activating the one or more character icons.

14. The electronic device of claim 12, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a character icon of the plurality of character icons associated with the respective character.

15. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the device to:

while the electronic device is in a locked, passcode-protected state with a screen reader accessibility mode turned on;

display a passcode screen for receiving a passcode on the display, the passcode including a predefined sequence of characters, the passcode screen including a plurality of character icons;

detect a sequence of gestures on the touch-sensitive surface that correspond to characters, wherein;

a respective gesture of the sequence of gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character;

determine whether the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

in response to determining that the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

unlock the electronic device; and

maintain the electronic device in the screen reader accessibility mode; and

,

in response to determining that the detected sequence of gestures does not correspond to the predefined sequence of characters in the passcode;

maintain the electronic device in the locked, passcode-protected state with the screen reader accessibility mode turned on.

16. The computer readable storage medium of claim 15, wherein:

the respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more character icons of the plurality of character icons without activating the one or more character icons.

17. The computer readable storage medium of claim 15, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a character icon of the plurality of character icons associated with the respective character.

18. A graphical user interface on an electronic device with a display and a touch-sensitive surface, a memory, and one or more processors to execute one or more programs stored in the memory, the graphical user interface comprising:

a passcode screen for receiving a passcode displayed on the display while the electronic device is in a locked, passcode-protected state with a screen reader accessibility mode turned on, the passcode including a predefined sequence of characters, the passcode screen including a plurality of character icons;

wherein;

a sequence of gestures that correspond to characters is detected on the touch-sensitive surface, wherein;

a respective gesture of the sequence of gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character;

whether the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode is determined;

in response to determining that the detected sequence of gestures corresponds to the predefined sequence of characters in the passcode;

the electronic device is unlocked; and

the electronic device is maintained in the screen reader accessibility mode; and

,

in response to determining that the detected sequence of gestures does not correspond to the predefined sequence of characters in the passcode;

the electronic device is maintained in the locked, passcode-protected state with the screen reader accessibility mode turned on.

19. The graphical user interface of claim 18, wherein:

the respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more character icons of the plurality of character icons without activating the one or more character icons.

20. The graphical user interface of claim 18, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a character icon of the plurality of character icons associated with the respective character.