Device, method, and graphical user interface for integrating recognition of handwriting gestures with a screen reader

• US 10,013,162 B2
• Filed: 03/16/2017
• Issued: 07/03/2018
• Est. Priority Date: 03/31/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Fleizach, Christopher B., Minifie, Darren C., Hughes, Gregory F., Dour, Ryan N., Fisch, Ian M., Lopes Da Silva, Joel M., Pedersen, II, Michael M., Seymour, Eric T., Naik, Devang K., Dixon, Ryan S.
• Primary Examiner(s): Korsak, Oleg
• Application Number:
  US15/460,857
• Publication Number:
  US 20170185285A1
• Time in Patent Office: 474 Days
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

While an electronic device with a display and a touch-sensitive surface is in a screen reader accessibility mode, the device displays a character input area and a keyboard, the keyboard including a plurality of key icons. The device detects a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters. A respective gesture of the one or more gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character. The respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons. In response to detecting the respective gesture, the device enters the corresponding respective character in the character input area of the display.

Claims

1. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

while the electronic device is in a screen reader accessibility mode;

displaying a character input area and a keyboard on the display, the keyboard including a plurality of key icons;

detecting a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters, wherein;

a respective gesture of the one or more gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character, and

the respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the respective gesture, entering the corresponding respective character in the character input area of the display.

2. The electronic device of claim 1, wherein the respective path is in the shape of the respective character.

3. The electronic device of claim 1, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a key icon of the plurality of key icons associated with the respective character.

4. The electronic device of claim 1, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to any of the plurality of key icons.

5. The electronic device of claim 1, wherein the one or more programs further include instructions for:

in response to detecting the respective gesture, outputting audible accessibility information about the corresponding respective character.

6. The electronic device of claim 1, wherein the one or more programs further include instructions for:

detecting a first predefined finger gesture on the touch-sensitive surface, wherein the first predefined finger gesture traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the first predefined finger gesture on the touch-sensitive surface;

entering a space in the character input area at a current location of a cursor, and

outputting audible accessibility information indicating that a space has been entered.

7. The electronic device of claim 6, wherein the first predefined finger gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a space key icon in the plurality of key icons.

8. The electronic device of claim 1, wherein the one or more programs further include instructions for:

detecting a second predefined finger gesture on the touch-sensitive surface, wherein the second predefined finger gesture traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the second predefined finger gesture on the touch-sensitive surface;

deleting a character in the character input area at a current location of a cursor, and

outputting audible accessibility information indicating that a character has been deleted.

9. The electronic device of claim 8, wherein the second predefined finger gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a backspace key icon in the plurality of key icons.

10. The electronic device of claim 1, wherein the one or more programs further include instructions for:

while the electronic device is in the screen reader accessibility mode, prior to detecting a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters;

detecting a predefined input; and

in response to detecting the predefined input, entering a handwriting gesture recognition mode within the screen reader accessibility mode.

11. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the electronic device to:

while the electronic device is in a screen reader accessibility mode;

display a character input area and a keyboard on the display, the keyboard including a plurality of key icons;

detect a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters, wherein;

a respective gesture of the one or more gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character, and

the respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the respective gesture, enter the corresponding respective character in the character input area of the display.

12. The computer readable storage medium of claim 11, wherein the respective path is in the shape of the respective character.

13. The computer readable storage medium of claim 11, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a key icon of the plurality of key icons associated with the respective character.

14. The computer readable storage medium of claim 11, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to any of the plurality of key icons.

15. The computer readable storage medium of claim 11, wherein the one or more programs further comprise instructions that cause the electronic device to:

in response to detecting the respective gesture, output audible accessibility information about the corresponding respective character.

16. The computer readable storage medium of claim 11, wherein the one or more programs further comprise instructions that cause the electronic device to:

detect a first predefined finger gesture on the touch-sensitive surface, wherein the first predefined finger gesture traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the first predefined finger gesture on the touch-sensitive surface;

enter a space in the character input area at a current location of a cursor, and

output audible accessibility information indicating that a space has been entered.

17. The computer readable storage medium of claim 16, wherein the first predefined finger gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a space key icon in the plurality of key icons.

18. The computer readable storage medium of claim 11, wherein the one or more programs further comprise instructions that cause the electronic device to:

detect a second predefined finger gesture on the touch-sensitive surface, wherein the second predefined finger gesture traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the second predefined finger gesture on the touch-sensitive surface;

delete a character in the character input area at a current location of a cursor, and

output audible accessibility information indicating that a character has been deleted.

19. The computer readable storage medium of claim 18, wherein the second predefined finger gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a backspace key icon in the plurality of key icons.

20. The computer readable storage medium of claim 11, wherein the one or more programs further comprise instructions that cause the electronic device to:

while the electronic device is in the screen reader accessibility mode, prior to detecting a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters;

detect a predefined input; and

in response to detecting the predefined input, enter a handwriting gesture recognition mode within the screen reader accessibility mode.

21. A method, comprising:

while an electronic device with a display and a touch-sensitive surface is in a screen reader accessibility mode;

displaying a character input area and a keyboard on the display, the keyboard including a plurality of key icons;

detecting a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters, wherein;

a respective gesture of the one or more gestures that corresponds to a respective character is a single finger gesture that moves across the touch-sensitive surface along a respective path that corresponds to the respective character, and

the respective path traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the respective gesture, entering the corresponding respective character in the character input area of the display.

22. The method of claim 21, wherein the respective path is in the shape of the respective character.

23. The method of claim 21, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a key icon of the plurality of key icons associated with the respective character.

24. The method of claim 21, wherein the respective gesture is independent of contacting a location on the touch-sensitive surface that corresponds to any of the plurality of key icons.

25. The method of claim 21, further comprising:

in response to detecting the respective gesture, outputting audible accessibility information about the corresponding respective character.

26. The method of claim 21, further comprising:

detecting a first predefined finger gesture on the touch-sensitive surface, wherein the first predefined finger gesture traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the first predefined finger gesture on the touch-sensitive surface;

entering a space in the character input area at a current location of a cursor, and

outputting audible accessibility information indicating that a space has been entered.

27. The method of claim 26, wherein the first predefined finger gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a space key icon in the plurality of key icons.

28. The method of claim 21, further comprising:

detecting a second predefined finger gesture on the touch-sensitive surface, wherein the second predefined finger gesture traverses one or more locations on the touch-sensitive surface that correspond to one or more key icons of the plurality of key icons without activating the one or more key icons; and

in response to detecting the second predefined finger gesture on the touch-sensitive surface;

deleting a character in the character input area at a current location of a cursor, and

outputting audible accessibility information indicating that a character has been deleted.

29. The method of claim 28, wherein the second predefined finger gesture is independent of contacting a location on the touch-sensitive surface that corresponds to a backspace key icon in the plurality of key icons.

30. The method of claim 21, further comprising:

while the electronic device is in the screen reader accessibility mode, prior to detecting a sequence of one or more gestures on the touch-sensitive surface that correspond to one or more characters;

detecting a predefined input; and

in response to detecting the predefined input, entering a handwriting gesture recognition mode within the screen reader accessibility mode.