Gesture-based Text Identification and Selection in Images

• US 20120131520A1
• Filed: 01/30/2012
• Published: 05/24/2012
• Est. Priority Date: 05/14/2009
• Status: Abandoned Application

• Current Assignee: ABBYY Production LLC (ABBYY Software)
• Original Assignee: ABBYY Production LLC (ABBYY Software)
• Inventors: Tang, Ding-Yuan, Budelli, Joey G.
• Application Number:
  US13/361,713
• Publication Number:
  US 20120131520A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 10/235 based on user input or interaction

Abstract

A device with a touch-sensitive screen supports tapping gestures for identifying, selecting or working with initially unrecognized text. A single tap gesture can cause a portion of a character string to be selected. A double tap gesture can cause the entire character string to be selected. A tap and hold gesture can cause the device to enter a cursor mode wherein a placement of a cursor relative to the characters in a character string can be adjusted. In a text selection mode, a finger can be used to move the cursor from a cursor start position to a cursor end position and to select text between the positions. Selected or identified text can populate fields, control the device, etc. Recognition of text can be performed upon access of an image or upon the device detecting a tapping gesture in association with display of the image on the screen.

Claims

1. A method comprising:

acquiring an image of text;

performing an identification function by a processor on the image of text;

displaying a representation of at least a portion of the acquired image of text on a touch-sensitive screen;

detecting a finger tapping gesture on the touch-sensitive screen on or adjacent a location of a portion of text of the image of text;

identifying characters based at least in part on the detected finger tapping gesture; and

performing a further processing based at least in part on the identified characters.

2. The method of claim 1 wherein said performing the further processing based at least in part on the identified characters includes performing processing based at least in part on the detected identified finger tapping gesture.

3. The method of claim 1 wherein the identification function includes one or more optical character recognition (OCR) functions performed on the text of the image of text.

4. The method of claim 1 wherein said identifying characters based at least in part on the detected finger tapping gesture includes:

performing one or more optical character recognition (OCR) functions on text in at least a portion of the image of text.

5. The method of claim 1 wherein said performing an identification function by a processor on the image of text includes:

identifying one or more regions of the image of text that likely include text.

6. The method of claim 1 wherein said performing the identification function by the processor on the image of text includes:

performing a recognition function to identify characters in the image of text.

7. The method of claim 6 wherein the recognition function to identify characters in the image of text recognizes groups of characters.

8. The method of claim 1 wherein said performing the identification function by the processor on the image of text is performed prior to the touch-sensitive screen being capable of interpreting a finger tapping gesture associated with said image of text.

9. The method of claim 1 wherein the method further comprises:

performing an optical character recognition (OCR) operation of text in the displayed representation of the at least portion of the acquired image of text on the touch-sensitive screen.

10. The method of claim 1 wherein said further processing based at least in part on the identified characters includes performing a processing based at least in part upon the meaning of one or more words of the identified characters.

detecting a finger tapping gesture on the touch-sensitive screen on or adjacent a location of a portion of text of the image of text;

11. The method of claim 1, wherein the finger tapping gesture comprises a single tap gesture, and wherein said identifying characters based at least in part on the detected finger tapping gesture includes:

selecting a word from a line of text of the representation of the at least the portion of the acquired image of text displayed on the touch-sensitive screen based on a proximity of the single tap gesture to the word in the line of the identified characters.

12. The method of claim 11, wherein said selecting the word from the line of text further comprises:

displaying the recognized text in a text box that is laterally offset from a line of recognized text.

13. The method of claim 1, wherein the finger tapping gesture comprises a double tap gesture, and wherein said identifying characters based at least in part on the detected finger tapping gesture includes:

selecting a line of characters based on a proximity of the double tap gesture to the line of the identified characters.

14. The method of claim 1, wherein the finger tapping gesture comprises a tap and hold gesture wherein a tap to the touch-sensitive screen includes maintaining contact with said touch-sensitive screen.

15. The method of claim 14, wherein the method further comprises:

responsive to said tap and hold gesture, entering a cursor mode in which maintaining contact with the touch-sensitive screen includes sliding a distance on said touch-sensitive screen, and wherein said sliding causes sympathetic movement of a cursor.

16. The method of claim 15, wherein the method further comprises:

entering a text selection mode upon release of contact with the touch-sensitive screen.

17. The method of claim 16, wherein while in said text selection mode, said sliding causes movement of the cursor from a cursor start position to a cursor end position and any characters between the cursor start position and the cursor end position are identified.

18. The method of claim 1, wherein said identifying characters based at least in part on the detected finger tapping gesture includes:

generating metadata associated with said portion of text associated with said detected finger tapping gesture.

19. The method of claim 1, wherein said performing the further processing based at least in part on the identified characters includes interpreting the identified characters based on their formatting.

20. The method of claim 1, wherein said performing the further processing based at least in part on the identified characters includes:

recognizing said identified characters; and

populating a field using said identified and recognized characters.

21. The method of claim 20, wherein said populating the field using said identified and recognized characters includes populating a field of a user interface.

22. The method of claim 1, wherein said performing the further processing based at least in part on the identified characters includes:

populating a field using metadata associated with said identified characters.

23. A device for facilitating identification of characters based upon a gesture to a touch sensitive display, the device comprising:

a processor;

the touch sensitive display coupled to the processor; and

a memory coupled to the processor, wherein the memory is capable of storing instructions which when executed cause the device to perform a method, the instructions comprising;

acquiring an image that includes an unrecognized character string;

performing by the processor a recognition function on the image that includes the unrecognized character string;

displaying a representation of said image that includes the unrecognized character string on the touch sensitive display;

detecting a tapping gesture adjacent or on a region of the representation of said image on or near a region that includes said character string;

selecting characters of said character string based on said tapping gesture; and

performing a further processing based on the selected characters of said character string.

24. The device of claim 23, wherein performing said recognition function includes recognizing all characters of the unrecognized character string.

25. The device of claim 23, wherein performing said recognition function includes recognizing all unrecognized text of the image.

26. A computer-readable medium having stored thereon instructions, which when executed by a processing system, cause the processing system to perform a method for interacting with text in an image, comprising:

accessing an image;

determining whether said image includes unrecognized text;

when said image includes unrecognized text, performing by the processing system an identification function on at least some of the unrecognized text of the image;

displaying a representation of at least a portion of the image on a touch-enabled portion of said processing system;

detecting a touch gesture on said touch-enabled portion of said processing system;

identifying a portion of text based at least in part on the touch gesture; and

performing by the processing system a further processing of the identified portion of text in response to the touch gesture.

27. The computer-readable medium of claim 26, wherein parts of said identified portion of text are collinear.

28. The computer-readable medium of claim 26, wherein said accessing the image includes accessing an image captured by processing system operating in a video capture mode.

29. The computer-readable medium of claim 26, wherein said accessing the image includes accessing an image that is stored in a persistent medium.