Gesture-based selection text recognition

• US 8,666,199 B2
• Filed: 07/26/2013
• Issued: 03/04/2014
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lee, Dar-Shyang, Chien, Lee-Feng, Ting, Pin, Hsieh, Aries, Wong, Kin
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/952,457
• Publication Number:
  US 20130335356A1
• Time in Patent Office: 221 Days
• US Class Current: 382/312
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1456 based on user interactions
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

An image is displayed on a touch screen. A user'"'"'s underline gesture on the displayed image is detected. The area of the image touched by the underline gesture and a surrounding region approximate to the touched area are identified. Skew for text in the surrounding region is determined and compensated. A text region including the text is identified in the surrounding region and cropped from the image. The cropped image is transmitted to an optical character recognition (OCR) engine, which processes the cropped image and returns OCR'"'"'ed text. The OCR'"'"'ed text is outputted.

Claims

1. A computer-implemented method for selectively recognizing text in an image, comprising:

displaying the image on a touch sensitive display device;

detecting a gesture made on the touch sensitive display device;

identifying an area of the image associated with the gesture;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

calculating a plurality of projection profiles of a plurality of angles for at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles;

determining a skew angle based on an angle of the horizontal projection profile;

correcting the skew angle by rotating the area of the image; and

identifying a text region in the rotated area; and

recognizing text in the text region using OCR technology.

2. The computer-implemented method of claim 1, wherein identifying the text region further comprises identifying the text region responsive to the gesture made on the touch sensitive display.

3. The computer-implemented method of claim 1, further comprising:

cropping the image to form a cropped image including the text region;

wherein recognizing the text comprises;

transmitting the cropped image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

4. The computer-implemented method of claim 1, wherein recognizing the text comprises:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

5. The computer-implemented method of claim 1, wherein identifying the area of the image associated with the gesture comprises detecting an underline gesture on the touch sensitive display device.

6. The computer-implemented method of claim 5, wherein the underline gesture is vertical relative to an orientation of the image and wherein the text in the text region comprises text displayed vertically.

7. The computer-implemented method of claim 1, wherein identifying the area of the image comprises detecting a circling gesture on the touch sensitive display device.

8. A non-transitory computer-readable storage medium encoded with executable computer program code for selectively recognizing text in an image, the computer program code comprising program code for:

displaying the image on a touch sensitive display device;

detecting a gesture made on the touch sensitive display device;

identifying an area of the image associated with the gesture;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

calculating a plurality of projection profiles of a plurality of angles for at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles;

determining a skew angle based on an angle of the horizontal projection profile;

correcting the skew angle by rotating the area of the image; and

identifying a text region in the rotated area; and

recognizing text in the text region using OCR technology.

9. The non-transitory computer-readable storage medium of claim 8, wherein identifying the text region further comprises identifying the text region responsive to the gesture made on the touch sensitive display.

10. The non-transitory computer-readable storage medium of claim 8, wherein the computer program code further comprises program code for:

cropping the image to form a cropped image including the text region;

wherein recognizing the text comprises;

transmitting the cropped image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

11. The non-transitory computer-readable storage medium of claim 8, wherein recognizing the text comprises:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

12. The non-transitory computer-readable storage medium of claim 8, wherein identifying the area of the image associated with the gesture comprises detecting an underline gesture on the touch sensitive display device.

13. The non-transitory computer-readable storage medium of claim 12, wherein the underline gesture is vertical relative to an orientation of the image and wherein the text in the text region comprises text displayed vertically.

14. The non-transitory computer-readable storage medium of claim 8, wherein identifying the area of the image comprises detecting a circling gesture on the touch sensitive display device.

15. A computer system for selectively recognizing text in an image, comprising:

a computer-readable storage medium comprising executable computer program code for;

an image User Interface (UI) module for displaying the image on a touch sensitive display device and detecting a gesture made on the touch sensitive display device;

a text region identification module for identifying a text region in the image displayed on the touch sensitive display device, the text region comprising text, wherein identifying the text in the image comprises;

identifying an area of the image associated with the gesture;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

calculating a plurality of projection profiles of a plurality of angles for at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles;

determining a skew angle based on an angle of the horizontal projection profile;

correcting the skew angle by rotating the area of the image; and

identifying the text region in the rotated area; and

an OCR module for recognizing text in the text region using OCR technology.

16. The computer system of claim 15, wherein the text region identification module configured for identifying the text region further comprises identifying the text region responsive to the gesture made on the touch sensitive display.

17. The computer system of claim 15, wherein the text region identification module is further configured for:

cropping the image to form a cropped image including the text region;

wherein the OCR module is further configured for;

transmitting the cropped image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

18. The computer system of claim 15, wherein the OCR module is further configured for:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

19. The computer system of claim 15, wherein identifying the area of the image associated with the gesture comprises detecting an underline gesture on the touch sensitive display device.

20. The computer system of claim 19, wherein the underline gesture is vertical relative to an orientation of the image and wherein the text in the text region comprises text displayed vertically.