Gesture-based selective text recognition

• US 8,520,983 B2
• Filed: 10/07/2009
• Issued: 08/27/2013
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chien, Lee-Feng, Ting, Pin, Hsieh, Aries, Wong, Kin, Lee, Dar-Shyang
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/575,015
• Publication Number:
  US 20110081083A1
• Time in Patent Office: 1,420 Days
• Field of Search: 382181-189, 382312-317
• US Class Current: 382/312
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1456 based on user interactions
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

An image is displayed on a touch screen. A user'"'"'s underline gesture on the displayed image is detected. The area of the image touched by the underline gesture and a surrounding region approximate to the touched area are identified. Skew for text in the surrounding region is determined and compensated. A text region including the text is identified in the surrounding region and cropped from the image. The cropped image is transmitted to an optical character recognition (OCR) engine, which processes the cropped image and returns OCR'"'"'ed text. The OCR'"'"'ed text is outputted.

Claims

1. A computer-implemented method for selectively recognizing text in an image, comprising:

displaying the image on a touch sensitive display device;

detecting an underline gesture associated with an area of the touch sensitive display device, the area of the touch sensitive display device associated with the underline gesture being vertical relative to an orientation of the image;

identifying a text region in the image associated with the area of the touch sensitive display device, the text region comprising text displayed vertically relative to the orientation of the image; and

recognizing the text in the text region using OCR technology.

2. The computer-implemented method of claim 1, further comprising:

identifying an area of the image displayed on the area of the touch sensitive display device;

wherein identifying the text region comprises identifying the text region in the image approximate to the area of the image.

3. The computer-implemented method of claim 1, further comprising:

cropping the image, the cropped image including the text region;

wherein recognizing the text comprises;

transmitting the cropped image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

4. The computer-implemented method of claim 1, wherein recognizing the text comprises:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

5. The computer-implemented method of claim 1, wherein identifying the text region in the image comprises:

identifying an area of the image displayed on the area of the touch sensitive display device;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

determining a skew angle of the text region based on the angle of the area of the image;

correcting the skew angle by rotating at least a portion of the image including the text region; and

identifying the text region in the at least a portion of the image.

6. The computer-implemented method of claim 5, wherein determining the skew angle of the text region comprises:

calculating a plurality of projection profiles of a plurality of angles for the at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles; and

determining the skew angle based on an angle of the horizontal projection profile.

7. The computer-implemented method of claim 1, further comprising detecting a circling gesture.

8. A non-transitory computer-readable storage medium encoded with executable computer program code for selectively recognizing text in an image, the computer program code comprising program code for:

displaying the image on a touch sensitive display device;

detecting an underline gesture associated with an area of the touch sensitive display device, the area of the touch sensitive display device associated with the underline gesture being vertical relative to an orientation of the image;

identifying a text region in the image associated with the area of the touch sensitive display device, the text region comprising text displayed vertically relative to the orientation of the image; and

recognizing the text in the text region using OCR technology.

9. The non-transitory computer-readable storage medium of claim 8, wherein the computer program code further comprises program code for:

identifying an area of the image displayed on the area of the touch sensitive display device;

wherein identifying the text region comprises identifying the text region in the image approximate to the area of the image.

10. The non-transitory computer-readable storage medium of claim 8, wherein the computer program code further comprises program code for:

cropping the image, the cropped image including the text region;

wherein recognizing the text comprises;

transmitting the cropped image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

11. The non-transitory computer-readable storage medium of claim 8, wherein recognizing the text comprises:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

12. The non-transitory computer-readable storage medium of claim 8, wherein identifying the text region in the image comprises:

identifying an area of the image displayed on the area of the touch sensitive display device;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

determining a skew angle of the text region based on the angle of the area of the image;

correcting the skew angle by rotating at least a portion of the image including the text region; and

identifying the text region in the at least a portion of the image.

13. The non-transitory computer-readable storage medium of claim 12, wherein determining the skew angle of the text region comprises:

calculating a plurality of projection profiles of a plurality of angles for the at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles; and

determining the skew angle based on an angle of the horizontal projection profile.

14. The non-transitory computer-readable storage medium of claim 8, wherein the gesture comprises a circling gesture.

15. A computer system for selectively recognizing text in an image, comprising:

a computer-readable storage medium comprising executable computer program code for;

an image User Interface (UI) module for;

displaying the image on a touch sensitive display device;

detecting an underline gesture associated with an area of the touch sensitive display device, the area of the touch sensitive display device associated with the underline gesture being vertical relative to an orientation of the image;

a text region identification module for identifying a text region in the image associated with the area of the touch sensitive display device, the text region comprising text displayed vertically relative to the orientation of the image; and

an OCR module for recognizing the text in the text region using OCR technology.

16. The computer system of claim 15, wherein the UI module is further configured for:

identifying an area of the image displayed on the area of the touch sensitive display device;

wherein the text region identification module is further configured for identifying the text region comprises identifying the text region in the image approximate to the area of the image.

17. The computer system of claim 15, wherein the text region identification module is further configured for:

cropping the image, the cropped image including the text region;

wherein

the OCR module is further configured for;

transmitting the cropped image to an OCR engine through a computer network, and

receiving the text from the OCR engine through the computer network.

18. The computer system of claim 15, wherein the OCR module is further configured for:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

19. The computer system of claim 15, wherein identifying the text region in the image comprises:

identifying an area of the image displayed on the area of the touch sensitive display device;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

determining a skew angle of the text region based on the angle of the area of the image;

correcting the skew angle by rotating at least a portion of the image including the text region; and

identifying the text region in the at least a portion of the image.

20. The computer system of claim 19, wherein determining the skew angle of the text region comprises:

calculating a plurality of projection profiles of a plurality of angles for the at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles; and

determining the skew angle based on an angle of the horizontal projection profile.

21. The computer system of claim 15, wherein the gesture comprises a circling gesture.