GESTURE-BASED SELECTIVE TEXT RECOGNITION

• US 20110081083A1
• Filed: 10/07/2009
• Published: 04/07/2011
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lee, Dar-Shyang, Chien, Lee-Feng, Wong, Kin, Hsieh, Aries, Ting, Pin
• Granted Patent: US 8,520,983 B2
• Time in Patent Office: Days
• US Class Current: 382/182
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1456 based on user interactions
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

An image is displayed on a touch screen. A user'"'"'s underline gesture on the displayed image is detected. The area of the image touched by the underline gesture and a surrounding region approximate to the touched area are identified. Skew for text in the surrounding region is determined and compensated. A text region including the text is identified in the surrounding region and cropped from the image. The cropped image is transmitted to an optical character recognition (OCR) engine, which processes the cropped image and returns OCR'"'"'ed text. The OCR'"'"'ed text is outputted.

Claims

1. A computer-implemented method for selectively recognizing text in an image, comprising:

displaying the image on a touch sensitive display device;

detecting a gesture associated with an area of the touch sensitive display device;

identifying a text region in the image associated with the area of the touch sensitive display device, the text region comprising text; and

recognizing the text in the text region using OCR technology.

2. The computer-implemented method of claim 1, further comprising:

identifying an area of the image displayed on the area of the touch sensitive display device;

wherein identifying the text region comprises identifying the text region in the image approximate to the area of the image.

3. The computer-implemented method of claim 1, further comprising:

cropping the image for a cropped image including the text region;

wherein recognizing the text comprises;

transmitting the cropped image to an OCR engine through a computer network, and

receiving the text from the OCR engine through the computer network.

4. The computer-implemented method of claim 1, wherein recognizing the text comprises:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network, and

receiving the text from the OCR engine through the computer network.

5. The computer-implemented method of claim 1, wherein identifying the text region in the image comprises:

identifying an area of the image displayed on the area of the touch sensitive display device;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

determining a skew angle of the text region based on the angle of the area of the image;

correcting the skew angle by rotating at least a portion of the image including the text region; and

identifying the text region in the at least a portion of the image.

6. The computer-implemented method of claim 5, wherein determining the skew angle of the text region comprises:

calculating a plurality of projection profiles of a plurality of angles for the at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles; and

determining the skew angle based on an angle of the horizontal projection profile.

7. The computer-implemented method of claim 1, wherein the gesture comprises an underline gesture.

8. The computer-implemented method of claim 7, wherein the area of the touch sensitive display device associated with the underline gesture is vertical relative to an orientation of the image and wherein the text in the text region comprises text displayed vertically.

9. The computer-implemented method of claim 1, wherein the gesture comprises a circling gesture.

10. A computer-readable storage medium encoded with executable computer program code for selectively recognizing text in an image, the computer program code comprising program code for:

displaying the image on a touch sensitive display device;

detecting a gesture associated with an area of the touch sensitive display device;

identifying a text region in the image associated with the area of the touch sensitive display device, the text region comprising text; and

recognizing the text in the text region using OCR technology.

11. The computer-readable storage medium of claim 10, wherein the computer program code further comprises program code for:

identifying an area of the image displayed on the area of the touch sensitive display device;

wherein identifying the text region comprises identifying the text region in the image approximate to the area of the image.

12. The computer-readable storage medium of claim 10, wherein the computer program code further comprises program code for:

cropping the image for a cropped image including the text region;

wherein recognizing the text comprises;

transmitting the cropped image to an OCR engine through a computer network; and

receiving the text from the OCR engine through the computer network.

13. The computer-readable storage medium of claim 10, wherein recognizing the text comprises:

transmitting the image and location information about the text region in the image to an OCR engine through a computer network, and

receiving the text from the OCR engine through the computer network.

14. The computer-readable storage medium of claim 10, wherein identifying the text region in the image comprises:

identifying an area of the image displayed on the area of the touch sensitive display device;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

determining a skew angle of the text region based on the angle of the area of the image;

correcting the skew angle by rotating at least a portion of the image including the text region; and

identifying the text region in the at least a portion of the image.

15. The computer-readable storage medium of claim 14, wherein determining the skew angle of the text region comprises:

calculating a plurality of projection profiles of a plurality of angles for the at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles; and

determining the skew angle based on an angle of the horizontal projection profile.

16. The computer-readable storage medium of claim 10, wherein the gesture comprises an underline gesture.

17. The computer-readable storage medium of claim 16, wherein the area of the touch sensitive display device associated with the underline gesture is vertical relative to an orientation of the image and wherein the text in the text region comprises text displayed vertically.

18. The computer-readable storage medium of claim 10, wherein the gesture comprises a circling gesture.

19. A computer system for selectively recognizing text in an image, comprising:

a computer-readable storage medium comprising executable computer program code for;

an image User Interface (UI) module for displaying the image on a touch sensitive display device and detecting a gesture associated with an area of the touch sensitive display device;

a text region identification module for identifying a text region in the image associated with the area of the touch sensitive display device, the text region comprising text; and

an OCR module for recognizing the text in the text region using OCR technology.

20. The computer system of claim 19, wherein the UI module is further configured for identifying an area of the image displayed on the area of the touch sensitive display device, and wherein the text region identification module is further configured for identifying the text region comprises identifying the text region in the image approximate to the area of the image.

21. The computer system of claim 19, wherein the text region identification module is further configured for cropping the image for a cropped image including the text region, and wherein the OCR module is further configured for transmitting the cropped image to an OCR engine through a computer network and receiving the text from the OCR engine through the computer network.

22. The computer system of claim 19, wherein the OCR module is further configured for transmitting the image and location information about the text region in the image to an OCR engine through a computer network and receiving the text from the OCR engine through the computer network.

23. The computer system of claim 19, wherein identifying the text region in the image comprises:

identifying an area of the image displayed on the area of the touch sensitive display device;

determining an angle of the area of the image and a horizontal axis of the touch sensitive display device;

determining a skew angle of the text region based on the angle of the area of the image;

correcting the skew angle by rotating at least a portion of the image including the text region; and

identifying the text region in the at least a portion of the image.

24. The computer system of claim 23, wherein determining the skew angle of the text region comprises:

calculating a plurality of projection profiles of a plurality of angles for the at least a portion of the image, the plurality of angles determined based on the angle of the area of the image;

identifying a horizontal projection profile in the plurality of projection profiles based on variances of the plurality of projection profiles; and

determining the skew angle based on an angle of the horizontal projection profile.

25. The computer system of claim 19, wherein the gesture comprises an underline gesture.

26. The computer system of claim 25, wherein the area of the touch sensitive display device associated with the underline gesture is vertical relative to an orientation of the image and wherein the text in the text region comprises text displayed vertically.

27. The computer system of claim 19, wherein the gesture comprises a circling gesture.