TEXT RECOGNITION DRIVEN FUNCTIONALITY

• US 20130329023A1
• Filed: 06/11/2012
• Published: 12/12/2013
• Est. Priority Date: 06/11/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Suplee, Richard Howard III, Heller, Geoffrey Scott, Bibireata, Daniel
• Granted Patent: US 9,916,514 B2
• Time in Patent Office: Days
• US Class Current: 348/61
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1456 based on user interactions
• CPC Class Codes: H04N 1/00331 with an apparatus performing optical character recognition character recognition G06V30/10
• CPC Class Codes: H04N 1/00968 by scanning marks on a sheet mark-sheet input means H04N1/00355; sensing record carriers G06K7/00
• CPC Class Codes: H04N 1/32048 Auto-dialling or Auto-calling H04N1/32053 - H04N1/32074 take precedence
• CPC Class Codes: H04N 2201/0084 Digital still camera

Abstract

Various approaches for providing textual information to an application, system, or service are disclosed. In particular, various embodiments enable a user to capture an image with a camera of a portable computing device. The computing device is capable of taking the image and processing it to recognize, identify, and/or isolate the text in order to forward the text to an application or function. The application or function can then utilize the text to perform an action in substantially real-time. The text may include an email, phone number, URL, an address, and the like and the application or function may be dialing the phone number, navigating to the URL, opening an address book to save contact information, displaying a map to show the address, and so on.

Claims

1. A computer-implemented method comprising:

under control of one or more computer systems configured with executable instructions,

obtaining image information captured by a camera of a portable computing device;

processing the image information to locate at least one region of the image having properties of a text string;

analyzing the at least one region using an optical character recognition algorithm to recognize text in the at least one region;

identifying a text pattern corresponding to the recognized text;

determining an application associated with the text pattern; and

automatically providing the recognized text to the application.

2. The computer-implemented method of claim 1, wherein the obtaining, processing, analyzing, identifying, determining, and providing occur automatically upon obtaining image information.

3. The computer-implemented method of claim 1, further comprising:

causing the application to perform at least one of dialing when the text pattern indicates a phone number, opening an email application for composing an email when the text pattern indicates an email, navigating to a URL when the text pattern indicates a web address, opening an address book for saving contact information when the text pattern indicates a presence of at least a phone number and an address, displaying a map to show an address when the text pattern indicates a physical address, performing language translation, or solving a math equation.

4. The computer-implemented method of claim 3, further comprising:

monitoring user activity over time to adjust an action performed by the application.

5. The computer-implemented method of claim 1, further comprising:

confirming user intent to provide the recognized text to the application by sensing at least one of a movement of the device or orientation of the device using at least one of an accelerometer or gyroscope.

6. A computer-implemented method comprising:

under control of one or more computer systems configured with executable instructions,

identifying image information associated with text in an image;

determining a type of text from the image information associated with text;

determining at least one function associated with the type of text; and

enabling a computing device to perform the at least one function using the text.

7. The computer-implemented method of claim 6, further comprising:

prompting a user to confirm intent to perform the at least one function.

8. The computer-implemented method of claim 6, further comprising:

prompting a user to choose between a primary action and at least a secondary action associated with the type of text.

9. The computer-implemented method of claim 6, further comprising:

obtaining the image, the image being obtained from a camera of the computing device.

10. The computer-implemented method of claim 6, wherein the image is one of:

obtained automatically when a camera of the computing device is focused on an object or obtained in response to a user input.

11. The computer-implemented method of claim 6, wherein the at least one function includes at least one of dialing when the type of text indicates a phone number, opening an email application for composing an email when the type of text indicates an email, navigating to a URL when the type of text indicates a web address, opening an address book for saving contact information when the type of text indicates a presence of at least a phone number and an address, displaying a map to show an address when the type of text indicates a physical address, performing language translation, or solving a math equation.

12. The computer-implemented method of claim 9, further comprising:

thresholding the image to standardized the image information associated with text to account for at least one of a blur, noise, or lighting variation of the image.

13. The computer-implemented method of claim 6, wherein at least one of a computer vision algorithm or an optical character recognition algorithm are used to at least identify the image information associated with text from the image.

14. The computer-implemented method of claim 6, wherein at least one of the identifying the image information associated with text, determining the type of the text, or determining the at least one function is performed by a remote server in communication with the computing device.

15. A computer-implemented method comprising:

under control of one or more computer systems configured with executable instructions,

recognizing a string of characters from a digital image captured by a camera of a portable computing device;

identifying a character pattern from the string of characters; and

determining an application associated with the character pattern; and

providing the character pattern to the application.

16. The computer-implemented method of claim 15, wherein the application when executed performs at least one of dialing when the character pattern indicates a phone number, navigating to a URL when the character pattern indicates a web address, opening an address book for saving contact information when the character pattern indicates a presence of at least a phone number and an address, displaying a map to show an address when the character pattern indicates a physical address, performing language translation, or solving a math equation.

17. The computer-implemented method of claim 15, further comprising:

prompting a user to choose between a primary action and at least a secondary action associated with the character pattern.

18. A portable computing device, comprising:

a processor; and

a memory device including instructions that, when executed by the processor, cause the processor to;

identify text in an image;

identify a text pattern from the identified text;

determine at least one function associated with the text pattern; and

perform the at least one function associated with the text pattern.

19. The portable computing device of claim 18, wherein identifying the text, identifying the text pattern, determining the at least one function, and performing the at least one function are performed locally on the portable computing device.

20. The portable computing device of claim 18, wherein at least one of the identifying the text, identifying the text pattern, or determining the at least one function are performed by a remote server in communication with the portable computing device.

21. The computer-implemented method of claim 18, further comprising:

a camera for obtaining the image automatically upon being focused on an object or in response to input from a user.

22. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:

identify text in an image;

identify a text pattern from the identified text;

determine at least one function associated with the text pattern; and

perform the at least one function using the text pattern.

23. The non-transitory computer-readable storage medium of claim 22, wherein at least one of identifying the text, identifying the text pattern, determining the at least one function, or performing the at least one function are performed locally on the portable computing device.

24. The non-transitory computer-readable storage medium of claim 22, storing instructions that, when executed by a processor, cause the processor to further:

monitoring user activity over time to adjust actions performed by the at least one function.

25. The non-transitory computer-readable storage medium of claim 22, wherein the text pattern includes at least one of a pattern indicative of a telephone number, a pattern indicative of an email address, a pattern indicative of a URL, or a pattern indicative of a street address.