Electronic device and method for visual text interpretation

• US 20060083431A1
• Filed: 10/20/2004
• Published: 04/20/2006
• Est. Priority Date: 10/20/2004
• Status: Abandoned Application

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Bliss, Harry M.
• Application Number:
  US10/969,372
• Publication Number:
  US 20060083431A1
• Time in Patent Office: Days
• US Class Current: 382/229
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/262 using context analysis, e.g. lexical, syntactic or semantic context

Abstract

An electronic device (700) captures an image (105, 725) that includes textual information having captured words that are organized in a captured arrangement. The electronic device performs optical character recognition (OCR) (110, 730) in a portion of the image to form a collection of recognized words that are organized in the captured arrangement. The electronic device selects a most likely domain (115, 735) from a plurality of domains, each domain having an associated set of domain arrangements, each domain arrangement comprising a set of feature structures and relationship rules. The electronic device forms a structured collection of feature structures (120, 740) from the set of domain arrangements that substantially matches the captured arrangement. The electronic device organizes the collection of recognized words (125, 745) according to the structured collection of feature structures into structured domain information. The electronic device uses the structured domain information (130) in an application that is specific to the domain (750-760).

Claims

1. A method used in an electronic device for visual text interpretation, comprising:

capturing an image that includes textual information having captured words that are organized in a captured arrangement;

performing optical character recognition (OCR) in a portion of the image to form a collection of recognized words that are organized in the captured arrangement;

selecting a most likely domain from a plurality of domains, each domain having an associated set of domain arrangements, each domain arrangement comprising a set of feature structures and relationship rules;

forming a structured collection of feature structures from the set of domain arrangements that substantially matches the captured arrangement;

organizing the collection of recognized words according to the structured collection of feature structures into structured domain information; and

using the structured domain information in an application that is specific to the domain.

2. The method according to claim 1, wherein the captured words are in a first language, and wherein using the structured domain information comprises:

translating the structured domain information into translated words of a second language using a domain specific machine translator of the second language; and

presenting the translated words, visually, using the captured arrangement.

3. The method according to claim 2, wherein the domain specific machine translator includes icon translations, and wherein, when the image includes an icon, translating includes translating the icon into a translated icon that includes at least one of a translated image and a translated word using the domain specific machine translator of the second language, and wherein presenting includes presenting the translated words and translated icon using the captured arrangement.

4. The method according to claim 2, wherein using the structured domain information further comprises:

identifying a user selected portion of the translated words; and

presenting a corresponding portion of the captured words that correspond to the user selected portion of the translated words.

5. The method according to claim 4, wherein identifying a user selected portion of the translated words comprises interacting with the user using a multimodal dialog manager.

6. The method according to claim 4, wherein the corresponding portion of the captured words are presented using one of a text to speech synthesized presentation and a visual presentation.

7. The method according to claim 1, wherein using the structured domain information further comprises:

identifying a user selected portion of the captured arrangement;

translating a corresponding portion of the structured domain information into translated words of a second language using a domain specific machine translator of the second language; and

presenting the translated words of the corresponding portion using the structured arrangement.

8. The method according to claim 1, wherein the structured domain information includes food items, and wherein using the structured domain information comprises:

determining nutritional contents of food items in the structured domain information; and

presenting the nutritional contents for a user according to the captured arrangement.

9. The method according to claim 1, wherein the structured domain information includes a transportation schedule, and wherein using the structured domain information comprises:

determining itinerary criteria from user input;

selecting one or more itinerary segments from the transportation schedule according to the itinerary criteria; and

presenting the one or more itinerary segments.

10. The method according to claim 1, wherein the structured domain information includes information from a business card, and wherein using the structured domain information comprises:

storing portions of the information into a contacts database according to the structured domain information.

11. The method according to claim 1, wherein the structured domain information includes a racing schedule for a race, and wherein using the structured domain information comprises:

identifying predicted leaders of the race from the structured domain information of the racing schedule and other data in the electronic device; and

presenting the one or more leaders.

12. The method according to claim 1, wherein the image is acquired by one of an optical scanner or a camera that is a portion of a hand-held device.

13. The method according to claim 1, wherein the most likely domain is at least partially selected using one or more inputs from a user.

14. The method according to claim 1, wherein the most likely domain is at least partially selected using a domain dictionary and one or more words from the collection of recognized words.

15. The method according to claim 1, wherein the most likely domain is selected using geographic location information acquired by the electronic device and a domain location data base stored in the electronic device.

16. The method according to claim 1, further comprising selecting the application that is specific to the domain from a set of domain specific applications.

17. A method used in an electronic device for visual text interpretation, comprising:

capturing an image that includes textual information having captured words that are organized in a captured arrangement;

performing optical character recognition (OCR) in a portion of the image to form a collection of recognized words that are organized in the captured arrangement;

selecting a most likely domain from a plurality of language independent domains, each domain having an associated set of domain arrangements, each domain arrangement comprising a set of feature structures and relationship rules;

forming a structured collection of feature structures from the set of domain arrangements that substantially matches the captured arrangement;

organizing the collection of recognized words according to the structured collection of feature structures into structured domain information;

translating the structured domain information into translated words of a second language using a domain specific machine translator of the second language; and

presenting the translated words, visually, using the captured arrangement.

18. The method according to claim 17, further comprising:

identifying a user selected portion of the translated words; and

presenting a corresponding portion of the captured words that correspond to the user selected portion of the translated words.

19. An electronic device for visual text interpretation, comprising:

a capture means for capturing an image that includes textual information having captured words that are organized in a captured arrangement;

an optical character recognition means for performing optical character recognition (OCR) in a portion of the image to form a collection of recognized words that are organized in the captured arrangement;

a domain determination means for selecting a most likely domain from a plurality of domains, each domain having an associated set of domain arrangements, each domain arrangement comprising a set of feature structures and relationship rules;

a structure forming means for forming a structured collection of feature structures from the set of domain arrangements that substantially matches the captured arrangement;

an information organization means for organizing the collection of recognized words according to the structured collection of feature structures into structured domain information; and

a plurality of domain specific applications from which one is selected to use the structured domain information.