CONTEXTUAL COMMENTARY OF TEXTUAL IMAGES

• US 20100299134A1
• Filed: 05/22/2009
• Published: 11/25/2010
• Est. Priority Date: 05/22/2009
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lam, Wilson
• Application Number:
  US12/471,257
• Publication Number:
  US 20100299134A1
• Time in Patent Office: Days
• US Class Current: 704/3
• CPC Class Codes: A61H 2201/501 connected to external computer devices or networks
• CPC Class Codes: A61H 2201/5015 using specific interfaces or standards, e.g. USB, serial, parallel
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

A mobile computing system includes an image capture device and an image-analysis module to receive a live stream of images from the image capture device. The image-analysis module includes a text-recognition module to identify a textual image in the live stream of images, and a text-conversion module to convert the textual image identified by the text-recognition module into textual data. The mobile computing system further includes a context module to determine a context of the textual image, and a commentary module to formulate a contextual commentary for the textual data based on the context of the textual image.

Claims

1. A mobile computing system, comprising:

an image capture device;

an image-analysis module to receive a live stream of images from the image capture device, the image-analysis module including;

a text-recognition module to identify a textual image of a nonnative language in the live stream of images; and

a translating module to convert the textual image identified by the text-recognition module into textual data of a native language; and

a visual synthesizer to display the textual image of the native language as an enhancement to the textual image of the nonnative language.

2. The mobile computing system of claim 1, further comprising:

a locator module to determine location data identifying a location of the mobile computing system;

a commentary module to formulate a contextual commentary for the textual data based on the location data; and

an audio synthesizer to output the contextual commentary as an audio signal.

3. The mobile computing system of claim 2, further comprising

an orientation-detection module to determine orientation data identifying a directional orientation of the image capture device.

4. The mobile computing system of claim 2, where the commentary module further formulates the contextual commentary for the textual data based on the orientation data.

5. The mobile computing system of claim 2, further comprising

a navigator module configured to formulate navigation directions to the textual image.

6. The mobile computing system of claim 2, where the image-analysis module is configured to analyze the live stream of images in accordance with entity extraction principles associated with the location identified by the location data.

7. A mobile computing system, comprising:

an image capture device;

an image-analysis module to receive a live stream of images from the image capture device, the image-analysis module including;

a text-recognition module to identify a textual image in the live stream of images; and

a text-conversion module to convert the textual image identified by the text-recognition module into textual data;

a context module to determine a context of the textual image; and

a commentary module to formulate a contextual commentary for the textual data based on the context of the textual image.

8. The mobile computing system of claim 7, where the context module includes a locator module to determine a location of the mobile computing system.

9. The mobile computing system of claim 8, where the commentary module is configured to include information derived from the location in the contextual commentary.

10. The mobile computing system of claim 7, where the image-analysis module includes an input-detection module to recognize in the live stream of images an input device including one or more textual images.

11. The mobile computing system of claim 7, where the image-analysis module includes a clock-detection module to recognize in the live stream of images a clock including hour-indicating numerals arranged in a circle.

12. The mobile computing system of claim 7, where the image-analysis module further includes a Braille-recognition module to identify a Braille image in the live stream of images and a Braille-conversion module to convert the Braille image identified by the Braille-recognition module into textual data.

13. The mobile computing system of claim 7, where the text-conversion module is configured to convert the textual image into textual data having a string data type.

14. The mobile computing system of claim 7, further comprising an audio synthesizer to output the contextual commentary as an audio signal.

15. The mobile computing system of claim 7, further comprising a visual synthesizer to output the contextual commentary as a video signal.

16. The mobile computing system of claim 7, further comprising a translating module to convert a textual image of a nonnative language into textual data of a native language.

17. The mobile computing system of claim 7, further comprising a unit-conversion module to convert textual data having a numeric value associated with a first unit to textual data having a numeric value associated with a second unit, and where the commentary module is configured to formulate the contextual commentary for the textual data having the numeric value associated with the second unit.

18. A method of providing audio assistance from visual information, the method comprising:

receiving a live stream of images;

identifying a textual image in the live stream of images;

identifying a context of the textual image;

converting the textual image into textual data;

associating a contextual commentary with the textual data based on the context of the textual image; and

outputting the contextual commentary.

19. The method of claim 18, where identifying a context of the textual image includes finding a geographic location of the textual image and retrieving information corresponding to the geographic location.

20. The method of claim 18, where identifying a context of the textural image includes checking the textual image for one or more predetermined visual characteristics, each such visual characteristic previously associated with a context.