Text stitching from multiple images

• US 7,840,033 B2
• Filed: 04/01/2005
• Issued: 11/23/2010
• Est. Priority Date: 04/02/2004
• Status: Expired due to Fees

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Kurzweil, Raymond C, Gibson, Lucy, Lvovsky, Lev, Albrecht, Paul
• Primary Examiner(s): Bhatnagar; Anand
• Assistant Examiner(s): CHU, RANDOLPH I
• Application Number:
  US11/097,094
• Publication Number:
  US 20060013444A1
• Time in Patent Office: 2,062 Days
• Field of Search: 382/114, 382/321
• US Class Current: 382/114
• CPC Class Codes: G06F 40/194 Calculation of difference between files
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G09B 21/006 using audible presentation of the information

Abstract

A reading machine has processing for detecting common text between a pair of individual images. The reading machine combines the text from the pair of images into a file or data structure if common text is detected, and determines if incomplete text phrases are present in the common text. If incomplete text phrases are present, the machine signals a user to move an image input device in a direction to capture more of the text.

Claims

1. A computer program product residing on a non-transitory computer readable storage medium for causing a computing device to examine multiple images, the computer program product comprising instructions for causing the computing device to:

detect common text between a pair of individual images;

combine the text from the pair of images into a file or data structure when common text is detected;

determine if incomplete text phrases are present in the combined text from the pair of images at edge portions of either of the images, and when incomplete text phrases are present in the combined text from the pair of images,

signal a user with an indication of instructions to guide a user to move an image input device that captures the pair of individual images in a direction to capture more of the text, with the instructions to guide based at least in part on the incomplete text phrases that were determined to be present.

2. The computer program product of claim 1 wherein common text between two strings from the pair of images is detected by searching for the longest common subsequence of characters in the strings.

3. The computer program product of claim 1 wherein common text between two strings is detected by producing a measure of a degree of match from any two strings, based on how many characters match in the strings.

4. The computer program product of claim 1 wherein stitching the text from the pair of images comprises:

stitching the text together from the two images in accordance with a measure of match between strings in the two images.

5. The computer program product of claim 1 wherein the detection of incomplete text comprises:

determining that text is very close to the edge of the image.

6. The computer program product of claim 1 wherein the detection of incomplete text comprises:

using a text string recognized at an edge of the image as an input to a dictionary and determining that the text string does not correspond to any words in the dictionary;

indicating that the text string corresponds to incomplete text.

7. The computer program product of claim 1 wherein the detection of incomplete text comprises:

using a text string recognized at an edge of the image as an input to a grammar checker;

determining that the text string is ungrammatical; and

indicating that the text string corresponds to incomplete text.

8. The computer program product of claim 5 further comprising instructions to:

announce to the user to capture another image if incomplete text is detected.

9. The computer program product of claim 1 further comprising instructions to:

combine the information from the two images either by performing text stitching or by performing image stitching; and

re-process the appropriate section of the combined image;

determine if incomplete text phrases are present, and if so,

signal the user to move the image input device in a direction to capture more of one or more of the images, and if incomplete text phases are not present,

process the text in the image to have the machine read the text to the user.

10. A computer implemented method comprising:

detecting by a computing device associated with an image input device, common text between a pair of individual images captured by the image input device; and

combining the text from the pair of images into a file or data structure stored in the computing device when common text is detected;

determining by the computing device that incomplete text phrases are present in the combined text from the pair of images at edge portions of either of the images, and if incomplete text phrases are present in the combined text from the pair of images,

signaling with an indication of instructions to guide a user a user to move the image input device in a direction to capture more of the text, with the instructions to guide based at least in part on the incomplete text phrases that were determined to be present.

11. The method of claim 10 wherein detecting common text between two strings from the pair of images is detected by searching for the longest common subsequence of characters in the strings.

12. The method of claim 10 wherein detecting common text between two strings is detected by producing a measure of a degree of match from any two strings, based on how many characters match in the strings.

13. The method of claim 10 wherein combining the text from the pair of images comprises:

stitching the text together from the two images in accordance with a measure of match between strings in the two images.

14. The method of claim 10 wherein determining incomplete text comprises:

determining that text is very close to the edge of image.

15. The method of claim 10 wherein determining incomplete text comprises:

using a text string recognized at an edge of the image as an input to a dictionary and determining that the text string does not correspond to any words in the dictionary; and

indicating that the text string corresponds to incomplete text.

16. The method of claim 10 wherein determining incomplete text comprises:

using a text string recognized at an edge of the image as an input to a grammar checker;

determining that the text string is ungrammatical; and

indicating that the text string corresponds to incomplete text.

17. The method of claim 10 further comprises:

forming by the computing device speech signals to render an announcement to the user to capture another image if incomplete text is detected.

18. The method of claim 10 further comprises:

combining in the computing device the information from the two images either by performing text stitching or by performing image stitching; and

re-processing the appropriate section of the combined image;

determine if incomplete text phrases are present, and if so,

signal the user to move the image input device in a direction to capture more of one or more of the images, and if incomplete text phases are not present,

process the text in the image to have the machine read the text to the user.

19. A reading machine comprises:

a computing device;

an image input device coupled to the computing device for capturing images; and

a computer program product residing on a computer readable medium comprising instructions for causing the computing device to;

detect common text between a pair of individual images; and

combine the text from the pair of images into a file or data structure when common text is detected;

determine if incomplete text phrases are present in the combined text from the pair of images at edge portions of either of the images, and when incomplete text phrases are present in the combined text from the pair of images,

signal with an indication of instructions to guide a user a user to move an image input device in a direction to capture more of the text, with the instructions to guide based at least in part on the incomplete text phrases that were determined to be present.

20. The reading machine of claim 19 wherein common text between two strings is detected by producing a measure of a degree of match from any two strings, based on how many characters match in the strings.

21. The reading machine of claim 19 wherein stitching the text from the pair of images together comprises:

stitching the text together from the two images in accordance with a measure of match between strings in the two images.

22. The reading machine of claim 19 wherein the computer program product further comprises instructions to:

combine the information from the two images either by performing text stitching or by performing image stitching; and

re-process the appropriate section of the combined image;

determine if text phrases are present, and if so,

signal the user to move the image input device in a direction to capture more of one or more of the images, and if incomplete text phases are not present,

process the text in the image to have the machine read the text to the user.

23. A reading machine comprises:

a computing device;

an image input device coupled to the computing device for capturing images; and

a computer program product tangible embodied on a computer readable medium for causing the computing device to examine multiple individual images, the computer program product comprising instructions for causing the computing device to;

perform optical character recognition on a pair of captured images to produce recognized text from each of the images;

detect text in one of the pair of individual images that is also found in the other one of the pair of images;

combine recognized text from each of the images based on the detected common text;

analyze the combined text to determine if incomplete text phrases are present in the combined text from the pair of images at edge portions of either of the images, and when incomplete text phrases are present in the combined text from the pair of images.

generate commands to signal a user with user-instructions that guide a user to move an image input device that captures the pair of individual images in a direction to capture more of the text, with the user-instructions to guide based at least in part on the detected incomplete text.

24. The reading machine of claim 23 wherein common text between two strings from the pair of images is detected by searching for the longest common subsequence of characters in the strings.

25. The reading machine of claim 23 wherein common text between two strings is detected by producing a measure of a degree of match from any two strings, based on how many characters match in the strings;

and

combine the text from the two images in accordance with the measure of match between strings in the two images.

26. The reading machine of claim 24 wherein the detection of incomplete text comprises:

analyze a text string recognized at an edge of the one of the image with a dictionary checker program that accesses a dictionary of words to determine that the text string does not correspond to any words in the dictionary;

produce an indication that the text string corresponds to incomplete text based on determining that the text string does not correspond to any words in the dictionary.

27. The reading machine of claim 24 wherein the detection of incomplete text comprises:

analyze a text string recognized at an edge of one of the images with a grammar checker to determine that the text string is ungrammatical; and

produce an indication that the text string corresponds to incomplete text based on determining that the text string was ungrammatical.