Document mode processing for portable reading machine enabling document navigation

• US 9,236,043 B2
• Filed: 04/01/2005
• Issued: 01/12/2016
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: KNFB Reader LLC
• Inventors: Kurzweil, Raymond C., Albrecht, Paul, Gibson, Lucy
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US11/097,447
• Publication Number:
  US 20060015342A1
• Time in Patent Office: 3,938 Days
• Field of Search: 704/260, 704/270, 704/271
• US Class Current: 1/1
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils

Abstract

Controlling a reading machine while reading a document to a user by receiving an image of a document, accessing a knowledge base that provides data that identifies sections in the document and processing user commands to select a section of the document. The reading machine applies text-to-speech to a text file that corresponds to the selected section of the document, to read the selected section of the document aloud to the user.

Claims

1. A computer program product stored on a computer readable storage device for controlling a reading machine while reading a document to a user, the computer program product comprises instructions for causing the reading machine to:

receive an image of a document;

apply the received image of the document to an optical character recognition engine to produce a text file;

access a knowledge base that provides data that identifies sections in one or more documents;

receive a user command that specifies processing of a section in the received image of the document;

process the received user command to have the program select the section of the document specified by the user command to process, with the instructions to process identifying the section based on data provided from the knowledge base and the text file; and

apply in a text-to-speech processing engine a portion of the text file that corresponds to the selected section of the document, to produce a speech file that when rendered reads the identified section of the document aloud to the user.

2. The computer program product of claim 1, wherein the knowledge base provides data that is specific to the type of the document.

3. The computer program product of claim 2, wherein accessing a knowledge base comprises:

classify the document according to spatial organization of the text in the document; and

retrieve data from the knowledge base that is associated with the classification of the document.

4. The computer program product of claim 2, wherein accessing a knowledge base comprises:

classify the document according to key words or phrases in the document; and

retrieve data from the knowledge base that is associated with the classification of the document.

5. The computer program product of claim 1 wherein the instructions are configured to permit a user to select sections of documents that are books, memos, newspapers or journals.

6. The computer program product of claim 1, further comprising instructions to:

save data associated with reading a document whose sections were identified, the associated data being data retrieved from a knowledge base for the document and data produced or used while processing the document.

7. The computer program product of claim 1, wherein user commands comprise:

read text, spell word, speak faster/slower, skip sentence or paragraph, and read named part of a template.

8. The computer program product of claim 1, wherein the knowledge base stores document models that represent an internal representation of the documents.

9. A method for controlling a reading machine while reading a document to a user, the method comprising:

receiving by a processor device an image of a document;

applying by the processor device the received image of the document to an optical character recognition engine to produce a text file;

accessing by the processor device a knowledge base that provides data that identifies sections in one or more documents;

processing, by the processor device, a received user command to select a section in the received image of the document;

identifying by the processor device the section based on data provided from the knowledge base and the text file; and

applying by the processor device text-to-speech processing to a portion of the text file that corresponds to the selected section of the document, to produce a speech file that when rendered reads the identified section of the document aloud to the user.

10. The method of claim 9, wherein accessing comprises:

accessing a knowledge base that provides data that is specific to the type of document.

11. The method of claim 10, wherein accessing a knowledge base comprises:

classifying the document according to spatial organization of the text in the document; and

retrieving data from the knowledge base that is associated with the classification of the document.

12. The method of claim 10, wherein accessing a knowledge base comprises:

classifying the document according to key words or phrases in the document; and

retrieving data from the knowledge base that is associated with the classification of the document.

13. The method of claim 9, wherein the documents include books, memos, newspapers or journals.

14. The method of claim 9, further comprising:

saving data associated with reading a document whose sections were identified, the associated data being data retrieved from a knowledge base for the document and data produced or used while processing the document.

15. The method of claim 9, wherein user commands comprise:

read text, spell word, speak faster/slower, skip sentence or paragraph, and read named part of a template.

16. The method of claim 9, wherein the knowledge base stores document models that represent an internal representation of the documents.

17. An apparatus comprises:

a computing device;

an image capture device; and

a computer readable medium storing a computer program product for controlling the apparatus while reading a document to a user, the computer program product comprises instructions for causing the apparatus to;

receive an image of a document captured by the image capture device;

apply the received image of the document to an optical character recognition engine to produce a text file;

access a knowledge base that provides data that identifies sections in one or more documents;

process a user command to select a section in the received image of the document;

identify the section based on data provided from the knowledge base and the text file; and

apply text-to-speech to the text file that corresponds to the selected section of the document, to produce a speech file that when rendered reads the identified section of the document aloud to the user.

18. The apparatus of claim 17, wherein instructions to access a knowledge base comprise instructions to:

classify the document according to spatial organization of the text in the document; and

retrieve data from the knowledge base that is associated with the classification of the document.

19. The apparatus of claim 17, wherein instructions to access a knowledge base comprise instructions to:

classify the document according to key words or phrases in the document; and

retrieve data from the knowledge base that is associated with the classification of the document.

20. The apparatus of claim 17, wherein instructions to access a knowledge base comprise instructions to:

save data associated with reading a document whose sections were identified, the associated data being data retrieved from a knowledge base for the document and data produced or used while processing the document.

21. A computer program product stored on a computer readable storage device for controlling a reading machine while reading a document to a user, the computer program product comprises instructions for causing the reading machine to:

receive an image of a document;

apply the image of the document to an optical character recognition engine to produce a text file and a file that includes spatial organization of text information;

access a knowledge base that provides data that identifies sections in the document;

receive a user command that specifies processing of a section in the received image of the document;

process the received user command and the data that classifies the document according to a spatial organization of the text in the document to select the section of the document corresponding to the user command; and

apply in a text-to-speech processing engine the text file to the selected section of the document, produce a speech file that when rendered reads the identified section of the document aloud to the user.

22. The computer program product of claim 21 wherein the knowledge base stores information on the document that classifies the document according to key words or phrases in the document.

23. The computer program product of claim 21 further comprising instructions to configure the optical character recognition engine to recognized text in at least the selected section of the document to produce the text file.

24. The computer program product of claim 21, further comprising instructions to:

save data associated with reading a document whose sections were identified, the associated data being data retrieved from a knowledge base for the document and data produced or used while processing the document.