Mode processing in portable reading machine

• US 7,505,056 B2
• Filed: 04/01/2005
• Issued: 03/17/2009
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Gibson, Lucy, Albrecht, Paul, Kurzweil, Raymond C., Gashel, James
• Primary Examiner(s): Dharia; Prabodh M
• Application Number:
  US11/097,944
• Publication Number:
  US 20060017810A1
• Time in Patent Office: 1,446 Days
• Field of Search: 345/698, 345/7, 345/473, 345/568, 345/502, 345/520, 345/423, 345/555, 345/505, 345/532, 345/98, 345/156, 345/696, 340/958, 455/450, 705/1, 705/10, 705/51, 704/235, 704/260, 704/275, 348/207.1, 235/379, 235/454, 382/114, 382/176, 382/101, 709/246, 707/102, 715/716
• US Class Current: 345/698
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet

Abstract

A reading machine that operates in various modes includes image correction processing is described. The reading machine operates in different modes that optimize performance for specific uses of the reading machine. The reading machine receives data that specifies a mode to use for processing an image. The reading machine accesses a knowledge base to provide data to the machine for the specified mode, with the data including specific target visual elements that are expected to be encountered in processing images for the specified mode. The reading machine captures one or several images and processes the one or several images to identify one or more of the target elements in the image using information obtained from the knowledge base.

Claims

1. A computer program product residing on a computer readable medium for causing a reading machine to operate in different modes that optimize performance for specific uses of the reading machine, the computer program product comprises instructions for causing the reading machine to:

receive data that specifies a mode of a plurality of modes that the reading machine is capable of operating in to use for processing images captured by the reading machine;

access a knowledge base to provide data to configure the computer program product for the specified mode, with the data including specific target visual elements that are expected to be encountered in processing images captured by the reading machine in the specified mode;

capture one or several images; and

process the one or several images to identify one or more of the target elements in the one or several images using information including the data including specific target visual elements obtained from the knowledge base.

2. The computer program product of claim 1 wherein the instructions to receive data, receives data from a user.

3. The computer program product of claim 1 wherein the instructions to receive data, receives data that automatically specifies the mode.

4. The computer program product of claim 1 wherein the program further comprises instructions to:

transiently switch modes for a few images or specify a setting to configure the computer program product that persists until the mode is changed.

5. The computer program product of claim 1 wherein the knowledge base resides on the reading machine.

6. The computer program product of claim 1 wherein the knowledge base is downloaded to the machine according to the specified mode.

7. The computer program product of claim 1 further comprising instructions to:

save information during operation of the reading machine in the mode in order to process images faster or more accurately for a subsequent use of the specified mode.

8. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize text in the image and convert recognized text in the image to text data including a font that of the text in the image; and

wherein the computer program is configured with data to allow a reading machine to operate in a restaurant modes that preferentially determines whether the text data comprises elaborate fonts and culinary phrases that are likely to be found on a menu.

9. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize text in the image and convert recognized text in text data;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow a reading machine to operate in a document mode to read aloud text in the captured images, with the text derived from images of books, magazines and paper copy.

10. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow a reading machine to operate in a clothing mode that forms instructions based on the text and reads the instructions aloud to help the user get dressed by identifying or matching clothing based on features regarding color and pattern.

11. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow a reading machine to operate in an outdoor mode that forms instructions based on the text and reads the instructions aloud to help the user with physical navigation in an outdoor environment.

12. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow a reading machine to operate in an indoor mode to form instructions based on the text to read the instructions aloud to help a person navigate in an indoor environment.

13. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow a reading machine to operate to form instructions based on the text and read the instructions aloud to help a person navigate a work area and/or desktop mode.

14. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize formats of printed matter in the image and is configured with data to allow a reading machine to operate in a newspaper mode to detect columns, titles and page numbers on printed documents.

15. The computer program product of claim 1 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech, and is configured with data to allow a reading machine to operate in a transaction mode to read aloud instructions based on the text to help a person operate a transaction-oriented device that has a standardized layout of controls.

16. A reading machine comprises:

a computing device;

a computer readable medium storing computer program product operative with the computing device, for causing the reading machine to operate in different modes that optimize performance for specific uses of the reading machine, the computer program product comprises instructions for causing the reading machine to;

receive data that specifies a mode of a plurality of modes that the reading machine is capable of operating in to use for processing images captured by the reading machine;

access a knowledge base to provide data to configure the computer program product for the specified mode, with the data including specific target visual elements that are expected to be encountered in processing images captured by the reading machine in the specified mode;

capture one or several images; and

process the one or several images to identify one or more of the target elements in the one or several images using information including the data including specific target visual elements obtained from the knowledge base.

17. The reading machine of claim 16 wherein the computer program product further comprises instructions to:

save information during operation of the reading machine in the mode in order to process images faster or more accurately for a subsequent use of the specified mode.

18. The reading machine of claim 16 wherein the computer program product is configured with data to allow the reading machine to operate in a restaurant mode that identifies elaborate fonts and culinary phrases that are likely to be found on a menu.

19. The reading machine of claim 16 wherein the computer program product is configured with data to allow the reading machine to operate in a document mode to read aloud text in the captured images, with the text derived from images of books, magazines and paper copy.

20. The reading machine of claim 16 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow the reading machine to operate in a clothing mode that helps the user get dressed by identifying or matching clothing based on features regarding color and pattern.

21. The reading machine of claim 16 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow the reading machine to operate in an outdoor mode that forms instructions based on the text and reads the instructions aloud helps the user with physical navigation in an outdoor environment.

22. The reading machine of claim 16 wherein the computer program product further comprises instructions to:

recognize objects in the image and produce text to identify features of the recognized objects;

convert recognized text to speech signals that when rendered produce audible speech; and

wherein the computer program is configured with data to allow the reading machine to operate in an indoor mode to help a person navigate in an indoor environment.

23. The reading machine of claim 16 wherein the computer program product is configured with data to allow the reading machine to operate in a transaction mode to help a person operate a transaction-oriented device that has a standardized layout of controls.

24. A method of operating a reading machine, the method comprises:

selecting a mode from a plurality of different modes that the reading machine is capable of operating in;

accessing a knowledge base to provide data corresponding to the selected mode to configure the reading machine for the specified mode, with the data including specific target visual elements that are expected to be encountered in processing images for the specified mode;

capturing one or several images; and

processing the one or several images to identify one or more of the target elements in the one or several images using information obtained from the knowledge base.

25. The method of claim 24 further comprising:

transiently switching modes for a few images or specifying a setting to configure the reading machine that persists until the mode is changed.

26. The method of claim 24 further comprising:

saving information during operation of the reading machine in the mode in order to process images faster or more accurately for a subsequent use of the specified mode.

27. The method of claim 24 wherein the reading machine operates in at least one of a restaurant mode that preferentially looks for elaborate fonts and culinary phrases that are likely to be found on a menu, a document mode to read books, magazines and paper copy, a clothing mode that helps the user get dressed by identifying or matching clothing based on color and pattern, an outdoor mode that helps the user with physical navigation in an outdoor environment, an indoor mode to help a person navigate in an indoor environment an Work area/Desk Mode to help a person read documents and papers in a work area;

a newspaper mode to detect columns, titles and page numbers on printed documents, and a transaction mode to help a person operate a transaction-oriented device that has a standardized layout of controls.