Cooperative processing for portable reading machine

• US 8,036,895 B2
• Filed: 04/01/2005
• Issued: 10/11/2011
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Gibson, Lucy, Albrecht, Paul, Kurzweil, Raymond C.
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): GODBOLD, DOUGLAS
• Application Number:
  US11/097,958
• Publication Number:
  US 20060015337A1
• Time in Patent Office: 2,384 Days
• Field of Search: 704/260, 704/270, 704/271, 382/321
• US Class Current: 704/270
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/127 with the intervention of an operator
• CPC Class Codes: G06V 30/141 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/146 Aligning or centring of the image pick-up or image-field
• CPC Class Codes: G06V 30/166 Normalisation of pattern dimensions
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems

Abstract

A handheld device includes an image input device capable of acquiring images, circuitry to send a representation of the image to a remote computing system that performs at least one processing function related to processing the image and circuitry to receive from the remote computing system data based on processing the image by the remote system.

Claims

1. A system for reading text to a user, the system comprises:

a portable handheld device including a camera and a transducer for rendering sound, the portable handheld device configure for;

acquiring images through the camera of scenes in an environment that the portable handheld device operates in;

sending over a communication channel, a representation of an acquired image of a scene to an external, remote computer system and user settings and navigation commands; and

the external remote computer system configured to;

communicate with the handheld device;

receive the acquired image, the user settings and the navigation commands,

perform optical character recognition processing on the acquired image in accordance with the user settings;

send to the handheld device results of the optical character recognition processing, the results sent in accordance with the navigation commands, which control how the external, remote computer system processes the results of optical character recognition processing of the acquired image;

the portable handheld device further configured to;

render speech corresponding to recognized text through the transducer of the portable handheld device.

2. The system of claim 1 wherein the handheld device has mobile phone capability and the user settings include a phone number corresponding to the handheld device;

and the external remote system is configured to;

initiate a call to the handheld device and send the recognized text according the navigation commands as speech in the initiated call, and

the portable device renders the recognized text directly as the speech through the transducer.

3. The system of claim 1 wherein the portable handheld device is configured to:

receive a text file of recognized text according to the processing based on the navigation commands;

process the text file with text-to-speech processing;

render the text as speech through the transducer.

4. The system of claim 1 wherein the external remote computer system is configured to:

process the image to produce the results of the optical character recognition as a text file with the navigation commands comprising a model of a document captured in the image, the model specifying the order in which recognized sections of the document are returned as text to the user;

send the text file to the handheld device; and

the handheld device performs text-to-speech processing on the text file to render the speech.

5. The system of claim 1 wherein the external remote computer system is further configured to:

send the results of the optical character recognition to text-to-speech processing in the external remote computer to produce based on the navigation commands a speech file that is sent to the handheld device.

6. The system of claim 1 wherein the user input device is a keypad on the handheld device.

7. The system of claim 6 wherein the handheld device is a telephone and the user settings include a phone number corresponding to the handheld device;

and the handheld device is further configured to;

receive from the remote, external computing system sends a text file of the results as speech signals.

8. The system of claim 1 wherein the camera in the handheld device acquires a low resolution image to determine the orientation of the camera relative to text or to determine whether the edges of a page are cut off from the field of view of the camera, with the handheld device processing the low resolution images and instructing the user to re-orientate the handheld device prior to sending the acquired image.

9. The system of claim 1 wherein the portable device comprises a voice recognition engine and with the user settings are inputted by a user through the voice recognition engine.

10. The system of claim 1 wherein the handheld device includes a keypad and wherein the user settings and navigation commands are inputted by a user using the keypad on the handheld device.

11. The system of claim 1 wherein the handheld device is a telephone and the external remote computer system sends a text file of the results processed according to the navigation commands to a text to speech engine in the computer system that converts the text to speech signals and sends the speech signals as a phone call to the telephone.

12. A handheld device comprises:

an image input device capable of acquiring images;

a user input device for inputting commands;

a transducer for rendering sound; and

a processing unit comprising;

a processor device; and

memory, the processing unit configured to;

send a representation of an acquired image along with user settings to an external, remote computing system that performs optical character recognition processing on the acquired image to recognize text in the acquired image, the user settings and navigation commands inputted by a user from the user input device, the navigation commands to control sending of results of the optical character recognition processing by the external remote system; and

circuitry to receive from the external remote computing system the results based on the processing of the image by the external remote computing system with the results in accordance with the navigation commands, the results being a representation of recognized text in the image that when rendered by the handheld device through the transducer provides speech corresponding to recognized text in the acquired image according to the user navigation commands.

13. The handheld device of claim 12 wherein the handheld device includes a mobile phone and the user settings include a phone number corresponding to the handheld device;

and the handheld device is configured to;

receive a call initiated from the remote, external computing system; and

render the recognized text directly as the speech through the transducer.

14. The handheld device of claim 12 wherein the portable handheld device is configured to:

receive a text file of recognized text according to the processing based on the navigation commands;

process the text file with text-to-speech processing;

render the text as speech through the transducer.

15. The handheld device of claim 12 wherein the handheld device is configured to:

receive a text file produced by the external, remote computing system in accordance with the navigation commands; and

perform text-to-speech processing on the received text file.

16. The handheld device of claim 12 wherein the handheld device is configured to:

receive from the remote, external computing system the results as a speech file.

17. The handheld device of claim 12 wherein the image input device is a camera that acquires a low resolution image to determine the orientation of the camera relative to text or to determine whether the edges of a page are cut off from the field of view of the camera, with the handheld device processing the low resolution images and instructing the user to re-orientate the handheld device prior to sending the acquired image.

18. The handheld device of claim 12 wherein the user settings further include mode commands inputted by a user to control a mode of operation of the handheld device.

19. A portable handheld device comprising:

a processing unit;

a transducer for rendering sound;

a camera for capturing images;

the processing unit configured to;

acquire image data from the camera, the image data of scenes of an environment that the portable handheld device operates in;

acquire user settings and navigation commands;

send over a communication channel a representation of an acquired image and the user settings and navigation commands to a remote external computer system;

receive a text file from the remote external computer system, the text file including text provided from performing optical character recognition processing on the acquired image by the remote external computing system, with the text file having the recognized text receive by the device based on the navigation commands that control how the remote external computer system processes the results of the optical character recognition processing of the acquired image; and

produce speech signals to render the text file as speech, through the transducer.

20. The device of claim 19 wherein the handheld device has mobile phone capability;

and the computing device initiates a call to the handheld device to render the recognized text file directly as the speech through the transducer.

21. The device of claim 19 wherein the processing unit in the portable handheld device is further configured to:

produce a speech file by applying the received text file to a text to speech processing engine in the portable handheld device; and

render the produced speech file as speech through the transducer.

22. The device of claim 19 further comprising:

a tactile feedback output device; and

the processor is configured to render the speech as tactile sensations imparted to a user through the tactile feedback output device.