Gesture processing with low resolution images with high resolution processing for optical character recognition for a reading machine

• US 7,627,142 B2
• Filed: 04/01/2005
• Issued: 12/01/2009
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Gibson, Lucy, Lvovsky, Lev, Albrecht, Paul, Kurzweil, Raymond C., Gashel, James
• Primary Examiner(s): Strege; John B
• Application Number:
  US11/096,712
• Publication Number:
  US 20060013483A1
• Time in Patent Office: 1,705 Days
• Field of Search: 382/114, 348/62
• US Class Current: 382/114
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields

Abstract

A portable reading machine that operates in several modes and performs image preprocessing to prior to optical character recognition. The portable reading machine receives a low resolution image and a high resolution image of a scene and processing the low resolution image to recognize a user-initiated gesture using a gesturing item that indicates a command from the user to the reading machine and the high resolution image to recognize text in the image of the scene, according to the command from the user to the machine.

Claims

1. A method of operating a portable reading machine, the method comprising:

receiving a low resolution image and a high resolution image of a scene;

mapping the scene to a template of the scene, the template defining different portions of the scene and the template identifying at least one portion of the scene in which to recognize text; and

processing by a computer the low resolution image to recognize a user-initiated gesture using a gesturing item that indicates a command from the user to the reading machine to process the image according to the template that defines the scene and the high resolution image to recognize text in at least the portion of the image of the scene, according to the command from the user to the machine and the template.

2. The method of claim 1 wherein the gesturing item is a portion of the user'"'"'s hand or an item held by the user.

3. The method of claim 1 wherein receiving a low resolution image comprises receiving a series of low resolution images;

and

processing the series of low resolution images to detect the presence of a gesture in the series of images.

4. The method of claim 3 wherein processing comprises:

receiving images from a first image input device that produces images at relatively high resolution for text recognition; and

receiving images from a second image input device that produces images at relatively high frame rates and relatively low resolution, compared to the resolution used by the first image input device, to capture the low resolution images for gesture recognition.

5. The method of claim 1 wherein processing comprises:

applying pattern-recognition processing to the low resolution image to detect one of a set of pre-defined gestures that are interpreted by the reading machine, as commands to the machine.

6. The method of claim 1 wherein commands include:

read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of the template and wherein the commands are pre-associated with defined gestures.

7. The method of claim 1 wherein if a gesture is detected, the method further comprises:

operating the reading machine according to the command associated with the detected gesture.

8. The method of claim 6 wherein the command to the portable reading machine is a command to navigate though a hierarchical document by processing different sections of the document for speech synthesis in an order determined by the command.

9. A computer program product tangibly embodied on a computer readable medium for operating a portable reading machine, the computer program product comprising instructions to:

receive a low resolution image and a high resolution image of a scene;

map the scene to a template of the scene, the template defining different portions of the scene and the template identifying at least one portion of the scene in which to recognize text;

process the low resolution image to recognize a user-initiated gesture using a gesturing item that indicates a command from the user to the reading machine to process the image according to the template that defines the scene; and

process the high resolution image to recognize text in at least the portion of the image of the scene, according to the command from the user to the machine and the template.

10. The computer program product of claim 9 wherein the computer program product is configured with data corresponding to a pattern to recognize a user'"'"'s hand as the gesturing item.

11. The computer program product of claim 9 wherein receiving a low resolution image comprises instructions to:

receive a series of low resolution images; and

process the series of low resolution images to detect the presence of a gesture in the series of images.

12. The computer program product of claim 11 wherein instructions to process, comprises instructions to:

receive images from a first image input device that produces images at relatively high resolution for text recognition; and

receive images from a second image input device that produces images at relatively high frame rates and relatively low resolution, compared to the resolution used by the first image input device, to capture the low resolution images for gesture recognition.

13. The computer program product of claim 11 wherein instructions to process, comprises instructions to:

apply pattern-recognition processing to the low resolution image to detect one of a set of pre-defined gestures that are interpreted by the reading machine, as commands to the machine.

14. The computer program product of claim 11 wherein the computer program product is configured to associate pre-defined patterns of gestures to commands that include read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template.

15. The computer program product of claim 11 wherein if a gesture is detected the computer program product operates the reading machine according to the command associated with the detected gesture.

16. The computer program product of claim 11 wherein the command to the portable reading machine is to navigate though a hierarchical document, by processing different sections of the document for speech synthesis, is in an order determine by the command.

17. A portable reading machine comprising:

a computing device;

an image input device coupled to the computing device for capturing low resolution images and high resolution images

a computer readable medium storing a computer program product for operating a portable reading machine, the computer program product comprising instructions to;

receive a low resolution image and a high resolution image of a scene from the image input device;

map the scene to a template of the scene, the template defining different portions of the scene and the template identifying at least one portion of the scene in which to recognize text;

process the low resolution image to recognize a user-initiated gesture using a gesturing item that indicates a command from the user to the reading machine to process the image according to the template that defines the scene; and

process the high resolution image to recognize text in at least the portion of the image of the scene, according to the command from the user to the machine and the template.

18. The reading machine of claim 17 wherein the computer program product is configured with data corresponding to a pattern to recognize a user'"'"'s hand as the gesturing item.

19. The reading machine of claim 17 wherein the computer program product, comprises instructions to:

apply pattern-recognition processing to the low resolution image to detect one of a set of pre-defined gestures that are interpreted by the reading machine, as commands to the machine.

20. The reading machine of claim 17 wherein the computer program product is pre-configured to associate pre-defined patterns of gestures to commands that include read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template.