Document Mode Processing For Portable Reading Machine Enabling Document Navigation

• US 20150054744A1
• Filed: 10/29/2014
• Published: 02/26/2015
• Est. Priority Date: 04/02/2004
• Status: Abandoned Application

• Current Assignee: KNFB Reader LLC
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Kurzweil, Raymond C., Albrecht, Paul, Gibson, Lucy
• Application Number:
  US14/526,886
• Publication Number:
  US 20150054744A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils

Abstract

Controlling a reading machine while reading a document to a user by receiving an image of a document, accessing a knowledge base that provides data that identifies sections in the document and processing user commands to select a section of the document. The reading machine applies text-to-speech to a text file that corresponds to the selected section of the document, to read the selected section of the document aloud to the user.

Claims

1-20. -20. (canceled)

21. A method of operating a portable device, the method comprising:

receiving by a processor in the portable device data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;

processing by the processor the image to recognize a user-initiated gesture in a scene captured in the image;

determining by the processor based on the recognized user-initiated gesture a function to perform by the portable device; and

performing the determined function by the portable device according to the recognized user-initiated gesture.

22. The method of claim 21 wherein the user selects a gesturing mode to configure the processor, the method further comprising:

determining by the processor that a pointer for providing the gesture was not detected; and

generating by the processor a notification to the user that the pointer was not detected by the portable device.

23. The method of claim 22 wherein a directed reading mode is selected by the user.

24. The method of claim 21 wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device includes a separate camera that captures high-resolution images for text recognition processing.

25. The method of claim 24 wherein processing comprises:

applying pattern-recognition processing to low resolution images captured by cameras on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.

26. The method of claim 24 wherein the eyeglasses have scanning sensors that produce signals that are translated into signals that allows the user to control the direction of scanning through motion of the user'"'"'s head.

27. The method of claim 21 wherein gesturing involves a gesturing item.

28. The method of claim 27 wherein the gesturing item is a portion of the user'"'"'s hand or an item held by the user and the gesturing indicates the command from the user to the portable device.

29. The method of claim 21 wherein commands include:

read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures.

30. The method of claim 26 wherein the command to the portable device to navigate through a hierarchical document, by processing different sections of the document for speech synthesis, is in an order determine by the command.

31. The method of claim 21 further comprising:

receiving a series of images from the image input device;

processing by the processor in the portable device, the images to extract a series of low resolution images; and

processing by the processor in the portable device the series of low resolution images to detect the presence of the user-initiated gesture in the series of images.

32. A portable device comprises:

a processor;

memory in communication with the processor, the processor configured to;

receive data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;

process the image to recognize a user-initiated gesture in a scene captured in the image;

determine based on the recognized user-initiated gesture a function to perform by the portable device; and

perform the determined function by the portable device according to the recognized user-initiated gesture.

33. The portable device of claim 32 wherein the user selects a gesturing mode to configure the processor for gesture processing, the method further comprising:

determine that a pointer for providing the gesture was not detected; and

provide a notification to the user that the pointer was not detected by the portable device.

34. The portable device of claim 32 wherein a directed reading mode is selected by the user.

35. The portable device of claim 32 wherein the camera mounted on the eyeglasses is a low resolution camera for gesture processing and the portable device includes a separate camera that captures high-resolution images for text recognition processing.

36. The portable device of claim 35 wherein the processor is further configured to:

apply pattern-recognition processing to low resolution images captured by low resolution camera on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.

37. The portable device of claim 32 wherein the commands include read text, describe scene, spell word, speak faster/slower, skip sentence or paragraph, zoom in or out, and read named part of template and wherein the commands are pre-associated with defined gestures.

38. The portable device of claim 32 wherein the processor is further configured to:

receive a series of images from the image input device;

process the images to extract a series of low resolution images; and

process the series of low resolution images to detect the presence of the user-initiated gesture in the series of images.

39. A computer program product tangibly stored on a computer readable hardware storage device for controlling a portable electronic device, the computer program product comprises instructions for causing a processor to:

receive data that corresponds to an image captured by an image input device mounted on a set of eyeglasses, the image input device in communication with the portable device;

process the image to recognize a user-initiated gesture in a scene captured in the image;

determine based on the recognized user-initiated gesture a function to perform by the portable device; and

perform the determined function by the portable device according to the recognized user-initiated gesture.

40. The computer program product of claim 39 further comprises instructions to:

apply pattern-recognition processing to low resolution images captured by low resolution camera on the eyeglasses to detect one of a set of pre-defined gestures that are interpreted by the portable device, as commands to the portable device.