Speech generation device with a projected display and optical inputs

• US 9,760,123 B2
• Filed: 08/03/2011
• Issued: 09/12/2017
• Est. Priority Date: 08/06/2010
• Status: Active Grant

• Current Assignee: Tobii Dynavox AB
• Original Assignee: Tobii Dynavox LLC (Tobii Technology AB)
• Inventors: Cunningham, Bob
• Primary Examiner(s): Hong, Thomas
• Application Number:
  US13/197,180
• Publication Number:
  US 20120035934A1
• Time in Patent Office: 2,232 Days
• Field of Search: 434116
• CPC Class Codes: G06F 1/1639 the display being based on projection
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems

Abstract

In several embodiments, a speech generation device is disclosed. The speech generation device may generally include a projector configured to project images in the form of a projected display onto a projection Surface, an optical input device configured to detect an input directed towards the projected display and a speaker configured to generate an audio output. In addition, the speech generation device may include a processing unit communicatively coupled to the projector, the optical input device and the speaker. The processing unit may include a processor and related computer readable medium configured to store instructions executable by the processor, wherein the instructions stored on the computer readable medium configure the speech generation device to generate text-to-speech output.

Claims

1. A speech generation device, comprising:

a projector configured to project images in the form of a projected display onto a projection surface disposed in front of a user;

a lap tray mounted to a portion of a wheelchair, the lap tray configured to be rotatable relative to a user sitting in the wheelchair, and the lap tray configured to provide the projection surface, wherein at least the projection surface of the lap tray is configured as a non-transparent surface;

an eye tracking device configured to capture images of a user'"'"'s eyes as the user views the projected display, wherein the eye tracking device is attached to the projection surface at a location outside of and adjacent to a portion of the projection surface configured to receive the projected display from the projector;

a speaker configured to generate an audio output; and

a processing unit communicatively coupled to the projector, the eye tracking device and the speaker, the processing unit including a processor and a related computer readable medium configured to store instructions executable by the processor,

wherein the instructions stored on the computer readable medium configure the processing unit to generate text-to-speech output,

wherein the instructions stored on the computer readable medium further configure the processing unit to track the user'"'"'s eye gaze relative to the user'"'"'s point of regard on the projected display based on the images provided by the eye tracking device and configure the processing unit to provide for display, via the projector, a cursor position at a location on the projected display that corresponds to the user'"'"'s point of regard on the projected display, and

wherein the instructions stored on the computer readable medium configure the processing unit to determine the cursor position corresponding to the user'"'"'s point of regard on the projected display by implementing a mapping function that maps eye gaze measurements determined from the images of the user'"'"'s eyes provided by the eye tracking device to a two-dimensional coordinate space associated with the projected display and convert the images of the user'"'"'s eyes into a cursor position displayed onto the projected display.

2. The speech generation device of claim 1, further comprising an optical input device configured to capture an image of a user'"'"'s hand, wherein the instructions stored on the computer readable medium further configure the processing unit to recognize hand gestures performed by the user as user inputs.

3. The speech generation device of claim 1, wherein the instructions stored on the computer readable medium further configure the processing unit to capture user inputs based on the images provided by the eye tracking device.

4. The speech generation device of claim 3, wherein the user inputs are associated with at least one of dwell time or eye blinking.

5. The speech generation device of claim 1, wherein the eye tracking device comprises an image capture device and an associated light source.

6. A system for generating text-to-speech output for a user of a wheelchair, the system comprising:

a housing attached to a portion of a wheelchair;

a lap tray mounted to a portion of the wheelchair, the lap tray configured to be rotatable relative to a user sitting in the wheelchair;

a projector disposed within or attached to the housing, the projector being configured to project images in the form of a projected display onto a projection surface disposed in front of the user, wherein the lap tray is configured to provide the projection surface, and wherein at least the projection surface of the lap tray is configured as a non-transparent surface;

an eye tracking device configured to capture images of a user'"'"'s eyes as the user views the projected display, wherein the eye tracking device is attached to the lap tray, and wherein the eye tracking device is disposed adjacent to the projected display on the lap tray in an area outside of the projection surface;

a speaker configured to generate an audio output; and

a processing unit communicatively coupled to the projector, the eye tracking device and the speaker, the processing unit including a processor and a related computer readable medium configured to store instructions executable by the processor,

wherein the instructions stored on the computer readable medium configure the processing unit to generate text-to-speech output,

wherein the instructions stored on the computer readable medium further configure the processing unit to track the user'"'"'s eye gaze relative to the user'"'"'s point of regard on the projected display based on the images provided by the eye tracking device and configure the processing unit to provide for display, via the projector, a cursor position at a location on the projected display that corresponds to the user'"'"'s point of regard on the projected display, and

wherein the instructions stored on the computer readable medium configure the processing unit to determine the cursor position corresponding to the user'"'"'s point of regard on the projected display by implementing a mapping function that maps eye gaze measurements determined from the images provided by the eye tracking device to a two-dimensional coordinate space associated with the projected display.

7. The system of claim 6, wherein the housing is attached to an armrest of the wheelchair.

8. The system of claim 6, wherein the instructions stored on the computer readable medium further configure the processing unit to capture eye-based user inputs based on the images provided by the eye tracking device.

9. The system of claim 8, wherein the user inputs are associated with at least one of dwell time or eye blinking.

10. The system of claim 6, wherein the eye tracking device comprises an image capture device and an associated light source.