Systems and methods for haptic augmentation of voice-to-text conversion

• US 8,902,050 B2
• Filed: 10/29/2009
• Issued: 12/02/2014
• Est. Priority Date: 10/29/2009
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Heubel, Robert W., Ramsay, Erin
• Primary Examiner(s): POPE, DARYL C
• Application Number:
  US12/608,707
• Publication Number:
  US 20110102160A1
• Time in Patent Office: 1,860 Days
• Field of Search: 340/407.1, 340/539.1, 340/539.11, 340/539.13, 607/45
• US Class Current: 340/407.1
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: G10L 21/16 Transforming into a non-visible representation devices or methods enabling ear patients to replace direct auditory perception by another kind of perception A61F11/04
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters

Abstract

Systems and methods for providing voice-to-text haptic augmentation in a user interface are disclosed. For example, one disclosed system for converting audible speech information to a haptic effect includes a microphone and a processor in communication with the microphone, the processor configured to receive an audio signal associated with a voice from the microphone, determine a characteristic of the audio signal, and generate an actuator signal based at least in part on the characteristic, the actuator signal configured to cause an actuator to output a haptic effect.

Claims

1. A computer-implemented method comprising:

receiving a plurality of input signals, by a processor, the input signals associated with a plurality of input types, the input types including a voice, an environmental sound, or a biorhythm;

determining, by a processor, characteristics of the input signals;

converting, by a processor, an input signal to text;

determining, by a processor, a haptic effect based at least in part on the characteristics; and

transmitting a message to a remote device, wherein the message comprises;

the text, and

data associated with the haptic effect, the data configured to cause the remote device to output the haptic effect.

2. The computer-implemented method of claim 1, wherein at least one of the characteristics comprises one of a frequency, an amplitude, or a duration.

3. The computer-implemented method of claim 2, wherein at least one of the characteristics comprises a high amplitude and wherein the actuator signal comprises a correspondingly high amplitude actuator signal.

4. The computer-implemented method of claim 1, wherein receiving the plurality of input signals comprises receiving an audio signal from a microphone.

5. The computer-implemented method of claim 4, further comprising transmitting, by a processor, the audio signal to the remote device, wherein the remote device is configured to output the audio signal.

6. The computer-implemented method of claim 4, further comprising:

determining a word associated with the audio signal; and

determining the haptic effect based in part on the word.

7. The computer-implemented method of claim 1, further comprising transmitting, by a processor, an actuator signal associated with the haptic effect to an actuator.

8. The computer-implemented method of claim 1, further comprising generating, by a processor, a plurality of haptic effects based at least in part on the characteristics.

9. The computer-implemented method of claim 1, wherein receiving the input signal comprises receiving a biorhythm signal from a biosensor, and wherein the haptic effect is further based a characteristic of the biological information.

10. A non-transitory computer-readable medium encoded with processor executable program code, the computer-readable medium comprising:

program code for receiving a plurality of input signals, by a processor, the input signals associated with a plurality of input types, the input types including a voice, an environmental sound, or a biorhythm;

program code for determining characteristics of the input signals;

program code for converting an input signal to text;

program code for determining a haptic effect based at least in part on the characteristics; and

program code for transmitting a message to a remote device, wherein the message comprises;

the text, and

data associated with the haptic effect, the data configured to cause the remote device to output the haptic effect.

11. The non-transitory computer-readable medium of claim 10, wherein at least one of the characteristics comprises a frequency, an amplitude, or a duration.

12. The non-transitory computer-readable medium of claim 10, further comprising program code for transmitting an actuator signal associated with the haptic effect to an actuator.

13. The non-transitory computer-readable medium of claim 12, further comprising program code for transmitting the input signals and the actuator signal to a device configured to output the input signals and the actuator signal.

14. The non-transitory computer-readable medium of claim 10, further comprising:

program code for determining a word associated with the input signals; and

program code for determining the haptic effect based in part on the word.

15. The non-transitory computer-readable medium of claim 10, further comprising program code for generating a plurality of haptic effects based at least in part on the characteristics.

16. A system comprising:

a memory; and

a processor in communication with the memory and configured to;

receive a plurality of input signals, by a processor, the input signals associated with a plurality of input types, the input types including a voice, an environmental sound, or a biorhythm;

determine characteristics of the input signals;

convert an input signal to text; and

determine a haptic effect based at least in part on the characteristics of the input signals; and

transmit a message to a remote device, wherein the message comprises;

the text; and

data associated with the haptic effect, the data configured to cause the remote device to output the haptic effect.

17. The system of claim 16, wherein the processor is further configured to:

determine a word associated with the input signals; and

determine the haptic effect based in part on the word.

18. The system of claim 16, wherein the processor is further configured to generate a plurality of actuator signals based at least in part on the characteristic.

19. The system of claim 16, further comprising a microphone in communication with the processor, the microphone configured to:

receive a spoken text input;

generate an audio signal corresponding to the spoken text input; and

transmit the audio signal to the processor as the input signal.

20. The system of claim 16, further comprising an actuator in communication with the processor, the actuator configured to receive an actuator signal associated with the haptic effect and output the haptic effect.

21. The system of claim 16, further comprising a biosensor configured to determine a biorhythm signal and generate the input signal based at least in part on the biorhythm signal, and wherein the haptic effect is further based a characteristic of the biorhythm signal.

22. The system of claim 16, further comprising a housing for containing the memory, the processor, and the actuator.