Systems and methods for haptic augmentation of voice-to-text conversion

• US 9,275,653 B2
• Filed: 10/30/2014
• Issued: 03/01/2016
• Est. Priority Date: 10/29/2009
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Heubel, Robert W., Ramsay, Erin
• Primary Examiner(s): POPE, DARYL C
• Application Number:
  US14/528,398
• Publication Number:
  US 20150084752A1
• Time in Patent Office: 488 Days
• Field of Search: 340/407.1, 340/407.45
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: G10L 21/16 Transforming into a non-visible representation devices or methods enabling ear patients to replace direct auditory perception by another kind of perception A61F11/04
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters

Abstract

Systems and methods for providing voice-to-text haptic augmentation in a user interface are disclosed. For example, in one embodiment, a system for converting audible speech information to a haptic effect includes a microphone and a processor in communication with the microphone, the processor configured to receive an audio signal associated with a voice from the microphone, determine a characteristic of the audio signal, and generate an actuator signal based at least in part on the characteristic, the actuator signal configured to cause an actuator to output a haptic effect.

Claims

1. A computer-implemented method comprising:

receiving an audio input signal associated with a voice;

determining a characteristic associated with the audio input signal;

determining a word associated with the audio input signal; and

generating an actuator signal based at least in part on the characteristic and the word, the actuator signal configured to cause an actuator to output a first haptic effect.

2. The computer-implemented method of claim 1, wherein the characteristic comprises one of a frequency, an amplitude, or a duration.

3. The computer-implemented method of claim 1, further comprising:

transmitting the actuator signal to a remote device.

4. The computer-implemented method of claim 1, further comprising:

converting at least a portion of the audio input signal to text; and

transmitting the text and the actuator signal to a remote device via an e-mail, a SMS message, or in a chat message.

5. The computer-implemented method of claim 1, wherein generating the actuator signal comprises generating the actuator signal based at least in part on data in a haptic effect library.

6. The computer-implemented method of claim 1, wherein generating the actuator signal comprises:

determining a second haptic effect based at least in part on the characteristic;

determining a third haptic effect based at least in part on the word; and

generating the actuator signal based at least in part on the second haptic effect and the third haptic effect.

7. The computer-implemented method of claim 1, further comprising:

receiving a sensor signal from a biosensor; and

generating the actuator signal based at least in part on the sensor signal.

8. The computer-implemented method of claim 1, wherein the word comprises a plurality of words, and the actuator signal is generated based at least in part on at least two of the plurality of words.

9. The computer-implemented method of claim 1, wherein the first haptic effect comprises a plurality of pulses associated with a tune of a song.

10. The computer-implemented method of claim 1, wherein the first haptic effect comprises Morse Code.

11. A system comprising:

a memory; and

a processor in communication with the memory and configured to;

receive an audio input signal associated with a voice;

determine a characteristic associated with the audio input signal;

determine a word associated with the audio input signal; and

generate an actuator signal based at least in part on the characteristic and the word, the actuator signal configured to cause an actuator to output a first haptic effect.

12. The system of claim 11, wherein the characteristic comprises one of a frequency, an amplitude, or a duration.

13. The system of claim 11, wherein the processor is further configured to:

transmit the actuator signal to a remote device.

14. The system of claim 11, wherein the processor is further configured to:

convert the audio input signal to text; and

transmit the text and the actuator signal to a remote device via an e-mail, a SMS message, or in a chat message.

15. The system of claim 11, wherein the processor is further configured to generate the actuator signal based at least in part on data in a haptic effect library.

16. The system of claim 11, wherein the processor is further configured to perform the following to generate the actuator signal:

determine a second haptic effect based at least in part on the characteristic;

determine a third haptic effect based at least in part on the word; and

generate the actuator signal based at least in part on the second haptic effect and the third haptic effect.

17. The system of claim 11, further comprising a biosensor in communication with the processor, wherein the processor is further configured to:

receive a sensor signal from the biosensor; and

generate the actuator signal based at least in part on the sensor signal.

18. The system of claim 11, wherein the word comprises a plurality of words, and the processor is further configured to generate the actuator signal based at least in part on at least two of the plurality of words.

19. The system of claim 11, wherein the first haptic effect comprises a plurality of pulses associated with a tune of a song.

20. The system of claim 11, wherein the first haptic effect comprises Morse Code.

21. A computer-implemented method comprising:

receiving an audio input signal from a first user;

determining a first characteristic associated with the audio input signal;

receiving a sensor signal from a biosensor;

determining a second characteristic associated with the sensor signal; and

generating an actuator signal based at least in part on the first characteristic and the second characteristic, the actuator signal configured to cause an actuator to output a haptic effect.

22. The computer-implemented method of claim 21, wherein the audio input signal comprises environmental sound.

23. The computer-implemented method of claim 21, wherein the sensor signal is associated with at least one of a heart rate, a respiration rate, a body temperature, and a biorhythm.

24. The computer-implemented method of claim 21, wherein the first characteristic comprises a word, and wherein the actuator signal is generated based at least in part on a haptic effect library.

25. The computer-implemented method of claim 21, further comprising:

transmitting the actuator signal to a remote device.

26. The computer-implemented method of claim 25, wherein the remote device is within a predetermine range from the first user.

27. The computer-implemented method of claim 25, wherein the remote device is associated with a predetermined user or a predetermined group of users.

28. The computer-implemented method of claim 21, wherein the haptic effect is based at least in part on a task associated with the first user.

29. The computer-implemented method of claim 21, wherein the haptic effect is based at least in part on a status associated with the first user.

30. The computer-implemented method of claim 29, wherein the status is based at least in part on the sensor signal.

31. A system comprising:

a memory; and

a processor in communication with the memory and configured to;

receive an audio input signal from a first user;

determine a first characteristic associated with the audio input signal;

receive a sensor signal from a biosensor;

determine a second characteristic associated with the sensor signal; and

generate an actuator signal based at least in part on the first characteristic and the second characteristic, the actuator signal configured to cause an actuator to output a haptic effect.

32. The system of claim 31, wherein the audio input signal comprises environmental sound.

33. The system of claim 31, wherein the sensor signal is associated with at least one of a heart rate, respiration rate, body temperature, and biorhythm.

34. The system of claim 31, wherein the first characteristic comprises a word, and wherein the actuator signal is generated based at least in part on a haptic effect library.

35. The system of claim 31, wherein the processor is further configured to:

transmit the actuator signal to a remote device.

36. The system of claim 35, wherein the remote device is within a predetermine range from the first user.

37. The system of claim 35, wherein the remote device is associated with a predetermined user or a predetermined group of users.

38. The system of claim 31, wherein the haptic effect is based at least in part on a task associated with the first user.

39. The system of claim 31, wherein the haptic effect is based at least in part on a physical status associated with the first user.

40. The system of claim 39, wherein the physical status is based at least in part on the sensor signal.