System and method for displaying sound as vibrations

• US 9,078,065 B2
• Filed: 05/26/2010
• Issued: 07/07/2015
• Est. Priority Date: 05/27/2009
• Status: Active Grant

• Current Assignee: Maria Karam
• Original Assignee: Maria Karam
• Inventors: Karam, Maria, Fels, Deborah I., Russo, Frank A.
• Primary Examiner(s): Lao, Lun-See
• Application Number:
  US12/788,056
• Publication Number:
  US 20110129093A1
• Time in Patent Office: 1,868 Days
• Field of Search: 381/333, 381/388, 381/151, 381/124, 381/301, 381/18, 846/20, 846/32, 846/30, 844/54, 844/56, 297/217.3, 297/217.4, 601/49, 601/57, 601/70, 601/47, 601/149
• US Class Current: 1/1
• CPC Class Codes: G09B 21/04 Devices for conversing with the deaf-blind
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 1/24 Structural combinations of separate transducers or of two parts of the same transducer and responsive respectively to two or more frequency ranges
• CPC Class Codes: H04R 2201/023 Transducers incorporated in garment, rucksacks or the like
• CPC Class Codes: H04R 2400/03 Transducers capable of generating both sound as well as tactile vibration, e.g. as used in cellular phones
• CPC Class Codes: H04R 2499/15 Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 9/063 using a plurality of acoustic drivers H04R1/24 and H04R1/403 take precedence

Abstract

There is provided a method and a system for presenting audio signals as vibrotactile stimuli to the body in accordance with a Model Human Cochlea (MHC). Audio signals are obtained for presentation. The audio signals are separated into multiple bands of discrete frequency ranges that encompass the complete audio signal. Those signals are output to multiple vibrotactile devices. The vibrotactile devices may be positioned in a respective housing to intensify and constrain a vibrational energy from the vibrotactile devices. Output of the vibrotactile devices stimulate the cutaneous receptors of the skin at the locations where the vibrotactile devices are placed. In one embodiment, a system implements this method using audio voice coils that are embedded in a chair, which make contact with the human body while seated.

Claims

1. A method for presenting audio signals as tactile stimuli to a user comprising:

receiving at least one source audio signal to be presented to the user; and

in accordance with a sensory substitution model, outputting respective output signals comprising the entire range of frequencies of the at least one source audio signal for driving four or more output channels of a vibrotactile display wherein each output channel is presented via at least one vibrotactile device positioned to provide tactile stimuli directly or indirectly to cutaneous mechanoreceptors of the user'"'"'s body to emulate stimulation of hair cells of the human cochlea such that perception of hearing experienced by a hearing person through the person'"'"'s ear is emulated on the user'"'"'s body.

2. The method of claim 1 wherein the sensory substitution model comprises a frequency model, wherein the at least one audio source signal, comprising a spectrum of frequencies, is separated into four or more distinct frequency bands to define output signals across the full spectrum of human hearing in order to emulate the perception of hearing through the ear on the user'"'"'s body.

3. The method of claim 1 wherein the sensory substitution model comprises a track model and further wherein at least one audio source signal comprises four or more audio tracks separated in response to the respective tracks to define the output signals across the full spectrum of human hearing in order to emulate the perception of hearing through the ear on the user'"'"'s body.

4. The method of claim 2 wherein the vibrotactile devices comprise four or more such devices arranged in a linear order in accordance with the sensory substitution model so as to align with perception of pitch by associated locations on the skin in order to emulate the perception of hearing through the user'"'"'s body.

5. The method of claim 4 wherein the vibrotactile devices are positioned in a chair having a seat and a back and wherein output signals for the highest frequencies are directed to vibrotactile devices placed at the base of a seat and output signals for lower frequencies being directed to vibrotactile devices placed progressively higher on the back of the seat in order to emulate the perception of hearing through the ear on the user'"'"'s body.

6. The method of claim 1 wherein the vibrotactile devices are positioned using a form factor comprising clothing or belts in order to emulate the perception of hearing as experienced through the ear on the user'"'"'s body.

7. The method of claim 1 wherein any respective output channel is provided by one or more vibrotactile devices.

8. The method of claim 1 wherein the vibrotactile devices are responsive to amplitude and frequency components of the output signals in order to emulate the perception of hearing through the ear on the user'"'"'s body.

9. The method of claim 1 wherein the vibrotactile devices are positioned in a respective housing to intensify and constrain a vibrational energy from the vibrotactile devices.

10. The method of claim 1 wherein at least some of the vibrotactile devices comprise voice coils for each of the four or more output channels, each channel outputting respective frequency bands of the at least one audio source signal such that the voice coils produce a combined sound that matches the audio source signal.

11. The method of claim 1 wherein the at least one audio signal comprises an analog or digital signal representing one or more of speech, music, soundtracks from either previously recorded or live sources.

12. The method of claim 1 comprising separating the at least one audio source signal in accordance with the sensory substitution model to define the output signals.

13. A computing device for presenting audio signals as tactile stimuli to a user, said device comprising a processor and memory coupled thereto, said memory storing instructions for configuring the processor to:

receive at least one source audio signal to be presented to the user; and

in accordance with a sensory substitution model, output respective output signals comprising the entire range of frequencies of the at least one source audio signal for driving four or more output channels of a vibrotactile display wherein each output channel is presented via at least one vibrotactile device positioned to provide tactile stimuli directly or indirectly to cutaneous mechanoreceptors of the user'"'"'s body to emulate stimulation of hair cells of the human cochlea such that perception of hearing experienced by a hearing person through the person'"'"'s ear is emulated on the user'"'"'s body.

14. The computing device of claim 13 wherein the processor is configured to separate the at least one audio source signal in accordance with the sensory substitution model to define the output signals.

15. The computing device of claim 14 wherein the sensory substitution model comprises a frequency model, wherein the at least one audio source signal, comprising a spectrum of frequencies, is separated into four or more distinct frequency bands to define output signals across the full spectrum of human hearing in order to emulate the perception of hearing through the ear on the user'"'"'s body.

16. The computing device of claim 14 wherein the sensory substitution model comprises a track model and further wherein at least one audio source signal comprises four or more audio tracks separated in response to the respective tracts to define the output signals across the full spectrum of human hearing in order to emulate the perception of hearing through the ear on the user'"'"'s body.

17. A computer program product comprising a computer memory storing computer readable instructions for presenting audio signals as tactile stimuli to a user, which instructions when executed by a computer processor configure the processor to:

receive at least one source audio signal to be presented to the user; and

in accordance with a sensory substitution model, output respective output signals comprising the entire range of frequencies of the at least one source audio signal for driving four or more output channels of a vibrotactile display wherein each output channel is presented via at least one vibrotactile device positioned to provide tactile stimuli directly or indirectly to cutaneous mechanoreceptors of the user'"'"'s body to emulate stimulation of hair cells of the human cochlea such that perception of hearing experienced by a hearing person through the person'"'"'s ear is emulated on the user'"'"'s body.

18. A system for presenting audio signals as tactile stimuli to a user comprising:

a signal processing sub-system having an output for output signals to drive four or more output channels in accordance with a sensory substitution model, said output signals comprising the entire range of frequencies of at least one source audio signal to be presented to the user; and

a vibrotactile display configured to receive the output signals and comprising the four or more output channels, wherein each output channel is presented via at least one vibrotactile device, each vibrotactile device configured for positioning to provide tactile stimuli directly or indirectly to cutaneous mechanoreceptors of the user'"'"'s body to emulate stimulation of hair cells of the human cochlea such that perception of hearing experienced by a hearing person through the person'"'"'s ear is emulated on the user'"'"'s body.

19. The system of claim 18 wherein the sensory substitution model comprises a frequency model, wherein the at least one audio source signal, comprising a spectrum of frequencies, is separated into four or more distinct frequency bands to define to output signals across the full spectrum of human hearing in order to emulate the perception of hearing through the ear on the user'"'"'s body.

20. The system of claim 18 wherein the sensory substitution model comprises a track model and further wherein the at least one audio source signal comprises four or more audio tracks separated in response to the respective tracts to define the output signals across the full spectrum of human hearing in order to emulate the perception of hearing through the ear on the user'"'"'s body.

21. The system of claim 18 comprising an audio interface for receiving at least one source audio signal to be presented to the user.

22. The system of claim 18 comprising a computing device for processing the at least one source audio signal.

23. The system of claim 18 wherein the signal processing sub-system comprises a plurality band-pass filters.

24. The system of claim 19 wherein the vibrotactile devices are positioned in a chair having a seat and a back and wherein output signals for the highest frequencies are directed to vibrotactile devices placed at the base of a seat and output signals for lower frequencies being directed to vibrotactile devices placed progressively higher on the back of the seat in order to emulate the perception of hearing through the ear on the user'"'"'s body.

25. The system of claim 19 wherein the vibrotactile devices comprise four or more such devices arranged in a linear order in accordance with the sensory substitution model so as to align with perception of pitch by associated locations on the skin in order to emulate the perception of hearing through the user'"'"'s body.

26. The system of claim 18 wherein the vibrotactile devices of the vibrotactile display are positioned using a form factor comprising clothing or belts in order to emulate the perception of hearing as experienced through the ear on the user'"'"'s body.

27. The system of claim 18 wherein any respective output channel is provided by one or more vibrotactile devices.

28. The system of claim 18 wherein the vibrotactile devices are positioned in a respective housing to intensify and constrain a vibrational energy from the vibrotactile devices.

29. The system of claim 18 wherein the vibrotactile devices are responsive to amplitude and frequency components of the output signals.

30. The system of claim 18 wherein at least some of the vibrotactile devices comprise voice coils for each of the four or more output channels, each channel outputting respective frequency bands of the at least one audio source signal such that the voice coils produce a combined sound that matches the audio source signal.