Multi-channel data sonification system with partitioned timbre spaces and modulation techniques

• US 8,247,677 B2
• Filed: 06/17/2010
• Issued: 08/21/2012
• Est. Priority Date: 06/17/2010
• Status: Expired due to Fees

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: Lester F. Ludwig
• Inventors: Ludwig, Lester F.
• Primary Examiner(s): Warren, David S.
• Application Number:
  US12/817,196
• Publication Number:
  US 20110308376A1
• Time in Patent Office: 796 Days
• Field of Search: 84600-602, 846/09, 846/49, 846/22, 846/23, 846/59
• US Class Current: 84/622
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10H 1/06 Circuits for establishing the harmonic content of tones , or other arrangements for changing the tone colour
• CPC Class Codes: G10H 2210/301 Soundscape or sound field simulation, reproduction or control for musical purposes, e.g. surround or 3D sound; Granular synthesis
• CPC Class Codes: G10H 2220/351 Environmental parameters, e.g. temperature, ambient light, atmospheric pressure, humidity, used as input for musical purposes
• CPC Class Codes: G10H 2250/471 General musical sound synthesis principles, i.e. sound category-independent synthesis methods

Abstract

A data sonification system for representing a plurality of channels of numerical information is described. The data sonification system includes a plurality of audio waveform generator elements. Each of the audio waveform generator elements generates an associated audio frequency waveform. Each audio frequency waveform has an audio frequency parameter and at least one timbre modulation parameter having a settable value. The timbre modulation parameter affects the timbre of the audio waveform. The data sonification system includes a mapping element for associating aspects of multidimensional numerical data with the timbre modulation parameter of each audio frequency waveform. The mapping element sets the value of the timbre modulation parameter in response to multidimensional numerical data.

Claims

1. A data sonification system for representing a plurality of channels of numerical information via a plurality of corresponding discernable variations of timbre of at least one audio-frequency waveform, the audio-frequency waveform being perceivable by a user as an audio tone having a plurality of corresponding discernable timbre attributes, the system comprising:

a plurality of audio-frequency waveform generators, each generating an associated audio-frequency waveform, wherein each audio-frequency waveform comprises a fixed audio-frequency and at least one adjustable timbre control parameter, each timbre control parameter having an associated adjustable value which can be discernibly varied within a timbre space occupied by the plurality of audio-frequency waveforms; and

a mapping element for associating aspects of multidimensional numerical data with the at least one timbre control parameter of each of the audio-frequency waveforms from the plurality of audio-frequency waveforms,

wherein for the plurality of audio-frequency waveforms the mapping element adjusts the value of the timbre control parameter responsive to values of the multidimensional numerical data, and

wherein for the plurality of audio-frequency waveforms the timbre of the audio-frequency waveform carries information responsive to values of underlying data for presentation to a user.

2. The system of claim 1 wherein the at least one audio-frequency waveform comprises a plurality of timbre control parameters, each timbre control parameter affecting the timbre of that audio-frequency waveform;

and

wherein the mapping element adjusts the value of each timbre control parameter responsive to values of the multidimensional numerical data.

3. A data sonification method for representing a plurality of channels of numerical information via a plurality of corresponding discernable variations of timbre of at least one audio-frequency waveform, the audio-frequency waveform being perceivable by a user as an audio tone with a plurality of corresponding discernable timbre attributes, the method comprising:

generating plurality of audio-frequency waveforms, each audio-frequency waveform comprising an associated fixed audio-frequency and at least one adjustable timbre attribute responsive to an associated timbre control parameter, each timbre control parameter having an associated adjustable value which can be discernibly varied within a timbre space occupied by the plurality of audio-frequency waveforms;

mapping aspects of multidimensional numerical data with the at least one timbre control parameter of at least one of the audio-frequency waveforms from the plurality of audio-frequency waveforms, and

adjusting the value of the timbre control parameter responsive to values of the multidimensional numerical data,

wherein the timbre of the audio-frequency waveform carries information responsive to values of underlying data for presentation to a user.

4. The method of claim 3 wherein the values of the multidimensional numerical data are sequentially selected by the user from the multidimensional numerical data, the values sequentially selected over an interval of time.

5. The method of claim 3 wherein the values of the multidimensional numerical data are sequentially selected according to a trajectory selected through a subset of the multidimensional numerical data.

6. The method of claim 3 wherein the values of the multidimensional numerical data are sequentially selected responsive to a position on a trajectory specified by the user through a subset of the multidimensional numerical data, the trajectory rendered in a data visualization.

7. The method of claim 3 wherein the values of the multidimensional numerical data are sequentially selected responsive to a position of a cursor within a rendered data visualization.

8. The method of claim 3 wherein the multidimensional numerical data is retrieved from storage.

9. The method of claim 3 wherein the multidimensional numerical data is provided by a live real-time data stream.

10. The method of claim 3 wherein the multidimensional numerical data is created from live sensor data.

11. The method of claim 3 wherein the mapping comprises pre-sonification operations involving the imposing of indexing on selected data.

12. The method of claim 3 wherein the mapping comprises pre-sonification operations involving data suppression.

13. The method of claim 3 wherein the mapping comprises pre-sonification operations involving data value normalization.

14. The method of claim 3 wherein the mapping comprises pre-sonification operations involving linear transformations.

15. The method of claim 3 wherein the mapping comprises pre-sonification operations involving affine transformations.

16. The method of claim 3 wherein the mapping comprises pre-sonification signal processing filtering operations on selected data.

17. The method of claim 3 wherein the mapping comprises pre-sonification operations involving nonlinear transformations.

18. The method of claim 3 wherein the mapping comprises pre-sonification operations involving linear predictor operations.

19. The method of claim 3 wherein the at least one timbre control parameter controls the width of a pulse waveform.

20. The method of claim 3 wherein at least one audio-frequency waveform comprises a plurality of timbre control parameters, each timbre control parameter affecting the timbre of the audio-frequency waveform;

and

wherein the mapping adjusts the value of each of the plurality of timbre control parameters responsive to values of the multidimensional numerical data.