System and methods for simulating real-time multisensory output

• US 9,898,249 B2
• Filed: 10/08/2013
• Issued: 02/20/2018
• Est. Priority Date: 10/08/2012
• Status: Active Grant

• Current Assignee: UNM Rainforest Innovations (f/k/a STC.UNM) (The University of New Mexico)
• Original Assignee: UNM Rainforest Innovations (f/k/a STC.UNM) (The University of New Mexico)
• Inventors: Panaiotis
• Primary Examiner(s): Fletcher, Marlon
• Application Number:
  US14/433,146
• Publication Number:
  US 20150268926A1
• Time in Patent Office: 1,596 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 8/38 for implementing user interfaces

Abstract

The invention is directed to a system and methods allowing a user to interact with one or more data applications—including for example, games, music, music composition and tactile stimulation—to simulate real-time multisensory output. The invention includes a music program, a music texture program and a simulation program.

Claims

1. A computer system for generating real-time multisensory output, comprising:

a user interface comprising one or more input elements;

a processor comprising a music texture program and a music program, wherein the music texture program uses an algorithm to generate a first audio output and the music program uses input received from the one or more input elements to generate real-time a second audio output;

a baton component configured to generate a pulse for broadcast to the other components as a trigger to perform additional processing;

a tonic cycler component configured to generate a pitch-class of a scale for the first audio output;

an ensemble component configured to generate a sequence of notes or chords for the first audio output;

a tuning component that produces a note value and microtonal offset for the first audio output, wherein the tuning component includes a conversion algorithm that converts pitch to frequency for the first output; and

an output component that simultaneously outputs the first audio output and the second audio output.

2. The computer system for generating real-time multisensory output according to claim 1, wherein the one or more input elements of the user interface is a grid comprising one or more octagons and one or more diamonds.

3. The computer system for generating real-time multisensory output according to claim 1 further comprising a simulation program comprising:

a design component configured to define curves and path descriptions of a visual output;

an initialization component configured to calculate and create animation data structures of the visual output; and

a runtime component configured to animate the visual output.

4. The computer system for generating real-time multisensory output according to claim 1 further comprising a coordinating program that integrates the first audio output of the music texture program with the second audio output of the music program.

5. The computer system for generating real-time multisensory output according to claim 3 further comprising a coordinating program that integrates the first audio output of the music texture program with the visual output of the simulation program.

6. The computer system for generating real-time multisensory output according to claim 3 further comprising a coordinating program that integrates the second audio output of the music program with the visual output of the simulation program.

7. The computer system for generating real-time multisensory output according to claim 1, wherein the coordinating program monitors user activity and coordinates with the music texture program to output a data stream.

8. The computer system for generating real-time multisensory output according to claim 7, wherein the data stream comprises a visual output.

9. The computer system for generating real-time multisensory output according to claim 1 further comprising an external source providing a data stream that is coordinated with one or more of the first output and the second output.

10. A computer system for generating real-time multisensory output, comprising:

a user interface comprising one or more input elements;

a processor comprising a simulation program, wherein the simulation program uses an algorithm to generate a visual output coordinated with music, the simulation program including;

a design component configured to define curves and path descriptions of the visual output coordinated with music,

an initialization component configured to calculate and create animation data structures of the visual output coordinated with music, wherein the animation data structures comprise a node path and trigger emitters that activate particles to follow the node paths; and

a runtime component configured to animate the visual output coordinated with music,

an output component that outputs the visual output coordinated with music.

11. The computer system for generating real-time multisensory output according to claim 10 further comprising a music texture program configured to use an algorithm to generate a first audio output, wherein the output component simultaneously outputs the first audio output and the visual output.

12. The computer system for generating real-time multisensory output according to claim 11 further comprising a music program configured to use the input received from the one or more input elements to generate real-time a second audio output, wherein the output component simultaneously outputs the first audio output, the second audio output, and the visual output.

13. The computer system for generating real-time multisensory output according to claim 10 further comprising a coordinating program, wherein the coordinating program monitors user activity and coordinates with the music texture program to output a data stream.

14. The computer system for generating real-time multisensory output according to claim 13, wherein the data stream comprises a visual output.

15. The computer system for generating real-time multisensory output according to claim 10 further comprising an external source providing a data stream that is coordinated with one or more of the first output and the second output.

16. A computer system for generating real-time multisensory output, comprising:

a user interface comprising one or more input elements;

a processor comprising a music texture program and a music program, wherein the music texture program uses an algorithm to generate a first output and the music program uses input received from the one or more input elements to generate real-time a second output;

a baton component configured to generate a pulse for broadcast to the other components as a trigger to perform additional processing;

a tonic cycler component configured to generate a pitch-class of a scale for the first output;

an ensemble component configured to generate a sequence of notes or chords for the first output;

a tuning component that produces a note value and microtonal offset for the first output, wherein the tuning component includes a conversion algorithm that converts pitch to frequency for the first output; and

an output component that simultaneously outputs the first output and the second output.

17. The computer system for generating real-time multisensory output according to claim 16 wherein one of the first output and the second output is a visual output.

18. The computer system for generating real-time multisensory output according to claim 16 wherein one of the first output and the second output is an audio output.

19. The computer system for generating real-time multisensory output according to claim 16 further comprising:

a runtime component configured to animate one of the first output and the second output as visual output; and

an output component that outputs the visual output.

20. The computer system for generating real-time multisensory output according to claim 16 further comprising:

an output component that outputs the one of the first output and the second output as audio output.

21. The computer system for generating real-time multisensory output according to claim 16 further comprising:

a design component configured to define curves and path descriptions of a visual output;

an initialization component configured to calculate and create animation data structures of the visual output;

a runtime component configured to animate the visual output; and

an output component that outputs the visual output.

22. The computer system for generating real-time multisensory output according to claim 16 further comprising a coordinating program that monitors user activity and coordinates with the music texture program to output a data stream.

23. The computer system for generating real-time multisensory output according to claim 22, wherein the data stream comprises a visual output.

24. The computer system for generating real-time multisensory output according to claim 16 further comprising an external source providing a data stream that is coordinated with one or more of the first output and the second output.