Method and system for controlling computer-generated virtual environment in response to audio signals

• US 5,513,129 A
• Filed: 07/14/1993
• Issued: 04/30/1996
• Est. Priority Date: 07/14/1993
• Status: Expired due to Term

• Current Assignee: Princeton Digital Image Corporation (Princeton Digital)
• Original Assignee: Fakespace, Incorporated
• Inventors: McDowall, Ian E., Bolas, Michael N., Bolas, Mark
• Primary Examiner(s): Ramirez, Ellis B.
• Application Number:
  US08/091,650
• Time in Patent Office: 1,021 Days
• Field of Search: 364/578, 364/514, 364/DIG. 1, 364/ DIG. 2, 360/14.2, 340/712, 358/81, 358/82, 345/156, 84/645, 84/610, 84/464 R, 84/601, 84/602, 84/609, 84/611, 84/634, 84/635, 84/641, 84/642, 84/DIG. 1, 84/DIG. 2, 84/DIG. 29, 395/152, 395/153, 395/154, 395/155, 395/156, 395/157, 395/159, 395/161, 395/200
• US Class Current: 703/13
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A method and apparatus for the control and manipulation of a virtual environment (such as virtual objects therein) in response to a music signal. The music is either interpreted directly to effect the control and manipulation. Alternatively, a control track corresponding to an audio signal (such as a music signal) is prerecorded, played back with the audio signal, and the control track is processed to control and manipulate the virtual world (or to control some other process of a computer system) as the audio signal is playing. In preferred embodiments, a computer creating a virtual world interprets the music, the control track, or both, and uses the resulting information to modify, create, and or control objects in the virtual environment. Preferred embodiments of the inventive system include apparatus for delaying input music to compensate for lag introduced by the system components, such as delay required to implement processing of control tracks corresponding to the input music.

Claims

1. A method for controlling production of a virtual environment by a virtual reality computer system, including the steps of:

(a) processing music signals to generate control signals having music and/or control information; and

(b) operating the virtual reality computer system in response to the control signals to generate said virtual environment.

2. The method of claim 1, wherein step (b) includes the step of:

producing a graphic display of the virtual environment on a display device of a type coupled to a head of a human user which provides an immersive visual experience of said virtual environment to the user.

3. The method of claim 2, wherein the graphic display is populated with at least one animated virtual object, where at least one characteristic of the virtual object changes in response to at least one of the music signals.

4. The method of claim 2, wherein the graphic display is a stereoscopic representation of a three-dimensional virtual environment.

5. A method for controlling production of a virtual environment by a virtual reality computer system, including the steps of:

(a) prerecording a control track having audio and/or control information corresponding to an audio signal; and

(b) operating the virtual reality computer system in response to said prerecorded control track to generate said virtual environment.

6. The method of claim 5, wherein step (b) includes the step of producing a graphic display of the virtual environment on a display device, and also including the steps of:

(c) supplying the audio signal to the virtual reality computer system; and

(d) operating the virtual reality computer system in response to both said audio signal and said prerecorded control track to generate said virtual environment.

7. The method of claim 6, wherein step (c) includes the step of supplying the audio signal to the virtual reality computer system with a first delay relative to the prerecorded control track, wherein the first delay is selected to enable generation of sounds in response to the audio signal in a manner so that the sounds have a desired time relationship to the graphic display.

8. The method of claim 5, wherein step (a) includes the step of automatically generating the control track by processing the audio signal.

9. The method of claim 5, wherein step (a) includes the step of manually operating an input device to generate the control track.

10. A method for controlling a computer system, including the steps of:

(a) prerecording a control track having audio and/or control information corresponding to an audio signal; and

(b) operating the computer system in response to said prerecorded control track.

11. The method of claim 10, also including the steps of:

(c) supplying the audio signal to the computer system; and

(d) operating the computer system in response to both the audio signal and the prerecorded control track.

12. A virtual reality computer system, including:

means for supplying a first signal selected from a group consisting of a control signal having music and/or control information generated in response to a music signal, a prerecorded control track having music and/or control information corresponding to the music signal, and a control signal having music and/or control information generated in response to the prerecorded control track; and

means for receiving the first signal and influencing action within a virtual environment in response to said first signal.

13. The apparatus of claim 12, wherein the means for supplying the first signal includes an analysis apparatus having means for receiving said music signal in digital or analog form, and processing said music signal to produce control information for modification of objects in the virtual environment.

14. The apparatus of claim 12, wherein said music signal is delayed in time to compensate for delays in other parts of the virtual reality computer system.

15. The apparatus of claim 12, wherein the means for supplying the first signal includes a music playing means for supplying said music signal.

16. A virtual reality computer system for producing a virtual environment, including:

means for prerecording a control track having music and/or control information corresponding to a music signal; and

means for producing the virtual environment in response to said prerecorded control track.

17. The system of claim 16, wherein the means for producing the virtual environment includes:

a display device; and

a means for producing a graphic display of the virtual environment on the display device.

18. The system of claim 16, also including:

means for supplying the music signal to the means for producing the virtual environment, and wherein the means for producing the virtual environment includes means for producing said virtual environment in response to both said music signal and said prerecorded control track.

19. Apparatus as in claim 16, wherein said control track contains additional information to that which can be extracted from the music signal.

20. The system of claim 16, wherein said control track is time shifted relative to the music signal to compensate for delays in said virtual reality computer system.

21. A virtual reality computer system, including:

a source of a music signal; and

an apparatus for extracting information from the music signal for modification of objects in a virtual environment.

22. A computer system, including:

means for prerecording a control track having audio and/or control information corresponding to an audio signal; and

a processor which receives the control track and which is programmed with software for operating the computer system in response to said control track.

23. The system of claim 22, also including:

means for supplying the audio signal to the processor, and wherein the processor is programmed with software for operating the computer system in response to both the audio signal and the control track.