Enhanced Immersive Soundscapes Production

• US 20090238378A1
• Filed: 03/17/2009
• Published: 09/24/2009
• Est. Priority Date: 03/18/2008
• Status: Abandoned Application

• Current Assignee: Invism, Inc.
• Original Assignee: Invism, Inc.
• Inventors: Krishnan, Rajesh, Kikinis, Dan, Schmidt, Richard, Weyl, Stephen, Gourjian, Meher, Phelps, Russel H. III
• Application Number:
  US12/405,950
• Publication Number:
  US 20090238378A1
• Time in Patent Office: Days
• US Class Current: 381/92
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

An immersive audio-visual system (and a method) for creating an enhanced interactive and immersive audio-visual environment is disclosed. The immersive audio-visual environment enables participants to enjoy true interactive, immersive audio-visual reality experience in a variety of applications. The immersive audio-visual system comprises an immersive video system, an immersive audio system and an immersive audio-visual production system. The video system creates immersive stereoscopic videos that mix live videos, computer generated graphic images and human interactions with the system. The immersive audio system creates immersive sounds with each sound resource positioned correct with respect to the position of an associated participant in a video scene. The immersive audio-video production system produces an enhanced immersive audio and videos based on the generated immersive stereoscopic videos and immersive sounds. A variety of applications are enabled by the immersive audio-visual production including casino-type interactive gaming system and training system.

Claims

1. A computer method for determining location of a sound source for an immersive audio-visual production system having one or more of cameras and microphones, the method comprising:

recording sound on multiple sound tracks, each sound track being associated with one of the microphones;

collecting sound source information from the multiple sound tracks;

analyzing the collected sound source information; and

determining location of the sound source.

2. The method of claim 1, further comprising generating a first sound texture map based on the determined location of the sound source.

3. The method of claim 1, further comprising constructing a three-dimensional video model using the cameras.

4. The method of claim 3, wherein the three-dimensional video model contains one or more sound sources.

5. The method of claim 1, wherein the sound source information from a sound track of the multiple sound tracks comprises one or more sound waves from the microphone associated with the sound track, a sound wave containing information about a sound source.

6. The method of claim 5, wherein the information of a sound wave includes at least one of a group of distance between a sound source and the microphone, latency of the sound wave, delay of the sound wave and phase shift of the sound wave.

7. The method of claim 1, further comprising reconciling the three-dimensional video model with the location of the sound source.

8. The method of claim 1, further comprising:

adding sounds from sound sources not contained in the three-dimensional video model; and

determining location of the added sound source;

9. The method of claim 8, further comprising:

adding the location of the added sound source to the first sound texture map to generate a composite sound texture map.

10. A computer system for determining location of a sound source for an immersive audio-visual production, having one or more of cameras and microphones, the system comprising:

a recording module configured to record sound on multiple sound tracks, each sound track being associated with one of the microphones; and

an immersive sound processing module configured to;

collect sound source information from the multiple sound tracks;

analyze the collected sound source information; and

determined location of the sound source.

11. The system of claim 10, wherein the immersive sound processing module is further configured to generate a first sound texture map based on the determined location of the sound source.

12. The system of claim 10, further comprising an immersive video module configured to construct a three-dimensional video model using the the cameras.

13. The system of claim 12, wherein the three-dimensional video model contains one or more sound sources.

14. The system of claim 10, wherein the sound source information from a sound track of the multiple sound tracks comprises one or more sound waves from the microphone associated with the sound track, a sound wave containing information about a sound source.

15. The system of claim 14, wherein the information of a sound wave includes at least one of a group of distance between a sound source and the microphone, latency of the sound wave, delay of the sound wave and phase shift of the sound wave.

16. The system of claim 10, wherein the immersive sound processing module is further configured to reconcile the three-dimensional video model with the location of the sound source.

17. The system of claim 10, wherein the immersive sound processing module is further configured to:

add sounds from sound sources not contained in the three-dimensional video model; and

determine location of the added sound source;

18. The system of claim 17, the immersive sound processing module is further configured to add the location of the added sound source to the first sound texture map to generate a composite sound texture map.