Teleconferencing method and apparatus with three-dimensional sound positioning

• US 6,125,115 A
• Filed: 02/12/1998
• Issued: 09/26/2000
• Est. Priority Date: 02/12/1998
• Status: Expired due to Term

• Current Assignee: Dolby International AB (Dolby Laboratories Incorporated)
• Original Assignee: QSound Labs, Inc.
• Inventors: Smits, Gerard Dirk
• Primary Examiner(s): Kizou, Hassan
• Assistant Examiner(s): Pezzlo, John
• Application Number:
  US09/022,952
• Time in Patent Office: 957 Days
• Field of Search: 370/260, 370/261, 370/263, 370/265, 370/266, 370/267, 370/389, 709/204, 709/207, 709/205, 379/93.01, 381/17, 381/25, 381/310, 369/4
• US Class Current: 370/389
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04M 7/006 Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP) , including next generation networks with a packet-switched transport layer H04L65/00 takes precedence; aspects not specific to the type of network H04M3/00; special services in those networks H04M3/42
• CPC Class Codes: H04M 7/12 for working between exchanges having different types of switching equipment, e.g. power-driven and step by step or decimal and non-decimal
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A computer network teleconferencing system is provided in which audio signals are coupled to or associated with source indicators, such as source addresses in TCP protocol packets. Three-dimensional spatialization transforms are applied to audio data according to the source of such data so that, upon reproduction, a listener perceives different participants as being located in different spaced-apart locations within a 3-D spatialization region. Preferably, the user may select or modify the apparent location of the teleconference participants.

Claims

1. A process for teleconferencing over a computer network, the network having at least first and second audio sources and at least a first audio reproduction station with at least first and second speakers, each of said first and second audio sources outputting, over said computer network, audio information coupled to source identification information, the process comprising:

receiving, at said first audio reproduction station, first audio information and first source identification information from said first audio source, wherein said first source identification information is coupled to said first audio information such that said first audio information, when received at said first audio reproduction station, can be identified as coming from said first source, and reproducing first audio output at said first audio reproduction station, using said speakers, controlled, using said first source information in said first audio reproduction station, to simulate a location for said first audio information in a first three-dimensional location, using three-dimensional spatialization control; and

receiving, at said first audio reproduction station, second audio information and second source identification information from said second audio source, wherein said second source identification information is coupled to said second audio information such that said second audio information, when received at said first audio reproduction station, can be identified as coming from said second source, and reproducing second audio output at said first audio reproduction station, using said speakers, controlled, using said second source information, to simulate a location for said second audio information in a second three-dimensional location, different from said first location, using three-dimensional spatialization control, wherein a listener at said first audio reproduction station can identify the source of reproduced audio by its simulated location;

wherein at least one of said first and second three-dimensional locations is a location which is not along a line between said first and second speakers.

2. A process, as claimed in claim 1, said network also including a second audio reproduction station wherein said location for said first audio information and said location for said second audio information defines a first spatial relationship for said first audio information relative to said second audio information, at said first audio reproduction station, and further comprising:

receiving, at said second audio reproduction station, said first audio information and first source identification information from said first audio source, wherein said first audio information, when received at said second audio reproduction station, can be identified as coming from said first source, and reproducing first audio output at said second audio reproduction station, using speakers at said second audio reproduction station, controlled, using said first source information, to simulate a location for said first audio information in a third three-dimensional location, using three-dimensional spatialization control;

receiving, at said second audio reproduction station, second audio information and second source identification information from said second audio source, wherein said second source identification information, when received at said second audio reproduction station, can be identified as coming from said second source, and reproducing second audio output, using said speakers at said second audio reproduction station, controlled, using said second source information, to simulate a location for said second audio information in a fourth three-dimensional location, different from said first, second or third location, using three-dimensional spatialization control, wherein a listener at said second audio reproduction station can identify the source of reproduced audio by its simulated location; and

wherein said third location and said fourth location defines a second spatial relationship for said first audio information relative to said second audio information, at said second audio reproduction station, said second spatial relationship being different from said first spatial relationship such that said spatial relationship of said first audio information relative to said second audio information at said first audio reproduction station is different from said spatial relationship of said first audio information relative to said second audio information at said second audio reproduction station.

3. A process as claimed in claim 1 wherein each of said first and second audio information is a monaural audio signal, and wherein said system is provided in the absence of a need to send a stereo audio signal from either of said first and second audio sources to said audio reproduction station.

4. A process as claimed in claim 1 wherein said reproducing of said first and second audio information comprises applying different transforms to said first and second audio information to provide said simulated first and second locations and wherein said transforms are applied at said audio reproduction station in the absence of a need for spatial encoding of said first and second audio information at said first and second sources.

5. A process for teleconferencing over a computer network, the network having at least first and second audio sources and at least one audio reproduction station with at least first and second speakers, the process comprising:

outputting, from said first audio source, over said computer network, a first plurality of packets, each of said first plurality of packets containing first audio information and first source identification information,

outputting, from said second audio source over said computer network, a second plurality of packets, each of said second plurality of packets containing second audio information and second source identification information;

receiving, at said audio reproduction station, at least said first and second plurality of packets;

using said first source information in each of said first plurality of packets, in said audio reproduction station to reproduce said first audio information to simulate a first location for said reproduced first audio information in a first three-dimensional location, using three-dimensional spatialization control; and

using said second source information in each of said second plurality of packets, in said audio reproduction station, to reproduce said second audio information to simulate a second location for said reproduced second audio information in a second three-dimensional location, different from said first three-dimensional location, using three-dimensional spatialization control wherein a listener at said audio reproduction station can identify the source of reproduced audio by its simulated location;

wherein at least one of said first and second three-dimensional locations is a location which is not along a line between said first and second speakers.

6. A process, as claimed in claim 5, wherein said computer network is selected from among a local area network, an internet network and an intranet network.

7. A process, as claimed in claim 5, wherein said first and second locations are at least initially selected automatically at said audio reproduction station.

8. A process, as claimed in claim 5, further comprising selecting, via a user input device coupled to said audio reproduction station, at least one of said first and second locations within a three-dimensional space in the region of said first and second speakers.

9. A process, as claimed in claim 5, wherein said first and second locations may be selected independently of relative physical location of said first and second audio sources.

10. A process, as claimed in claim 5, further comprising sending audio information from said audio reproduction station for receipt and reproduction at at least one of said first and second audio sources substantially simultaneously with said steps of using to reproduce said first and second signals at said audio reproduction station.

11. A process, as claimed in claim 10, further comprising sending audio information from at least one location, different from said audio reproduction station, for receipt and reproduction at said one of said first and second audio sources to simulate a first location for reproduced audio from said at least one location in a first three-dimensional location and to simulate a second location for reproduced audio from said audio reproduction station in a second three-dimensional location.

12. A process as claimed in claim 5 wherein said first and second source identification information is used for both of first and second steps wherein said first step includes combining audio information from two or more packets with the same source identification information to reproduce audio information, and said second step includes simulating first and second three-dimensional locations for packets having first and second source identification information, respectively.

13. Apparatus for teleconferencing over a computer network, the network having at least first and second audio sources and at least one audio reproduction station with at least first and second speakers, said first audio source outputting, over said computer network, a first plurality of packets, each of said first plurality of packets containing first audio information and first source identification information, said second audio source outputting, over said computer network, a second plurality of packets, each of said second plurality of packets containing second audio information and second source identification information, the apparatus comprising:

means for receiving, at said audio reproduction station, at least said first and second plurality of packets;

means in said audio reproduction station, using said first source information in each of said first plurality of packets, for reproducing said first audio information to simulate a first location for said reproduced first audio information in a first three-dimensional location, using three-dimensional spatialization control; and

means, using said second source information in each of said second plurality of packets for reproducing said second audio information to simulate a second location for said reproduced second audio information in a second three-dimensional location, different from said first three-dimensional location, using three-dimensional spatialization control wherein a listener at said audio reproduction station can identify the source of reproduced audio by its simulated location;

wherein at least one of said first and second three-dimensional locations is a location which is not along a line between said first and second speakers.

14. Apparatus for teleconferencing over a computer network, the network having at least first and second audio sources and at least one audio reproduction station with at least first and second speakers, said first audio source outputting, over said computer network, a first plurality of packets, each of said first plurality of packets containing first audio information and first source identification information, said second audio source outputting over said computer network, a second plurality of packets, each of said second plurality of packets containing second audio information and second source identification information, the apparatus comprising:

a computer, at said audio reproduction station, coupled to said network to receive at least said first and second plurality of packets;

said computer being programmed to;

use said first source information in each of said first plurality of packets to control reproduction of said first audio information to simulate a first location for said reproduced first audio information in a first three-dimensional location, using three-dimensional spatialization control; and

to

use said second source information in each of said second plurality of packets to reproduce said second audio information to simulate a second location for said reproduced second audio information in a second three-dimensional location, different from said first three-dimensional location, using three-dimensional spatialization control;

wherein a listener at said audio reproduction station can identify the source of reproduced audio by its simulated location;

wherein at least one of said first and second three-dimensional locations is a location which is not along a line between said first and second speakers.

15. A software product, embodied in a computer-readable medium, for teleconferencing over a computer network, the network having at least first and second audio sources and at least one audio reproduction station with at least first and second speakers, said first audio source outputting, over said computer network, a first plurality of packets, each of said first plurality of packets containing first audio information and first source identification information, said second audio source outputting over said computer network, a second plurality of packets, each of said second plurality of packets containing second audio information and second source identification information, the software configured to control a computer at said audio reproduction station to:

receive at least said first and second plurality of packets;

use said first source information in each of said first plurality of packets to control reproduction at said first audio information to simulate a first location for said reproduced first audio information in a first three-dimensional location, using three-dimensional spatialization control; and

use said second source information in each of said second plurality of packets to reproduce said second audio information to simulate a second location for said reproduced second audio information in a second three-dimensional location, different from said first three-dimensional location, using three-dimensional spatialization control;

wherein a listener at said audio reproduction station can identify the source of reproduced audio by its simulated location;

wherein at least one of said first and second three-dimensional locations is a location which is not along a line between said first and second speakers.