AUDIO TRANSFORMS IN CONNECTION WITH MULTIPARTY COMMUNICATION

• US 20100195812A1
• Filed: 02/05/2009
• Published: 08/05/2010
• Est. Priority Date: 02/05/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Garms, Jason, Buxton, William, Quinn, Kori M., Chou, Phillip A., Huitema, Christian, Zhang, Zhengyou, Acero, Alejandro, Cutler, Ross G., Rosenfeld, Daniel Allen, Florencio, Dinei A.
• Granted Patent: US 8,340,267 B2
• Time in Patent Office: Days
• US Class Current: 379/202.01
• CPC Class Codes: G10L 2021/0135 Voice conversion or morphing
• CPC Class Codes: H04M 3/56 Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities video conference systems H04N7/15
• CPC Class Codes: H04M 3/565 relating to time schedule aspects

Abstract

The claimed subject matter relates to an architecture that can preprocess audio portions of communications in order to enrich multiparty communication sessions or environments. In particular, the architecture can provide both a public channel for public communications that are received by substantially all connected parties and can further provide a private channel for private communications that are received by a selected subset of all connected parties. Most particularly, the architecture can apply an audio transform to communications that occur during the multiparty communication session based upon a target audience of the communication. By way of illustration, the architecture can apply a whisper transform to private communications, an emotion transform based upon relationships, an ambience or spatial transform based upon physical locations, or a pace transform based upon lack of presence.

Claims

1. A computer implemented system that preprocesses audio portions of communication to enrich multiparty communication environments, comprising:

a connection component that hosts or joins a multiparty communication session in which at least one party is a remote party;

an interface component that provides a public channel and further provides a private channel; and

an audio transformation component that applies an audio transform to an audio portion of a communication over the public channel or the private channel in order to preprocesses the audio portion prior to presentation or transmission of the communication by the interface component, the transform applied is based upon a target audience of the communication.

2. The system of claim 1, the multiparty communication session is a telepresence session in which the remote party maintains a presence in a host environment.

3. The system of claim 1, the public channel presents or transmits a public communication received by substantially all connected parties.

4. The system of claim 1, the private channel presents or transmits a private communication received by a selected subset of all connected parties.

5. The system of claim 1, the audio transform is a whisper transform.

6. The system of claim 5, the whisper transform employs speech processing techniques to convert normal speech input to the communication into simulated whispered speech.

7. The system of claim 5, the audio transformation component applies the whisper transform to a private communication propagated over the private channel and, optionally, precedes the private communication with a tone or melody indicative of private communication, the private channel is delivered to one or more devices associated with a select subset of all connected parties, employs audio beaming techniques to broadcast the private communication to a limited physical area associated with one or more members of the select subset, or combinations thereof.

8. The system of claim 1, the audio transform is an emotion transform that employs speech processing techniques to dampen, modify, or remove speech characteristics associated with an undesired emotion or behavior.

9. The system of claim 8, the audio transformation component applies the emotion transform selectively to a public communication or selectively to a private communication.

10. The system of claim 1, the audio transform is an ambience transform.

11. The system of claim 10, the ambience transform employs speech or audio processing techniques to at least one of (1) insert, substitute, or interpolate target ambient acoustic features;

(2) insert, substitute, or interpolate host environment ambient acoustic features;

or (3) dampen, modify, or remove source ambient acoustic features.

12. The system of claim 10, the audio transformation component applies the ambience transform based upon a relative geography between the target audience and a host environment or a communication source.

13. The system of claim 1, the audio transform is a pace transform that employs speech processing techniques to simulate normal speech pitch or frequency characteristics for communications output at a faster-than-normal playback rate.

14. The system of claim 13, further comprising a recall component that records the multiparty communication session and that automatically replays a relevant portion of the communication session by way of a playback channel provided by the interface component, the relevant portion is characterized by a lack of presence and is output at a faster-than-normal playback rate with application of the pace transform.

15. The system of claim 1, the audio transform is a spatial transform that employs speech or audio processing techniques to provide audio three-dimensional (3D) stereo features based upon a location map and a device array map.

16. The system of claim 15, further comprising a mapping component that constructs the device array map based upon respective locations associated with local audio output devices;

and that further constructs or receives the location map that is constructed based upon acoustic characteristics of a host environment and positions within a model of the host environment virtually assigned to each party in the multiparty communication session.

17. A method for enriching multiparty communication environments based upon acoustic preprocessing, comprising:

facilitating a multiparty communication session for communicatively connecting at least three participants;

providing a public channel for presenting or transmitting a public communication received by substantially all connected participants;

providing a private channel for presenting or transmitting a private communication received by a select subset of all connected participants; and

employing a processor for applying an audio transform to an audio portion of a communication propagated by way of the public channel or the private channel based upon a target audience of the communication.

18. The method of claim 17, further comprising at least one of the following acts:

applying a whisper transform when the target audience is limited to a select subset of all connected participants, and, optionally, preceding the whisper transform with a tone or melody indicative of selective communication;

applying the whisper transform to private communications directed to a single participant by delivering the private communication to devices associated with the select subset or beaming the private communication to a physical area associated with one or more member of the select subset;

employing speech processing techniques for converting normal speech input into output resembling corresponding whispered speech vocalization in order to effectuate the whisper transform;

implementing the multiparty communication session as a telepresence session based upon a host setting;

constructing a location map based upon acoustic features associated with the host setting and virtual position assignments of each participant of the communication session mapped to a 3D model of the host setting;

constructing a device array map based upon respective locations associated with audio output devices included in a remote setting;

or

applying a spatial transform for simulating spatial acoustics consistent with the 3D model when the target audience is remote from the host setting.

19. The method of claim 17, further comprising at least one of the following acts:

applying an emotion transform when an identified vocal characteristic associated with an emotion or behavior is deemed inappropriate for the target audience;

employing speech processing techniques for suppressing or changing the identified vocal characteristic in order to effectuate the emotion transform;

applying an ambience transform for suppressing or mimicking aspects of ambience when the target audience is remote from a host setting;

or

applying a pace transform for simulating normal speech pitch or frequency characteristics during rapid playback of recorded communications when the target audience is associated with missing presence during the communication session.

20. A computer implemented system that preprocesses audio streams with a whisper transform in order to enrich multiparty communication environments, comprising:

a connection component that hosts a multiparty communication session in which at least one party is a remote party;

an interface component that maintains a public channel that propagates public communication to substantially all connected parties; and

that further maintains a private channel that propagates private communication to a selected subset of all connected parties; and

an audio transformation component that applies a whisper transform to an audio portion of the private communication, the whisper transform employs speech processing techniques to convert normal speech input into simulated whispered output that corresponds to the normal speech.