Serverless peer-to-peer multi-party real-time audio communication system and method

• US 7,460,495 B2
• Filed: 02/23/2005
• Issued: 12/02/2008
• Est. Priority Date: 02/23/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Jin
• Primary Examiner(s): Nguyen; Chau
• Assistant Examiner(s): CHRISS, ANDREW W
• Application Number:
  US11/066,137
• Publication Number:
  US 20060187860A1
• Time in Patent Office: 1,378 Days
• Field of Search: 370/260
• US Class Current: 370/267
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/4046 with distributed floor control
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1076 Resource dissemination mechanisms or network resource keeping policies for optimal resource availability in the overlay network
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC

Abstract

A serverless peer-to-peer (P2P) multi-party real-time audio communication system and method in which each of the peers takes a turn mixing and compressing the audio and redelivering the compressed audio. An input audio stream is divided or split into frames. At each frame, one peer node is selected to mix and redeliver the audio to the remainder of the peers in the network. The number of frames mixed and redelivered by a certain peer is proportional to its available resources (such as the upload bandwidth or computational power). The P2P audio communication system and method flexibly balances the load of the peers, such that peers having more resources are able to assist peers having fewer resources. This enables the P2P audio communication system and method to conduct a multi-party audio communication session without the need for powerful servers or peers.

Claims

1. A method for a multi-party, real-time communication session using peer nodes in a peer-to-peer computer network, comprising:

dividing an input stream into a plurality of frames;

assigning each frame to the peer nodes such that a peer node receives a number of frames proportional to the available resources of the peer node; and

rotating mixing and redelivery of the plurality of frames on a frame-by-frame basis among at least some of the peer nodes to provide the multi-party real-time communication session.

2. The method as set forth in claim 1, wherein rotating mixing and redelivery of the plurality of frames further comprises mixing and redelivering at least part of the input stream in a round robin manner.

3. The method as set forth in claim 2, wherein the input stream is at least one of:

(a) an arriving audio stream containing audio content;

(b) an arriving microphone signal containing audio content, and further comprising mixing and redelivering the audio content.

4. The method as set forth in claim 3, wherein mixing further comprises:

performing one of the following processes to generate encoded audio coefficient rackets;

(a) entropy decoding and inverse quantizing of the arriving audio stream to generate encoded audio coefficient packets;

(b) transforming of the arriving microphone signal into audio coefficient blocks and encoding the audio coefficient blocks to generate encoded audio coefficient packets; and

combining the audio coefficient packets and blocks to generate a resultant coefficient block that is a combination of the coefficient packets and blocks for a certain frame.

5. The method as set forth in claim 4, further comprising:

quantizing the resultant coefficient block; and

re-encoding the quantized resultant coefficient block to generate a mixed audio packet.

6. The method as set forth in claim 5, further comprising:

decoding the mixed audio packet; and

inverse quantizing the decoded mixed audio packet to obtain mixed transform coefficients; and

applying an inverse transform to the mixed transform coefficients to generate an audio stream containing mixed audio content.

7. The method as set forth in claim 3, further comprising mixing and redelivering audio content from each of the peer nodes except from a certain peer node to which the audio stream and microphone signal is sent.

8. The method as set forth in claim 3, further comprising:

mixing and redelivering at least one of the following;

(a) the arriving audio stream containing audio content;

(b) the arriving microphone signal containing audio content; and

causing each peer node to subtract its own audio content from a mixed audio content to reduce echo.

9. The method as set forth in claim 1, further comprising:

defining a clique as the peer nodes that form a fully connected mesh; and

arranging the clique such that it serves as a super gateway node in the multi-party, real-time communication session.

10. The method as set forth in claim 1, further comprising:

defining a clique as the peer nodes that form a fully connected mesh; and

arranging the clique such that it serves as a super server in a star topology having additional nodes that act as client nodes in the multi-party, real-time communication session such that client nodes are exempted from the mixing task.

11. A computer-readable medium having computer-executable instructions for performing the computer-implemented method recited in claim 1.

12. A computer-implemented process for conducting a multi-party real-time audio communication session between peer nodes in a peer-to-peer computer network, each of the peer nodes having an audio stream, comprising:

a packet splitting step for splitting the audio stream from each of the peer nodes into audio packets;

an audio mixing step for combining audio packets from a certain time from each of the peer nodes to generate a mixed audio packet at that time; and

a rotation step for rotating audio mixing of mixed audio packets and redelivery of the mixed audio packets on a packet-by-packet basis among each of the peer nodes, such that at least some of the peer nodes have a turn performing audio mixing and redelivery.

13. The computer-implemented process as set forth in claim 12, further comprising rotating the audio mixing and redelivery between at least some of the peer nodes in a round robin manner.

14. The computer-implemented process as set forth in claim 13, wherein the audio mixing step further comprises:

encoding the audio packets using an entropy encoder to generate entropy- encoded audio packets; and

mixing on a packet-by-packet basis the entropy-encoded audio packets.

15. The computer-implemented process as set forth in claim 14, further comprising:

selecting entropy-encoded audio packets at the certain time from each of the peer nodes;

entropy decoding and inverse quantizing the selected audio packets to generate coefficient blocks corresponding to each of the selected audio packets;

combining the coefficient blocks to generate a resultant coefficient block for each of the peer nodes at the certain time; and

quantizing and entropy re-encoding the resultant coefficient block to produce the mixed audio packet.

16. The computer-implemented process as set forth in claim 12, further comprising:

entropy decoding and inverse quantizing the mixed audio packets at a receiving peer node to obtain mixed transform coefficients; and

inverse modified discrete cosine transforming the mixed transform coefficients to recover audio content.

17. The computer-implemented process as set forth in claim 16, wherein the audio content further comprises audio content from each of the peer nodes except a source peer node that is performing mixing and redelivery.

18. The computer-implemented process as set forth in claim 16, wherein the audio content includes audio content from each of the peer nodes, and further comprising having each of the peer nodes subtract out its own audio content.

19. The computer-implemented process as set forth in claim 12, further comprising:

fully connecting the peer nodes to define a clique; and

organizing a network topology such that the clique behaves as one of;

(a) a super gateway node that interacts with other nodes outside of the clique;

(b) a super server such that the clique is in a position of a server in a star network topology interacting with client nodes outside of the clique.

20. A peer-to-peer audio communication system for engaging in a multi-party real-time audio communication session between peer nodes in a peer-to-peer network, comprising:

an audio packet module that divides an input audio stream having audio content into a plurality of audio frames;

an audio mixer that performs mixing of the audio frames from each of the peer nodes at a particular frame to produce a mixed audio packet at each of the plurality of audio frames;

a round robin processing technique that rotates audio mixing and redelivery of the mixed audio content one at a time on a frame-by-frame basis to each of the peer nodes in a round robin manner; and

an audio decoder that decodes the redelivered mixed audio content to obtain audio for the audio communication session.