Multi-party mesh conferencing with stream processing

• US 8,984,156 B2
• Filed: 07/20/2012
• Issued: 03/17/2015
• Est. Priority Date: 07/21/2011
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Spencer, Percy L.
• Primary Examiner(s): SERRAO, RANODHI N
• Application Number:
  US13/554,910
• Publication Number:
  US 20130191485A1
• Time in Patent Office: 970 Days
• Field of Search: None
• US Class Current: 709/231
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Conferencing between multiple parties can be performed with mesh conferencing. In one example, conference streams are received directly or indirectly from different client nodes at a mesh conferencing node. A priority is assigned to each stream before or after the streams are received. At least one stream is selected based on the assigned priorities to send to another node, such as a client node or a second mesh conferencing. node. Using the priorities, the streams may also be scaled or compressed before being sent to the next node.

Claims

1. A method comprising:

receiving a plurality of conference streams at a network interface of a mesh conferencing node of a multi-party mesh networking system at least one stream being received from a downstream client node and at least one stream being received from an upstream mesh conferencing node of the multi-party mesh networking system;

assigning a priority to each stream including receiving a priority assignment for at least some of the streams from a remote mesh conferencing node;

identifying each stream as either local or remote and sending the local streams to the upstream mesh conferencing node;

selecting a subset of the local streams to send to the upstream mesh conferencing node using the assigned priorities;

selecting at least one stream to send to the downstream client node based on the assigned priorities using a processor of the mesh conferencing node; and

sending the selected streams to the downstream client node from the network interface of the mesh conferencing node.

2. The method of claim 1, wherein selecting at least one stream comprises selecting a plurality of streams, the method further comprising multiplexing the selected streams before sending the selected streams to the client node.

3. The method of claim 2, further comprising scaling the selected streams based on the assigned priority before multiplexing the streams.

4. The method of claim 3, wherein scaling the streams comprises at least one of compressing audio, compressing video, reducing a frame rate of video, reducing an image size of video, reducing detail of an image, and transcoding video.

5. The method of claim 1, wherein selecting at least one stream to send to the client node comprises excluding the stream from the client node.

6. The method of claim 1, further comprising selecting at least one stream to send to a second mesh conferencing node based on the assigned priorities.

7. The method of claim 6, wherein selecting at least one stream to send to the upstream mesh conferencing node comprises excluding any stream received from the upstream mesh conferencing node.

8. The method of claim 1, wherein assigning a priority comprises assigning a priority based on at least one of the source of the stream, the activity of a user at the source of the stream, a designation of a user at the source of the stream as at least one of presenter, leader, panel member, and moderator, and recognition of a user at the source of the stream by a conference moderator.

9. A non-transitory machine-readable medium having instructions that when operated on by the machine cause the machine to perform operations comprising:

receiving a plurality of conference streams at a mesh conferencing node of a multi-party mesh networking system at least one stream being received from a downstream client node and at least one stream being received from an upstream mesh conferencing node of the multi-party mesh networking system;

assigning a priority to each stream including receiving a priority assignment for at least some of the streams from a remote mesh conferencing node;

identifying each stream as either local or remote and sending the local streams to the upstream mesh conferencing node;

selecting a subset of the local streams to send to the upstream mesh conferencing node using the assigned priorities;

selecting at least one stream to send to the downstream client node based on the assigned priorities; and

sending the selected streams to the downstream client node.

10. The medium of claim 9, wherein selecting at least one stream comprises selecting a plurality of streams, the operations further comprising multiplexing the selected streams before sending the selected streams to the client node.

11. The medium of claim 9, the operations further comprising scaling the streams by at least one of compressing audio, compressing video, reducing a frame rate of video, reducing an image size of video, reducing detail of an image, and transcoding video.

12. A mesh conferencing node of a multi-party mesh networking system coupled to a plurality of downstream client nodes and to a second upstream mesh conferencing node, the mesh conferencing node comprising:

a client network interface to receive a plurality of conference streams from the plurality of client nodes and a plurality of streams from the upstream mesh conferencing node;

a demultiplexer to demultiplex the received conference streams;

a prioritization table having priority assignments for each conference stream including priority assignments for at least some of the streams received from a remote mesh conferencing node;

a controller to identify each stream as either local or remote and to select a subset of the local streams to send to the upstream mesh conferencing node using the assigned priorities;

a multiplexer to multiplex the selected subset of demultiplexed conference streams based on priorities from the prioritization table; and

a mesh network interface to the second mesh conferencing node to send the multiplexed conference streams to the second mesh conferencing node.

13. The node of claim 12, wherein the mesh network interface is further to receive further conference streams from further client nodes, and wherein the multiplexer excludes any streams received from the second mesh conferencing node.

14. The node of claim 12, further comprising a client node multiplexer for each of the plurality of client nodes to multiplex conference streams for each client node based on the priority assignments and wherein the client network interface sends the respective multiplexed stream to each respective client node.

15. A mesh conferencing node of a multi-party mesh networking system comprising:

a client node interface coupled to at least two different downstream client nodes to receive a conference stream from each of the two different client nodes;

a demultiplexer to demultiplex each received conference stream into constituent components;

a controller to identify each stream as either local or remote and to select a subset of the local streams to send to the upstream mesh conferencing node using the assigned priorities;

a prioritization table having a priority for each client node including priority assignments for at least some of the streams received from a remote mesh conferencing node;

a scaler to scale the constituent components of each stream based on the priority of the respective client node;

a multiplexer to combine the scaled components to form a single output stream; and

a network interface to send the single output stream to a coupled upstream mesh conferencing node and to send the local streams to the upstream mesh conferencing node.

16. The node of claim 15, wherein the network interface is further to receive at least one multiplexed conference stream containing conference streams from a plurality of remote client nodes, the node further comprising a remote node demultiplexer to demultiplex the remote client node conference stream into constituent components.

17. The node of claim 16, further comprising an output multiplexer for each client node to each multiplex the scaled components and the remote client components to form a single output stream for each client node.

18. The node of claim 17, wherein each respective client output multiplexer excludes the scaled components received from its respective client node.

19. The node of claim 15, wherein the scaler is to alternately compress, block, scale, or pass each constituent component based on the priority of the respective client node.