Distributed real-time media composer

• US 8,289,369 B2
• Filed: 07/02/2009
• Issued: 10/16/2012
• Est. Priority Date: 11/14/2003
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Johansen, Tom-Ivar, Sandbakken, Geir Arne
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US12/497,316
• Publication Number:
  US 20100033550A1
• Time in Patent Office: 1,202 Days
• Field of Search: 348 1401- 1416, 370260-261
• US Class Current: 348/14.09
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04M 3/562 where the conference facilities are distributed
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04N 7/14 Systems for two-way working H04N7/12, H04N7/173 take precedence
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A system and a method allowing simultaneous exchange of audio, video or data information between a plurality of units over a communication network, supported by a central unit, wherein the central unit is, based on knowledge regarding one or more of the units, adapted to instruct said one or more units to generate multimedia data streams adjusted to fit into certain restrictions to be presented on other units participating in a same session.

Claims

1. A controller for videoconferencing, comprising:

a communications interface through which command signals are routed to a plurality of endpoints over a network;

a capabilities module that exchanges information with the plurality of endpoints to determine encoding capabilities of said plurality of endpoints;

a command signal generator configured to generate command signals to be sent to said plurality of endpoints via said communications interface, said command signals instructing said plurality of endpoints to establish sessions between respective of said plurality of endpoints, said sessions being compatible with the encoding capabilities of the plurality of endpoints so that encoding is performed at the endpoints but not in the controller; and

a processor that forms a combined frame from a first partial frame sent from a first endpoint and a second partial frame sent from a second endpoint, said processor configured to operate in a robust mode of operation when an observed packet loss rate from a stream from said first endpoint is greater than a predetermined level.

2. The controller of claim 1, further comprising:

a processor being programmed to implement said capabilities module.

3. The controller of claim 2, wherein said processor is disposed in a personal computer.

4. The controller of claim 1, wherein

the command signal generator instructs each endpoint to encode a stream such that said controller does not also perform encoding on said stream.

5. The controller of claim 1, wherein:

said command signal generator generates command signals requesting the plurality of endpoints to generate streams with partial frames.

6. The controller of claim 1, wherein:

said controller forms a combined frame using a partial frame sent from an endpoint in a stream encoded at the endpoint in a scalable format.

7. The controller of claim 1, further comprising:

a multipoint controller configured to receive partial frames from at least 3 endpoints; and

a processor programmed to form a combined frame from the partial frames of the at least 3 endpoints.

8. The controller of claim 1, further comprising:

a layout generator that identifies a combined frame layout having at least two partial frames, wherein

said capabilities module is configured to determine a capability of a first endpoint, and said layout generator defines said combined frame layout to include a partial frame that is consistent with a capability of the first endpoint, and

said command signal generator is configured to generate a command to be sent to a second endpoint to generate the partial frame that is consistent with the capability of the first endpoint.

9. The controller of claim 1, wherein said controller is a router.

10. The controller of claim 1, wherein said controller is hosted as an endpoint and said sessions are established between endpoints without a central controller.

11. The controller of claim 1, wherein:

when said processor operates in the robust mode of operation the processor selects a different resolution from said stream from said first endpoint than when not in said robust mode of operation.

12. A method for controlling videoconference between endpoints, comprising:

generating command signals and sending command signals through an interface of a controller to a plurality of endpoints over a network;

exchanging information via a capabilities module of the controller with the plurality of endpoints to determine encoding capabilities of said plurality of endpoints;

instructing with said command signals said plurality of endpoints to establish sessions between respective of said plurality of endpoints, said sessions being compatible with the encoding capabilities of the plurality of endpoints; and

conducting the videoconference between the endpoints using encoded video streams generated at the endpoints without the controller also encoding the video streams,

wherein said conducting step includes conducting the videoconference by forming with the controller a combined frame from a first partial frame sent from a first endpoint and a second partial frame sent from a second endpoint, said controller operating in a robust mode of operation when an observed packet loss rate from a stream from said first endpoint is greater than a predetermined level.

13. The method of claim 12, wherein:

said instructing step includes instructing an endpoint to encode a stream such that said controller does not also perform encoding on said stream.

14. The method of claim 12, wherein:

said generating step includes generating command signals requesting the plurality of endpoints to generate streams for partial frames.

15. The method of claim 12, further comprising:

forming at the controller a combined frame from using a partial frame sent from an endpoint in a stream encoded at the endpoint in a scalable format.

16. The method of claim 12, further comprising:

identifying at the controller a combined frame layout having at least two partial frames, wherein

said exchanging step includes determining a capability of a first endpoint, and

said identifying step defines said combined frame layout to include a partial frame that is consistent with the capability of the first endpoint, and

said generating step generates a command to be sent to a second endpoint to generate the partial frame that is consistent with the capability of the first endpoint.

17. The method of claim 12, wherein said controller is a router.

18. The method of claim 12, wherein said conducting step includes conducting the videoconference between endpoints without a central controller.

19. The method of claim 12, wherein when said controller operates in the robust mode of operation the controller selecting a different resolution from said stream from said first endpoint than when not in said robust mode of operation.

20. A non-transitory computer readable storage medium encoded with computer readable instructions that when executed by a processor perform steps for establishing a videoconference comprising:

generating command signals and sending command signals through an interface of a controller to a plurality of endpoints over a network;

exchanging information via a capabilities module of the controller with the plurality of endpoints to determine encoding capabilities of said plurality of endpoints;

instructing with said command signals said plurality of endpoints to establish sessions between respective of said plurality of endpoints, said sessions being compatible with the encoding capabilities of the plurality of endpoints; and

conducting the videoconference between the endpoints using encoded video streams generated at the endpoints without the controller also encoding the video streams,

wherein said conducting step includes conducting the videoconference by forming with the controller a combined frame from a first partial frame sent from a first endpoint and a second partial frame sent from a second endpoint, said controller operating in a robust mode of operation when an observed packet loss rate from a stream from said first endpoint is greater than a predetermined level.

21. The computer program product of claim 20, wherein:

said instructing step includes instructing each endpoint to encode a stream such that said controller does not also perform encoding on said stream.

22. The computer program product of claim 21, wherein:

said generating step includes generating command signals requesting the plurality of endpoints to generate streams for partial frames.

23. The computer program product of claim 20, further comprising:

forming at the controller a combined frame from using a partial frame sent from an endpoint in a stream encoded at the endpoint in a scalable format.

24. The computer program product of claim 20, further comprising:

identifying at the controller a combined frame layout having at least two partial frames, wherein

said exchanging step includes determining a capability of a first endpoint, and

said identifying step defines said combined frame layout to include a partial frame that is consistent with the capability of the first endpoint, and

said generating step generates a command to be sent to a second endpoint to generate the partial frame that is consistent with the capability of the first endpoint.

25. The computer program product of claim 20, wherein said controller is a router.

26. The computer program product of claim 20, wherein said conducting step includes conducting the videoconference between endpoints without a central controller.

27. The computer program product of claim 20, wherein when said controller operates in the robust mode of operation the processor selecting a different resolution from said stream from said first endpoint than when not in said robust mode of operation.

28. An apparatus for controlling a videoconference between endpoints, comprising:

means for generating command signals and sending command signals through an interface of a controller to a plurality of endpoints over a network;

means for exchanging information in a capabilities module of the controller with the plurality of endpoints to determine encoding capabilities of said plurality of endpoints;

means for instructing with said command signals said plurality of endpoints to establish sessions between respective of said plurality of endpoints, said sessions being compatible with the encoding capabilities of the plurality of endpoints; and

means for conducting the videoconference between the endpoints using encoded video streams generated at the endpoints without the controller also encoding the video streams,

wherein said means for conducting conducts the videoconference by forming with the controller a combined frame from a first partial frame sent from a first endpoint and a second partial frame sent from a second endpoint, said controller operating in a robust mode of operation when an observed packet loss rate from a stream from said first endpoint is greater than a predetermined level.