Distributed real-time media composer

• US 7,561,179 B2
• Filed: 11/15/2004
• Issued: 07/14/2009
• Est. Priority Date: 11/14/2003
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Tandberg Telecom AS (Cisco Systems, Inc.)
• Inventors: Sandbakken, Geir Arne, Johansen, Tom-Ivar
• Primary Examiner(s): Ramakrishnaiah; Melur
• Application Number:
  US10/986,932
• Publication Number:
  US 20050122392A1
• Time in Patent Office: 1,702 Days
• Field of Search: 348 1401- 1409, 348/14.1, 348 1411- 1416, 370260-261, 370/401, 709/204
• US Class Current: 348/14.09
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04M 3/562 where the conference facilities are distributed
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04N 7/14 Systems for two-way working H04N7/12, H04N7/173 take precedence
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A system and a method allowing simultaneous exchange of audio, video or data information between a plurality of units over a communication network, supported by a central unit, wherein the central unit is, based on knowledge regarding one or more of the units, adapted to instruct said one or more units to generate multimedia data streams adjusted to fit into certain restrictions to be presented on other units participating in a same session.

Claims

1. A system allowing simultaneous exchange of audio, video or data information, comprising:

a plurality of endpoints configured to communicate over a network; and

a central control unit configured to determine encoding capabilities and display capabilities of the plurality of endpoints, to determine a layout of how video data is to be displayed for each of the plurality of endpoints based on determined encoding and display capabilities, and to instruct each of the plurality of endpoints to transmit partial frames in a multimedia data stream transmitted to the central control unit, respectively, in accordance with layouts determined by the central control unit for each of the plurality of endpoints.

2. The system according to claim 1,

wherein said encoding capabilities are based on information regarding one of the following:

transmission bandwidth, processing requirements and multiple combinations thereof.

3. The system according to claim 1,

wherein the central control unit is configured to engage in a bidirectional exchange with the plurality of endpoints.

4. The system according to claim 1,

wherein the central control unit is configured to generate a combined data stream from two or more data streams sent from two or more endpoints and to route said combined data stream to other endpoints.

5. The system according to claim 1,

wherein the endpoints are configured to communicate in a videoconference.

6. The system according to claim 1,

wherein communication between said central control unit and the plurality of endpoints uses scalable compression techniques.

7. The system according to claim 1,

wherein said multimedia data streams received from the plurality of endpoints at the central control unit are unidirectional.

8. The system according to claim 1,

wherein the central control unit is configured to adapt each multimedia data stream received to specifications of at least one endpoint.

9. The system according to claim 8,

wherein the central control unit is configured to adapt each multimedia data stream to the specifications of the at least one endpoint based on the encoding capabilities and display capabilities of the at least one endpoint.

10. The system according claim 3, further comprising:

a display configured to be divided into a grid pattern of cells,

wherein each of said partial frames occupies one or more cells of the grid pattern and the display is configured to display images with various aspect ratios.

11. The system according to claim 10,

wherein each of said partial frames is occupying a varying number of adjacent cells.

12. The system according to claim 5,

wherein said video conference session is one of a centralized conference, a decentralized conference, and a hybrid conference.

13. A method to exchange audio, video and data information simultaneously between a plurality of endpoints in a network supported by a central video conference control apparatus, comprising steps of:

determining, at the video conference control apparatus, encoding capabilities and display capabilities of the plurality of endpoints;

determining, at the video conference control apparatus, a layout of how video data is to be displayed for each of the plurality of endpoints based on determined encoding and display capabilities; and

instructing, at the video conference control apparatus, each of the plurality of endpoints to transmit partial frames in a multimedia data stream transmitted to the video conference control apparatus, respectively, in accordance with layouts determined by the video conference control apparatus for each of the plurality of endpoints.

14. The method according to claim 13,

wherein said encoding capabilities include one of the following:

transmission bandwidth,

processing requirements and

multiple combinations thereof.

15. The method according to claim 13, further comprising a step of:

bidirectionally exchanging said multimedia data streams between the video conference control apparatus and the plurality of endpoints.

16. The method according to claim 13, further comprising steps of:

generating, at the video conference control apparatus, a combined data stream composed of two or more data streams sent from two or more endpoints; and

routing said combined stream to other endpoints.

17. The method according to claim 13, further comprising:

establishing a videoconference between the plurality of endpoints.

18. The method according to claim 13, further comprising a step of:

using scalable compression techniques when exchanging data between said video conference control apparatus and the plurality of endpoints.

19. The method according to claim 15, further comprising:

dividing a display into a grid pattern of cells,

disposing each of said partial frames in one or more cells of the grid pattern; and

displaying images in the display having varying aspect ratios.

20. The method according to claim 19, further comprising a step of:

letting each partial frame occupy a varying number of adjacent cells.

21. The method according to claim 17, wherein

the video conference session is one of a centralized conference, a decentralized conference and a hybrid conference.

22. The method according to claim 13, further comprising:

receiving unidirectional multimedia data streams that include partial frames from the plurality of endpoints.

23. The method according to claim 13, further comprising a step of:

adapting the multimedia data stream received by the video conference control apparatus to specifications of at least one endpoint.

24. The method according to claim 23,

wherein the step of adapting is based on the encoding capabilities and the display capabilities of the at least one endpoint.

25. A computer readable storage device encoded with instructions for execution on a computer system of a video conference control apparatus, which when executed by the computer system, causes the computer system to perform the method recited in claim 13.

26. A system for exchanging audio, video and data information simultaneously between a plurality of units in a network supported by a central unit, comprising:

means for determining encoding capabilities and display capabilities of the plurality of endpoints;

means for determining a layout of how video data is to be displayed for each of the plurality of endpoints based on determined encoding and display capabilities; and

means for instructing each of the plurality of endpoints to transmit partial frames in a multimedia data stream transmitted to the central control unit, respectively, in accordance with layouts determined by the central control unit for each of the plurality of endpoints.

27. A method for a video conference control apparatus to communicate with a plurality of endpoints over a network, comprising steps of:

determining, at the video conference control apparatus, encoding capabilities and display capabilities of the plurality of endpoints;

determining, at the video conference control apparatus, a layout of how video data is to be displayed for each of the plurality of endpoints based on determined encoding and display capabilities;

instructing, at the video conference control apparatus, each of the plurality of endpoints to transmit partial frames in a multimedia data stream transmitted to the video conference control apparatus, respectively, in accordance with layouts determined by the video conference control apparatus for each of the plurality of endpoints;

bidirectionaly exchanging said multimedia data streams between the video conference control apparatus and the plurality of endpoints;

generating, at the video conference control apparatus, a combined data stream composed of two or more data streams sent from two or more of the plurality of units endpoints; and

routing, at the video conference control apparatus, said combined stream to other endpoints.

28. A computer readable storage device encoded with instructions for execution on a computer system, which when executed by the computer system of a video conference control apparatus, causes the computer system to perform the method recited in claim 27.

29. A video conference control apparatus configured to communicate with a plurality of endpoints over a network, comprising:

means for determining encoding capabilities and display capabilities of the plurality of endpoints;

means for determining a layout of how video data is to be displayed for each of the plurality of endpoints based on determined encoding and display capabilities;

means for instructing each of the plurality of endpoints to transmit partial frames in a multimedia data stream transmitted to the central control unit, respectively, in accordance with layouts determined by the central control unit for each of the plurality of endpoints;

means for bidirectionaly exchanging said multimedia data streams between the central unit and the plurality of endpoints;

means for generating a combined data stream composed of two or more data streams sent from two or more of the plurality of endpoints; and

means for routing said combined stream to other endpoints.