Method and apparatus for multipoint conferencing

• US 7,627,629 B1
• Filed: 12/19/2002
• Issued: 12/01/2009
• Est. Priority Date: 10/30/2002
• Status: Expired due to Fees

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Chen, Wen-hsiung, Wu, Fang, Graham, Philip R., Pelton, Gregory D.
• Primary Examiner(s): ISMAIL, SHAWKI SAIF
• Application Number:
  US10/326,617
• Time in Patent Office: 2,539 Days
• Field of Search: 709/204, 709/224, 709/227, 709/230, 709/231, 709/233, 348/14.09, 379/202.01
• US Class Current: 709/204
• CPC Class Codes: H04L 12/1818 Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method of controlling a multimedia multipoint conference that includes two or more participating multimedia terminals that each includes one or more codecs. Also a controller to control the multimedia multipoint conference. The method includes receiving from each participant information on its codec capabilities, determining codec output characteristics for each participant, including one or both of a codec type and a bit rate, and sending instructions to the participants to set their respective codec to the respective determined codec output characteristics. After each participant'"'"'s one or more codecs are set to the respective codec output characteristics, the method includes receiving a media stream from one or more of the participants. The media stream from any particular participant is encoded at the determined codec output characteristics for the particular participant. The method further includes passing the received media streams to each participant. The determined codec output characteristics is such that the received media streams may be passed to each participant without any decoding and re-encoding.

Claims

1. A method of controlling a multimedia multipoint conference that includes a plurality of participants, each participant a multimedia terminal that includes one or more codecs or a gateway that includes one or more codecs, the method implemented in all participant of the conference, the method comprising:

receiving from each other participant information on the capabilities of its one or more codecs as part of a terminal capabilities exchange between the participants;

ascertaining if there is a set of codec output characteristics for all participants such that each participant'"'"'s codec capabilities can be simultaneously met without any media stream from one participant to the other participants needing to be decoded and re-encoded en route to the other participants, and if so, determining, in cooperation with the other participant or participants, the codec output characteristics for all participants that simultaneously meet each participant'"'"'s codec capabilities without any decoding and re-encoding being needed, the codec output characteristics including one or both of a codec type and a bit rate; and

sending instructions to the participants to set the participant'"'"'s respective one or more codecs to the respective determined codec output characteristics;

such that after each participant'"'"'s one or more codecs are set to the respective codec output characteristics, the participants can successfully send and receive media streams encoded at the respective determined codec output characteristics of the respective terminal without any en-route decoding and re-encoding of any of the media streams and without any participant needing to carry out video mixing to generate its output video stream, and

wherein Multipoint Control Unit (MCU) functionality is carried out by the participants operating jointly and cooperatively to jointly determine how to communicate with each other without any transcoding, with no single terminal or device needing act as a Multipoint Controller (MC), such that no distinct MCU device or functionality is needed for conferencing between the participants;

wherein if the step of ascertaining ascertains that there are not codec output characteristics that simultaneously meet the codec capabilities of all participants, the method includes;

ascertaining whether there is a subset of participants and a first set of codec output characteristics that simultaneously meet the codec capabilities of all members of the subset;

if such a subset and first set of common codec output characteristics exist, directing the subset members to communicate using the common codec output characteristics; and

directing those participants whose codec characteristics cannot be met by a common set of codec output characteristics shared by more than one participant to communicate via an (MCU), the MCU including one or more Multipoint Processors (MPs) that can decode, scale, and re-encode streams as necessary to meet the codec requirements of the participants.

2. A method as recited in claim 1, wherein the sending instructions is only to those of the other participants whose one or more modems need to be re-set in order for the respective participant to send media streams at the respective determined codec output characteristics.

3. A method as recited in claim 1, wherein the conference includes a gatekeeper coupled to the network, the method further comprising:

receiving information from the gatekeeper on the input and output bandwidth availability for each of the conference participants,

wherein the ascertaining ascertains if there are a set of codec output characteristics for all participants such that each participant'"'"'s codec capabilities and available bandwidth can be simultaneously met without any media stream from one participant to the other participants needing to be decoded and re-encoded en route to the other participants,

wherein the set of determined codec output characteristics for all participants further simultaneously meet each participant'"'"'s available bandwidth without any decoding and re-encoding being needed.

4. A method as recited in claim 1, wherein the network is a packet network.

5. A method as recited in claim 4, wherein the media streams conform to ITU-T Recommendation H.323.

6. A method as recited in claim 1, wherein the network is a switched circuit network.

7. A method as recited in claim 1, wherein the media streams conform to ITU-T Recommendation H.320.

8. A method as recited in claim 1, wherein the receiving information on codec characteristics is in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

9. A method as recited in claim 8, wherein the instruction sent to a particular other participant is in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

10. A method as recited in claim 1, wherein the one or more participants each have at least one video codec and wherein the media streams include video streams.

11. A method as recited in claim 10, wherein the information received from any particular other participant on the particular other participant'"'"'s codec capabilities may include a receive video capability that is different than the particular other participant'"'"'s transmit video capability.

12. A method as recited in claim 10, wherein a particular participant includes a screen and displays the streams from other participants in different windows on the screen such that all video streams may be simultaneously displayed on the screen of the particular participant without the video streams needing to be decoded, mixed, and re-encoded as a mixed stream prior to being sent to the particular participant.

13. A method as recited in claim 1, wherein the media streams include audio streams.

14. A method as recited in claim 1, wherein the steps of receiving, ascertaining, determining and instructing are repeated every time a new participant joins the conference, the repeating of the receiving being from the new participant.

15. A method as recited in claim 1, wherein the steps of ascertaining, determining and instructing are repeated every time a participant leaves the conference.

16. A controller to control a multimedia multipoint conference, the conference including a plurality of participants, each participant being a multimedia terminal device that includes the controller and one or more codecs, the controller comprising:

At least one processor, wherein the controller further comprises;

means for receiving from each other participant information on its codec capabilities as part of a terminal capabilities exchange between the participants;

means for ascertaining if there is a set of codec output characteristics for all participants such that each participant'"'"'s codec capabilities can be simultaneously met without any media stream from one participant to the other participants needing to be decoded and re-encoded en route to the other participants, and if so, determining, in cooperation with the other participant or participants, the codec output characteristics for all participants that simultaneously meet each participant'"'"'s codec capabilities without any decoding and re-encoding being needed, the codec output characteristics including one or both of a codec type and a bit rate;

means for sending instructions to the participants to set each participant'"'"'s respective one or more codec(s) to the respective determined codec output characteristics;

after each participant'"'"'s one or more codecs are set to the respective codec output characteristics, means for receiving a media stream from one or more of the other participants, the media stream from any particular other participant being encoded at the determined codec output characteristics for the particular other participant; and

means for passing the received media streams to each other participant, the determined codec output characteristics being such that the received media streams may be passed to each participant without any en-route decoding and re-encoding of any of the media stream, and without any participant needing to carry out video mixing to generate its output video stream,

wherein Multipoint Control Unit (MCU) functionality is carried out by the participants operating jointly and cooperatively to jointly determine how to communicate with each other without any transcoding, with no single terminal or device necessarily needing to act as a Multipoint Controller (MC), such that no distinct MCU device or functionality is needed for conferencing between the participants;

wherein if the means for ascertaining ascertains that there are not codec output characteristics that simultaneously meet the codec capabilities of all participants, then ascertaining whether there is a subset of participants and a first set of codec output characteristics that simultaneously meet the codec capabilities of all members of the subset;

wherein if such a subset and first set of common codec output characteristics exist, directing the subset members to communicate using the common codec output characteristics; and

directing those participants whose codec characteristics cannot be met by a common set of codec output characteristics shared by more than one participant to communicate via an (MCU), the MCU including one or more Multipoint Processors (MPs) that can decode, scale, and re-encode streams as necessary to meet the codec requirements of the participants.

17. A controller as recited in claim 16, the controller coupled to a packet network, the means for sending via the packet network, and the means for receiving via the packet network.

18. A controller as recited in claim 17, wherein the media streams conform to ITU-T Recommendation H.323.

19. A controller as recited in claim 16, the controller coupled to a switched circuit network, the means for sending via the switched circuit network, and the means for receiving via the switched circuit network.

20. A controller as recited in claim 16, wherein the media streams conform to ITU-T Recommendation H.320.

21. A controller as recited in claim 16, wherein the receiving information on codec characteristics is in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

22. A controller as recited in claim 21, wherein the means for sending instructions sends the instructions in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

23. A controller as recited in claim 16, wherein the one or more participants each have at least one video codec and wherein the media streams include video streams.

24. A controller as recited in claim 23, wherein the information received from any particular other participant on the particular other participant'"'"'s codec capabilities may include a receive video capability that is different than the particular other participant'"'"'s transmit video capability.

25. A controller as recited in claim 16, further comprising:

means for receiving information that a new participant is joining or that one or more participants are leaving the conference, the information that a new participant is joining including information on the new participant'"'"'s codec capabilities,

wherein the means for determining re-determines codec output characteristics for each remaining participant, including any new participant; and

wherein the means for sending instructions sends instructions to any new participant and at least those participants whose respective re-determined codec output characteristics are changed by the re-determining to set their respective one or more codec(s) to the respective re-determined codec output characteristics,

the re-determined codec output characteristics being such that the received media streams may be passed to each participant including any new participant without any decoding and re-encoding.

26. A computer readable storage medium storing computer-executable instructions that when executed by processors of a processing system in a plurality of participants of a multimedia multipoint conference, cause the participants to implement a method of controlling the multimedia multipoint conference, the conference including the plurality of participants, each a multimedia terminal device that includes one or more codecs, the method comprising:

receiving from each other participant information on its codec capabilities as part of a terminal capabilities exchange between the participants;

ascertaining if there is a set of codec output characteristics for all participants such that each participant'"'"'s codec capabilities can be simultaneously met without any media stream from one participant to the other participants needing to be decoded and re-encoded en route to the other participants, and if so, determining, in cooperation with the other participant or participants, the codec output characteristics for all participants that simultaneously meet each participant'"'"'s codec capabilities without any decoding and re-encoding being needed, the codec output characteristics including one or both of a codec type and a bit rate;

sending instructions to the participants to set each participant'"'"'s respective one or more codecs to the respective determined codec output characteristics;

after each participant'"'"'s one or more codecs are set to the respective codec output characteristics, receiving a media stream from one or more of the other participants, the media stream from any particular other participant being encoded at the determined codec output characteristics for the particular other participant; and

passing the received media streams to each participant, the determined codec output characteristics being such that the received media streams may be passed to each participant without en-route decoding and re-encoding of any of the media stream, and without any participant needing to carry out video mixing to generate its output video stream,

wherein Multipoint Control Unit (MCU) functionality is carried out by the participants operating jointly and cooperatively to jointly determine how to communicate with each other without any transcoding, with no single terminal or device necessarily needing to act as a Multipoint Controller (MC), such that no distinct MCU device or functionality is needed for conferencing between the participants;

wherein if the step of ascertaining ascertains that there are not codec output characteristics that simultaneously meet the codec capabilities of all participants, the method includes;

ascertaining whether there is a subset of participants and a first set of codec output characteristics that simultaneously meet the codec capabilities of all members of the subset;

if such a subset and first set of common codec output characteristics exist, directing the subset members to communicate using the common codec output characteristics; and

directing those participants whose codec characteristics cannot be met by a common set of codec output characteristics shared by more than one participant to communicate via an (MCU), the MCU including one or more Processors (MPs) that can decode, scale, and re-encode streams as necessary to meet the codec requirements of the participants.

27. A tangible computer readable medium as recited in claim 26, wherein the receiving information on codec characteristics is in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

28. A tangible computer readable medium as recited in claim 27, wherein the instruction sent to a particular other participant is in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

29. A tangible computer readable medium as recited in claim 26, wherein the sending and receiving is via a packet network.

30. A tangible computer readable medium as recited in claim 29, wherein the media streams conform to ITU-T Recommendation H.323.

31. A tangible computer readable medium as recited in claim 26, wherein the sending and receiving is via a switched circuit network.

32. A tangible computer readable medium as recited in claim 26, wherein the media streams conform to ITU-T Recommendation H.320.

33. A tangible computer readable medium as recited in claim 26, wherein the one or more participants each have at least one video codec and wherein the media streams include video streams.

34. A tangible computer readable medium as recited in claim 33, wherein the information received from any particular other participant on the particular other participant'"'"'s codec capabilities may include a receive video capability that is different than the particular other participant'"'"'s transmit video capability.

35. A tangible computer readable medium as recited in claim 26, wherein the method further includes:

receiving information that a new participant is joining or that one or more participants are leaving the conference, the information that a new participant is joining including information on the new participant'"'"'s codec capabilities;

re-determining codec output characteristics for each remaining participant, including any new participant; and

sending instructions to any new participant and at least those participants whose respective re-determined codec output characteristics are changed by the re-determining to set their respective one or more codec(s) to the respective re-determined codec output characteristics,

the re-determined codec output characteristics being such that the received media streams may be passed to each participant including any new participant without any decoding and re-encoding.

36. A controller to control a multimedia multipoint conference, the conference including a plurality of participants, each a multimedia terminal that includes one or more codecs, the controller being on a first participant of the participants, the controller comprising:

a processor;

memory coupled to the processor; and

a network interface device coupled to the processor, the network interface device for communicating via a network with the participants,

wherein the processor is programmed to receive messages from and send messages to the other participants via the network interface device, including receiving from each other participant information on its codec capabilities as part of a terminal capabilities exchange between the participants,

wherein the processor is programmed further to ascertain if there is a set of codec output characteristics for all participants such that each participant'"'"'s codec capabilities can be simultaneously met without any media stream from one participant to the other participants needing to be decoded and re-encoded en route to the other participants, and if so, determining, in cooperation with the other participant or participants, the codec output characteristics for all participants that simultaneously meet each participant'"'"'s codec capabilities without any decoding and re-encoding being needed, the codec output characteristics including one or both of a codec type and a bit rate;

wherein the processor is programmed further to send instructions to the participants via the network interface device to set each participant'"'"'s respective one or more codec(s) to the respective determined codec output characteristics,

wherein the processor is programmed further to receive a media stream from one or more of the other participants via the network interface device after each participant'"'"'s one or more codecs are set to the respective codec output characteristics, the media stream from any particular other participant being encoded at the determined codec output characteristics for the particular other participant, and

wherein the processor is programmed further to pass the received media streams to each other participant via the network interface device, the determined codec output characteristics being such that the received media streams may be passed to each participant without any en-route decoding and re-encoding of any of the media stream, and without any participant needing to carry out video mixing to generate its output video stream,

wherein Multipoint Control Unit (MCU) functionality is carried out by the participants operating jointly and cooperatively to jointly determine how to communicate with each other without any transcoding, with no single terminal or device necessarily needing to act as a Multipoint Controller (MC), such that no distinct MCU device or functionality is needed for conferencing between the participants;

wherein if the processor ascertains that there are not codec output characteristics that simultaneously meet the codec capabilities of all participants, ascertaining whether there is a subset of participants and a first set of codec output characteristics that simultaneously meet the codec capabilities of all members of the subset;

if such a subset and first set of common codec output characteristics exist, directing the subset members to communicate using the common codec output characteristics; and

directing those participants whose codec characteristics cannot be met by a common set of codec output characteristics shared by more than one participant to communicate via an (MCU), the MCU including one or more Multipoint Processors (MPs) that can decode, scale, and re-encode streams as necessary to meet the codec requirements of the participants.

37. A controller as recited in claim 36, wherein the receiving information on codec characteristics is in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

38. A controller as recited in claim 37, wherein the instruction sent to a particular other participant is in the form of a TerminalCapabilitySet message conforming to ITU-T Recommendation H.245.

39. A controller as recited in claim 36, wherein the network is a packet network.

40. A controller as recited in claim 36, the network is a—

circuit switched network.

41. A controller as recited in claim 39, wherein the media streams conform to ITU-T Recommendation H.323.

42. A controller as recited in claim 41, wherein the participants communicate with a gatekeeper whose function includes allocating bandwidth, and wherein the determining of the codec output characteristics for each participant includes using information on the bandwidth allocated to one or more of the participants.

43. A controller as recited in claim 36, wherein the one or more participants each have at least one video codec and wherein the media streams include video streams.

44. A controller as recited in claim 43, wherein the information received from any particular other participant on the particular other participant'"'"'s codec capabilities may include a receive video capability that is different than the particular other participant'"'"'s transmit video capability.

45. A controller as recited in claim 43, wherein a particular participant includes a screen and displays the streams from other participants in different windows on the screen such that all video streams may be simultaneously displayed on the screen of the particular participant without the video streams needing to be decoded, mixed, and re-encoded as a mixed stream prior to being sent to the particular participant.

46. A controller as recited in claim 36, wherein the media streams include audio streams.

47. A controller as recited in claim 36, wherein the processor further is programmed to:

receive information that a new participant is joining or that one or more participants are leaving the conference, the information that a new participant is joining including information on the new participant'"'"'s codec capabilities;

re-determine codec output characteristics for each remaining participant, including any new participant; and

send instructions to any new participant and at least those participants whose respective re-determined codec output characteristics are changed by the re-determining to set their respective one or more codec(s) to the respective re-determined codec output characteristics,

the re-determined codec output characteristics being such that the received media streams may be passed to each participant including any new participant without any decoding and re-encoding.