Multipoint processing unit

• US 7,257,641 B1
• Filed: 03/30/2000
• Issued: 08/14/2007
• Est. Priority Date: 03/30/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ferriere, Philippe, Han, Mu, VanBuskirk, Michael R.
• Primary Examiner(s): Etienne; Ario
• Assistant Examiner(s): EL CHANTI, HUSSEIN A
• Application Number:
  US09/539,026
• Time in Patent Office: 2,693 Days
• Field of Search: 709203-205, 709/209, 709/217, 709238-239, 709/242, 709/246, 709/247, 709/223, 709/231
• US Class Current: 709/238
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04L 65/103 in the network
• CPC Class Codes: H04L 65/104 in the network
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 69/08 Protocols for interworking; Protocol conversion
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A system to provide a multipoint processing terminal and a multicast bridging terminal to provide mixing, switching, and other processing of media streams under the control of H.323 components. Application Programming Interfaces defined for the multipoint processing terminal provide a multipoint control unit with the capability to change the default behavior of the multipoint processing terminal by allowing the multipoint control unit to control the routing audio and video streams in the multipoint processing terminal and control the media formats in a multipoint conference. Multipoint processing acceleration functionality is provided by providing interfaces to allow hardware accelerated implementations of multipoint processing terminals. The multicast bridging terminals enables clients using one type of control signaling and media streaming to join other conferences using different types of control signaling and media streaming by receiving audio or video data from an incoming media stream and performing any processing necessary to transform the media stream from the incoming stream data format to the outgoing stream data format.

Claims

1. A computer-readable medium having computer-executable instructions for communicating between an application and a multipoint processing module adapted to mix, switch, and process media streams, the multipoint processing module having at least one audio processor module for processing audio data in a multipoint conference and at least one video processor module for processing video data in a multipoint conference, the computer-executable instructions performing the step of:

exposing at least one application program interface by the multipoint processing module to receive a request from the application to command the multipoint processing module to modify a default operation of the multipoint processing module to alter at least one attribute of at least one of the audio processor module and video processor module, the application program interface for interfacing software components

wherein said at least one application program interface comprises an audio interface, the application using said audio interface to request the multipoint processing module to change a routing of at least one audio input stream towards at least one audio output stream and

wherein the request is selected from the group consisting of;

a command to retrieve an audio crossbar topology, the audio crossbar topology indicating how a set of audio input streams is being routed to a set of audio output streams;

a command to change the audio crossbar topology to indicate to the multipoint processing module how the set of audio input streams should be routed to a set of audio output streams;

a command to retrieve a value of an audio crossbar control parameter;

a command to set a value of an audio crossbar control parameter;

a command to retrieve a minimum value, a maximum value, and a default value for an audio crossbar control parameter;

a command to retrieve a mixing capability and a transcoding capability of the audio crossbar; and

a command to retrieve an audio level of a list of audio input streams.

2. The computer-readable medium of claim 1 wherein the audio crossbar control parameter is selected from a group of audio crossbar control parameters, the group comprising:

a setting to specify a periodicity of an interrupt service routine;

a setting to specify a maximum number of mixed input signals;

a setting to enable and disable silence detection;

a setting to enable and disable silence compression; and

a setting to enable and disable automatic gain control.

3. The computer-readable medium of claim 1 wherein the multipoint processing module disables the command to set a value of an audio crossbar control parameter when a control flag is set.

4. A computer-readable medium having computer-executable instructions for communicating between an application and a multipoint processing module adapted to mix, switch, and process media streams, the multipoint processing module having at least one audio processor module for processing audio data in a multipoint conference and at least one video processor module for processing video data in a multipoint conference, the computer-executable instructions performing the step of:

exposing at least one application program interface by the multipoint processing module to receive a request from the application to command the multipoint processing module to modify a default operation of the multipoint processing module to alter at least one attribute of at least one of the audio processor module and video processor module, the application program interface for interfacing software components

wherein said at least one application program interface comprises a video interface, the application using said video interface to request the multipoint processing module to change a routing of at least one video input stream towards at least one video output stream and

wherein the request is selected from the group consisting of;

a command to retrieve a video crossbar topology, the video crossbar topology indicating how a set of video input streams is being routed to a set of video output streams based on a content of associated audio input streams;

a command to change the video crossbar topology to indicate to the multipoint processing module how the set of video input streams should be routed to a set of video output streams based on a content of associated audio input streams;

a command to retrieve a value of a video crossbar control parameter;

a command to set a value of a video crossbar control parameter;

a command to retrieve a minimum value, a maximum value, and a default value for a video crossbar control parameter;

a command to retrieve a mixing capability and a transcoding capability of the video crossbar; and

a command to retrieve a video level of a list of video input streams.

5. The computer-readable medium of claim 4 wherein the video crossbar control parameter is selected from a group of video crossbar control parameters, the group comprising:

a setting to specify a first time to evaluate whether a speaker is continuing to speak;

a setting to specify a second time during which a speaker and a video switching process can not be taken over by a second speaker; and

a setting to specify a third time, the third time being the time when a switch is made and when a fast update request is sent to the speaker'"'"'s system.

6. The computer-readable medium of claim 4 wherein the multipoint processing module disables the command to set a value of a video crossbar control parameter when a control flag is set.

7. A computer-readable medium having computer-executable instructions for communicating between an application and a multipoint processing module adapted to mix, switch, and process media streams, the multipoint processing module having at least one audio processor module for processing audio data in a multipoint conference, and at least one video processor module for processing video data in a multipoint conference, the computer-executable instructions performing the step of:

exposing at least one application program interface by the multipoint processing module to receive a request from the application to command the multipoint processing module to modify a default operation of the multipoint processing module to alter at least one attribute of at least one of the audio processor module and video processor module, the application program interface for interfacing software components

wherein said at least one application program interface further comprises a video interface, the application using said video interface to request the multipoint processing module to change a routing of at least one video input stream towards at least one video output stream and

wherein the request to route at least one audio input stream is selected from the group consisting of;

a command to retrieve an audio crossbar topology, the audio crossbar topology indicating how a set of audio input streams is being routed to a set of audio output streams;

a command to change the audio crossbar topology to indicate to the multipoint processing module how the set of audio input streams should be routed to a set of audio output streams;

a command to retrieve a value of an audio crossbar control parameter;

a command to set a value of an audio crossbar control parameter;

a command to retrieve a minimum value, a maximum value, and a default value for an audio crossbar control parameter;

a command to retrieve a mixing capability and a transcoding capability of the audio crossbar; and

a command to retrieve an audio level of a list of audio input streams; and

the request to route at least one video input stream is selected from the group consisting of;

a command to retrieve a video crossbar topology, the video crossbar topology indicating how a set of video input streams is being routed to a set of video output streams based on a content of associated audio input streams;

a command to change the video crossbar topology to indicate to the multipoint processing module how the set of video input streams should be routed to a set of video output streams based on a content of associated audio input streams;

a command to retrieve a value of a video crossbar control parameter;

a command to set a value of a video crossbar control parameter;

a command to retrieve a minimum value, a maximum value, and a default value for a video crossbar control parameter;

a command to retrieve a mixing capability and a transcoding capability of the video crossbar; and

a command to retrieve a video level of a list of video input streams.

8. The method of claim 7 wherein said at least one application program interface further comprises a format control interface, the application using said format control interface to retrieve and set an audio format and a video format, the format control interface comprising:

a command to retrieve a preferred audio and video format for a conference;

a command to set the preferred audio and video format for the conference;

a command to retrieve a format structure and configuration capability structure pair of a conference, the format structure and configuration capability structure pair describing an audio and video format supported by the conference;

a command to retrieve a number of audio and video format structure and configuration capability structure pairs that are available in a conference;

a command to reorder a list of preferred audio formats; and

a command to reorder a list of preferred video formats.

9. A multipoint processing accelerator apparatus for transmitting audio and video data over a plurality of channels in a multipoint conference being controlled by an application, the application controlling the apparatus by an application program interface of the apparatus, the apparatus comprising:

at least one hardware module having a default operation for applying signal processing operations to at least one of the audio and video data; and

a minidriver, said minidriver communicating with the application through at least one property set to do one of receiving a command to modify the default operation of the at least one hardware module and sending a command to the application

wherein the at least one property set comprises an audio topology property set and

wherein the audio topology property set comprises;

a property to do one of updating an audio crossbar content and retrieving an audio crossbar content;

a property to retrieve mixing and transcoding capabilities of an audio crossbar;

a property to do one of setting a periodicity of an interrupt service routine and getting a periodicity of an interrupt service routine;

a property to do one of setting a maximum number of mixed input signals and getting a maximum number of mixed input signals;

a property to do one of enabling silence detection and disabling silence detection;

a property to do one of enabling automatic gain control and disabling automatic gain control; and

a property to retrieve a value of an audio level of a list of audio input streams.

10. A multipoint processing accelerator apparatus for transmitting audio and video data over a plurality of channels in a multipoint conference being controlled by an application, the application controlling the apparatus by an application program interface of the apparatus, the apparatus comprising:

at least one hardware module having a default operation for applying signal processing operations to at least one of the audio and video data; and

a minidriver, said minidriver communicating with the application through at least one property set to do one of receiving a command to modify the default operation of the at least one hardware module and sending a command to the application

wherein the at least one property set comprises a video topology property set and

wherein the video topology property set comprises;

a property to do one of updating a video crossbar content and retrieving a video crossbar content;

a property to retrieve picture composition capabilities of the video crossbar;

a property to do one of setting a periodicity of an interrupt service routine and getting a periodicity of an interrupt service routine;

a property to do one of setting a time to evaluate whether a speaker is continuing to speak and getting a time to evaluate whether a speaker is continuing to speak;

a property to do one of setting a second time during which a speaker and a video switching process can not be taken over by a second speaker and getting a second time during which a speaker and a video switching process can not be taken over by a second speaker; and

a property to do one of setting a third time and getting a third time, the third time being the time when a switch is made and when a fast update request is sent to the speaker'"'"'s system.

11. The apparatus according to claim 10 wherein the at least one property set comprises a decoder property set.

12. The apparatus according to claim 11 wherein the decoder property set comprises:

a property to specify that a video frame update be completed and a video frame be displayed until receiving a release signal; and

a property to indicate a video temporal and spatial trade-off of an encoder.

13. The apparatus according to claim 10 wherein the at least one property set comprises a video encoder send property set.

14. The apparatus according to claim 13 wherein the at least one hardware module comprises a video encoder, the video encoder send property set comprises:

a property to signal to the application that it needs to send a command to the video encoder.

15. The apparatus according to claim 10 wherein the at least one property set comprises a stream topology property set.

16. The apparatus according to claim 15 wherein the stream topology property set comprises:

a property to retrieve a direction and crossbar positional index of a stream.

17. The apparatus according to claim 10 wherein the at least one property set comprises a video encoder property set.

18. The apparatus according to claim 17 wherein the video encoder property set comprises:

a property to command a video output stream to enter a fast update picture mode;

a property to command the video output stream to perform a fast update of a group of blocks;

a property to command the video output stream to perform a fast update of a macroblock;

a property to command the video output stream to use sync for every group of blocks; and

a property to provide an indication that a set of macroblocks has been received with errors and has been treated as not coded.

19. The apparatus according to claim 10 wherein the at least one property set comprises a network statistics property set.

20. The apparatus according to claim 19 wherein the network statistics property set comprises:

a property to do one of informing a video output pin of error channel conditions and supplying a media service provider component the error channel conditions; and

a property to do one of informing the video output pin of a channel packet rate loss and supplying the media service provider component the channel packet rate loss.

21. The apparatus according to claim 10 wherein the at least one property set comprises a bandwidth property set.

22. The apparatus according to claim 21 wherein the bandwidth property set comprises:

a property to do one of specifying an upper limit in bandwidth transmission to a video output pin and supplying the upper limit bandwidth transmission of the video output pin to a media service provider component.

23. The apparatus according to claim 10 wherein the at least one property set comprises a frame rate property set.

24. The apparatus according to claim 23 wherein the frame rate property set comprises:

a property to do one of specifying a video frame'"'"'s average display time to a video output pin and supplying the video frame average display time to a media service provider component.

25. The apparatus according to claim 10 wherein the at least one property set comprises a RTP control property set.

26. The apparatus according to claim 25 wherein the RTP control property set comprises:

a property to do one of retrieving a maximum RTP packet size and setting the maximum RTP packet size.