Video teleconferencing system with digital transcoding

• US 20030231600A1
• Filed: 04/29/2003
• Published: 12/18/2003
• Est. Priority Date: 01/27/1995
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Tandberg Telecom AS (Cisco Systems, Inc.)
• Inventors: Polomski, Mark D.
• Granted Patent: US 7,499,416 B2
• Time in Patent Office: Days
• US Class Current: 370/263
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A video teleconferencing system uses digital transcoding to obtain algorithm transcoding, transmission rate matching, and spatial mixing. The video teleconferencing system comprises a multipoint control unit (MCU) for allowing multiple audiovisual terminals, which send and receive compressed digital data signals, to communicate with each other in a conference. The MCU has a video processing unit (VPU) that performs algorithm transcoding, rate matching, and spatial mixing among the terminals within a conference. The VPU includes a time division multiplex pixel bus and a plurality of processors. Each processor is assignable to an audiovisual terminal in the conference and is coupled to the pixel bus. In a receive mode, each processor receives and decodes compressed video signals from its assigned terminal and puts the decoded signal onto the pixel bus. In a transmit mode, each processor receives from the pixel bus uncompressed video signals from any terminal in the conference. The uncompressed video signals are processed and encoded for transmission to the respective assigned terminal. Video encoding time due to motion displacement search is reduced by passing displacement information from the compressed video signals to the encoder to be used directly or as a seed for further refinements of the motion displacement field.

Claims

1. A method of operating a conference, comprising:

receiving a first input signal from a first audiovisual terminal, the first input signal including first compressed video data;

receiving a second input signal from a second audiovisual terminal, the second input signal including second compressed video data;

decompressing the first compressed video data and thereby generating first decompressed digital data;

decompressing the second compressed video data and thereby generating second decompressed digital data;

outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots;

outputting at least some of the second decompressed digital data to the bus during a second plurality of timeslots;

reading at least some of the first decompressed digital data from the bus during the first plurality of timeslots;

reading at least some of the second decompressed digital data from the bus during the second plurality of timeslots;

combining at least some of the first and second decompressed digital data read from the bus to form third decompressed data representative of a first composite image;

compressing at least some of the third decompressed data and thereby forming third compressed data;

using at least some of the third compressed data to generate a first output signal;

sending the first output signal to a third audiovisual terminal.

2. A method according to claim 1, the first output signal representing a sequence of composite images.

3. A method according to claim 1, wherein the bus is a time division multiplexed bus.

4. A method according to claim 1, wherein outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots comprises periodically outputting a selected portion of the first decompressed digital data to the bus.

5. A method according to claim 1, further comprising:

receiving a third input signal from the third audiovisual terminal, the third input signal including fourth compressed video data;

decompressing the fourth compressed video data and thereby generating fourth decompressed digital data;

outputting at least some of the fourth decompressed digital data to the bus during a third plurality of timeslots;

reading at least some of the fourth decompressed digital data from the bus during the third plurality of timeslots;

combining at least some of the first and fourth decompressed digital data read from the bus to form fifth decompressed data representative of a second composite image;

compressing at least some of the fifth decompressed data and thereby forming fifth compressed data;

using at least some of the fifth compressed data to generate a second output signal;

sending the second output signal to the second audiovisual terminal.

6. A method according to claim 5, further comprising:

combining at least some of the second and fourth decompressed digital data read from the bus to form sixth decompressed data representative of a third composite image;

compressing at least some of the sixth decompressed data and thereby forming sixth compressed data;

using at least some of the sixth compressed data to generate a third output signal;

sending the third output signal to the first audiovisual terminal.

7. A method according to claim 1, wherein the first compressed video data has been compressed according to a first data compression algorithm and wherein the second compressed video data has been compressed according to a second data compression algorithm.

8. A method according to claim 7, wherein decompressing the first compressed video data comprises decompressing the first compressed video data according to the first data compression algorithm.

9. A method according to claim 8, wherein decompressing the second compressed video data comprises decompressing the second compressed video data according to the second data compression algorithm.

10. A method according to claim 1, wherein receiving the first input signal comprises receiving the first input signal at a first data transmission rate, and wherein sending the first output signal comprises sending the first output signal at a second data transmission rate, the first data transmission rate being different than the second data transmission rate.

11. A method of operating a conference, comprising:

receiving a first input signal from a first audiovisual terminal, the first input signal including first compressed video data that has been compressed according to a first data compression algorithm;

receiving a second input signal from a second audiovisual terminal, the second input signal including second compressed video data that has been compressed according to a second data compression algorithm;

decompressing the first compressed video data according to the first data compression algorithm and thereby generating first decompressed digital data;

decompressing the second compressed video data according to the second data compression algorithm and thereby generating second decompressed digital data;

outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots;

outputting at least some of the second decompressed digital data to the bus during a second plurality of timeslots;

reading at least some of the first decompressed digital data from the bus during the first plurality of timeslots;

reading at least some of the second decompressed digital data from the bus during the second plurality of timeslots;

compressing at least some of the first decompressed digital data read from the bus according to the second data compression algorithm and using at least some of that compressed data to generate a first output signal;

compressing at least some of the second decompressed digital data read from the bus according to the first data compression algorithm and using at least some of that compressed data to generate a second output signal;

sending the first output signal to the second audiovisual terminal; and

sending the second output signal to the first audiovisual terminal.

12. A method according to claim 11, the first output signal representing a sequence of frames.

13. A method according to claim 11, wherein the bus is a time division multiplexed bus.

14. A method according to claim 11, wherein outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots comprises periodically outputting a selected portion of the first decompressed digital data to the bus.

15. A method according to claim 11, further comprising:

combining at least some of the first and second decompressed digital data read from the bus to form third decompressed data representative of a composite image;

compressing at least some of the third decompressed data and thereby forming third compressed data;

using at least some of the third compressed data to generate a third output signal;

sending the third output signal to a third audiovisual terminal.

16. A method according to claim 11, wherein receiving the first input signal comprises receiving the first input signal at a first data transmission rate, and wherein sending the first output signal comprises sending the first output signal at a second data transmission rate, the first data transmission rate being different than the second data transmission rate.

17. A method of operating a conference, comprising:

receiving a first input signal at a first data transmission rate from a first audiovisual terminal, the first input signal including first compressed video data;

receiving a second input signal at a second data transmission rate from a second audiovisual terminal, the second input signal including second compressed video data;

decompressing the first compressed video data and thereby generating first decompressed digital data;

decompressing the second compressed video data and thereby generating second decompressed digital data;

outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots;

outputting at least some of the second decompressed digital data to the bus during a second plurality of timeslots;

reading at least some of the first decompressed digital data from the bus during the first plurality of timeslots;

reading at least some of the second decompressed digital data from the bus during the second plurality of timeslots;

compressing at least some of the first decompressed digital data read from the bus and using at least some of that compressed data to generate a first output signal;

compressing at least some of the second decompressed digital data read from the bus and using at least some of that compressed data to generate a second output signal;

sending the first output signal at the second data transmission rate to the second audiovisual terminal; and

sending the second output signal at the first data transmission rate to the first audiovisual terminal.

18. A method according to claim 17, the first output signal representing a sequence of frames.

19. A method according to claim 17, wherein the bus is a time division multiplexed bus.

20. A method according to claim 17, wherein outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots comprises periodically outputting a selected portion of the first decompressed digital data to the bus.

21. A method according to claim 17, wherein the first data transmission rate is slower than the second data transmission rate.

22. A method according to claim 21, wherein the data that is compressed and used to generate the second output signal comprises only a portion of the second decompressed digital data.

23. A method according to claim 17, further comprising:

combining at least some of the first and second decompressed digital data read from the bus to form third decompressed data representative of a composite image;

compressing at least some of the third decompressed data and thereby forming third compressed data;

using at least some of the third compressed data to generate a third output signal;

sending the third output signal to a third audiovisual terminal.

24. A method according to claim 17, wherein the first compressed video data has been compressed according to a first data compression algorithm and wherein the second compressed video data has been compressed according to a second data compression algorithm.

25. A method according to claim 24, wherein decompressing the first compressed video data comprises decompressing the first compressed video data according to the first data compression algorithm.

26. A method according to claim 25, wherein decompressing the second compressed video data comprises decompressing the second compressed video data according to the second data compression algorithm.

27. A method of operating a conference, comprising:

receiving a first input signal from a first audiovisual terminal, the first input signal including first compressed video data that has been compressed according to a first data compression algorithm;

receiving a second input signal from a second audiovisual terminal, the second input signal including second compressed video data that has been compressed according to a second data compression algorithm;

decompressing the first compressed video data according to the first data compression algorithm and thereby generating first decompressed digital data;

decompressing the second compressed video data according to the second data compression algorithm and thereby generating second decompressed digital data;

outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots;

outputting at least some of the second decompressed digital data to the bus during a second plurality of timeslots;

reading at least some of the first decompressed digital data from the bus during the first plurality of timeslots;

reading at least some of the second decompressed digital data from the bus during the second plurality of timeslots;

combining at least some of the first and second decompressed digital data read from the bus to form third decompressed data representative of a first composite image;

compressing at least some of the third decompressed data and thereby forming third compressed data;

using at least some of the third compressed data to generate a first output signal;

sending the first output signal to a third audiovisual terminal.

28. A method according to claim 27, the first output signal representing a sequence of composite images.

29. A method according to claim 27, wherein the bus is a time division multiplexed bus.

30. A method according to claim 27, wherein outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots comprises periodically outputting a selected portion of the first decompressed digital data to the bus.

31. A method of operating a conference, comprising:

receiving a first input signal at a first data transmission rate from a first audiovisual terminal, the first input signal including first compressed video data;

receiving a second input signal at a second data transmission rate from a second audiovisual terminal, the second input signal including second compressed video data, the second data transmission rate being different than the first data transmission rate;

decompressing the first compressed video data and thereby generating first decompressed digital data;

decompressing the second compressed video data and thereby generating second decompressed digital data;

outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots;

outputting at least some of the second decompressed digital data to the bus during a second plurality of timeslots;

reading at least some of the first decompressed digital data from the bus during the first plurality of timeslots;

reading at least some of the second decompressed digital data from the bus during the second plurality of timeslots;

combining at least some of the first and second decompressed digital data read from the bus to form third decompressed data representative of a first composite image;

compressing at least some of the third decompressed data and thereby forming third compressed data;

using at least some of the third compressed data to generate a first output signal;

sending the first output signal to a third audiovisual terminal.

32. A method according to claim 31, the first output signal representing a sequence of composite images.

33. A method according to claim 31, wherein the bus is a time division multiplexed bus.

34. A method according to claim 31, wherein outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots comprises periodically outputting a selected portion of the first decompressed digital data to the bus.

35. A method of operating a conference, comprising:

receiving a first input signal at a first data transmission rate from a first audiovisual terminal, the first input signal including first compressed video data that has been compressed according to a first data compression algorithm;

receiving a second input signal at a second data transmission rate from a second audiovisual terminal, the second input signal including second compressed video data that has been compressed according to a second data compression algorithm;

decompressing the first compressed video data according to the first data compression algorithm and thereby generating first decompressed digital data;

decompressing the second compressed video data according to the second data compression algorithm and thereby generating second decompressed digital data;

outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots;

outputting at least some of the second decompressed digital data to the bus during a second plurality of timeslots;

reading at least some of the first decompressed digital data from the bus during the first plurality of timeslots;

reading at least some of the second decompressed digital data from the bus during the second plurality of timeslots;

compressing at least some of the first decompressed digital data read from the bus according to the second data compression algorithm and using at least some of that compressed data to generate a first output signal;

compressing at least some of the second decompressed digital data read from the bus according to the first data compression algorithm and using at least some of that compressed data to generate a second output signal;

sending the first output signal at the second data transmission rate to the second audiovisual terminal; and

sending the second output signal at the first data transmission rate to the first audiovisual terminal.

36. A method according to claim 35, the first output signal representing a sequence of frames.

37. A method according to claim 35, wherein the bus is a time division multiplexed bus.

38. A method according to claim 35, wherein outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots comprises periodically outputting a selected portion of the first decompressed digital data to the bus.

39. A method of operating a conference, comprising:

receiving a first input signal at a first data transmission rate from a first audiovisual terminal, the first input signal including first compressed video data that has been compressed according to a first data compression algorithm;

receiving a second input signal at a second data transmission rate from a second audiovisual terminal, the second input signal including second compressed video data that has been compressed according to a second data compression algorithm;

decompressing the first compressed video data according to the first data compression algorithm and thereby generating first decompressed digital data;

decompressing the second compressed video data according to the second data compression algorithm and thereby generating second decompressed digital data;

outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots;

outputting at least some of the second decompressed digital data to the bus during a second plurality of timeslots;

reading at least some of the first decompressed digital data from the bus during the first plurality of timeslots;

reading at least some of the second decompressed digital data from the bus during the second plurality of timeslots;

combining at least some of the first and second decompressed digital data read from the bus to form third decompressed data representative of a first composite image;

compressing at least some of the third decompressed data and thereby forming third compressed data;

using at least some of the third compressed data to generate a first output signal;

sending the first output signal to a third audiovisual terminal.

40. A method according to claim 39, the first output signal representing a sequence of composite images.

41. A method according to claim 39, wherein the bus is a time division multiplexed bus.

42. A method according to claim 39, wherein outputting at least some of the first decompressed digital data to a bus during a first plurality of timeslots comprises periodically outputting a selected portion of the first decompressed digital data to the bus.

43. A method of videoconferencing comprising:

receiving compressed video signals from audiovisual terminals;

processing the received compressed video signals;

retransmitting the processed video signals to appropriate audiovisual terminals;

wherein for dissimilar audiovisual terminals that operate using different compression algorithms and/or data transmission rates, processing includes digitally transcoding and/or rate matching the compressed video signals and switching transcoded and/or rate matched video signals.

44. A method according to claim 43, wherein for similar audiovisual terminals that operate using similar compression algorithms and similar data transmission rates, processing includes switching compressed video signals.

45. A method of videoconferencing comprising:

between terminals that operate using similar compression algorithms and at similar data transmission rates,

switching compressed video signals between respective terminals;

between terminals that operate using different compression algorithms and/or at different data transmission rates,

decoding compressed video signals from respective terminals to uncompressed video signals;

outputting at least some of the uncompressed video signals to a bus;

selecting at least some of the uncompressed video signals from the bus;

encoding the selected uncompressed video signals for transmission to respective terminals.

46. A method according to claim 45, wherein the bus is a time division multiplexed bus.

48. A method according to claim 46, further including digitally rate matching video signals from incompatible terminals wherein the incompatible terminals further operate using different data transmission rates.

47. A method of videoconferencing comprising:

between compatible terminals that operate using similar compression algorithms, switching video signals between the compatible terminals;

between incompatible terminals that operate using different compression algorithms, digitally transcoding video signals from the incompatible terminals and switching the transcoded video signals between incompatible terminals.