Method and system for multimedia communication control

• US 20020015092A1
• Filed: 09/14/2001
• Published: 02/07/2002
• Est. Priority Date: 01/13/2000
• Status: Active Grant

• Current Assignee: Polycom Israel Limited (Plantronics, Inc.)
• Original Assignee: Polycom Israel Limited (Plantronics, Inc.)
• Inventors: Eshkoli, Noam, Elbaz, Moshe, Eisenberg, Aviv, Yona, Iian, Feder, Meir
• Granted Patent: US 6,496,216 B2
• Time in Patent Office: Days
• US Class Current: 348/14.13
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A multipoint control unit (MCU) or other digital video-processing apparatus operates to manipulate compressed digital video from several compressed digital video sources. The apparatus has a plurality of video input modules and a plurality of video output module. Each of the video input modules receives a compressed video signal from one of the sources and generally decodes the data into a primary data stream and a secondary data stream. The video output module receives the primary and secondary data streams, from at least one of the input module for generally encoding to a compressed output stream for transmission.

Claims

1. An apparatus for manipulating compressed digital video, the manipulated compressed digital video being a manipulation of data from at least one of a plurality of compressed digital video sources, the apparatus comprising:

at least one video input module for receiving compressed video input data from at least one source of said plurality of sources, the video input module comprising a generalized decoder operative to decode the compressed video input data and generate a primary video data stream and to process the compressed video input data and said primary video data stream to generate a secondary data stream; and

at least one video output module for receiving primary video data stream and secondary data stream from at least one input module of said plurality of input modules, and operative to encode the primary video data stream with referring to the secondary data stream into manipulated compressed video output data,

whereby the use of said secondary data stream by the output module improves the speed of encoding and the quality of the compressed video data.

2. The apparatus of claim 1, wherein the video output module comprises:

a rate control unit; and

a generalized encoder, in communication with the rate control unit and operative to receive the primary video data from at least one input modules and encode the primary video data into manipulated compressed video output data.

3. The apparatus of claim 2, wherein said rate control unit comprises:

means to read said secondary data stream;

means to process said secondary data stream; and

means to control a generalized encoded based upon said processed secondary data stream.

4. The apparatus of claim 2, wherein said rate control unit comprises:

means to read feedback data from a generalized encoder;

means to process said secondary data stream with feedback data; and

means to control a generalized encoded based upon said processed secondary data stream and said feedback.

5. The apparatus of claim 4, wherein said side information includes at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter usage indication, quantizer identifications, coded/uncoded decisions, the amount of information within each macroblock, image segmentation indication, scene cut off indication, camera zoom identification, camera pan identification, camera movements identification and statistical information.

6. The apparatus of claim 2, wherein each said secondary data stream is associated with a primary data stream and said video output module receives at least one of said primary video data streams and it'"'"'s associated said secondary data stream and control information from an external device.

7. The apparatus of claim 6, wherein said rate control unit of said video output module comprises:

means to read said secondary data stream;

means to read said control information;

means to process said secondary data stream;

means to process said control information; and

means to control said generalized encoder based upon the processed secondary data stream and the processed control information.

8. The apparatus of claim 7, wherein said rate control unit of said video output module comprise:

means to read feedback data from a generalized encoder; and

means to process said secondary data stream with said control information and said feedback; and

means to control said generalized encoder based upon the results.

9. The apparatus of claim 6, wherein the control information includes at least one type of information selected from a group consisting of:

region of interest indication, screen layout requirements, user quality preferences and special effects.

10. The apparatus of claim 6, wherein the control information is bi-directional information.

11. The apparatus of claim 2, wherein said video output module receives said primary video data stream and said secondary data stream, and said rate control unit of said video output module comprises:

means to read said secondary data stream;

means to read data on the actual amount of bits used after variable length coding;

means to process the respective secondary data stream with the variable length coding information; and

means to control said generalized encoder based on the said processed information, whereby the use of said variable length coding information and the secondary data stream by generalized encoder improves the speed of encoding and the quality of the compressed video output signal by improving the output bits allocation.

12. The apparatus of claim 11 wherein said video output module receives said primary video data stream and said secondary data stream, and said rate control unit of said video output module comprises:

means to read feedback data from a generalized encoder;

means to process the respective secondary data stream with the variable length coding information and said feedback data; and

means to control said generalized encoder based on the said processed information.

13. The apparatus of claim 1, further comprising:

means to route said primary video data from at least one input module to at least one output module; and

means to route said secondary data stream from at least one input module to at least on output module.

14. The apparatus of claim 13, wherein said means to route said primary video data stream includes a common interface selected from a group consisting of:

shared memory, ATM bus, TDM bus, switching and direct connection.

15. The apparatus of claim 13, wherein said means to route the secondary data stream includes a common interface selected from a group consisting of:

shared memory, ATM bus, TDM bus, switching and direct connection.

16. The apparatus of claim 1, wherein the manipulation of the compressed video input data includes at least one type of manipulations selected from a group consisting of:

transcoding and compositing.

17. The apparatus of claim 1, wherein the association between the secondary data stream and the primary video data stream is that the secondary data stream includes side information.

18. The apparatus of claim 1, wherein the compressed video input data includes at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter indication, DCT coefficients and quantizer values.

19. The apparatus of claim 1, wherein said primary video data stream includes information in the DCT domain.

20. The apparatus of claim 1, wherein said primary video data stream includes information in the spatial domain.

21. An apparatus for manipulating compressed digital video, the manipulated compressed digital video being a manipulation of data from at least one of a plurality of compressed digital video sources, the apparatus comprising:

at least one video input module for receiving compressed video input data from at least one source of said plurality of sources, the video input module comprising a generalized decoder operative to decode the compressed video input data and generate a primary video data stream and to process the compressed video input data to generate a secondary data stream; and

at least one video output module for receiving primary video data stream and secondary data stream from at least one input module of said plurality of input modules, and operative to encode the primary video data stream, with referring to the secondary data stream, into manipulated compressed video output data,

whereby the use of said secondary data stream by the output module improves the speed of encoding and the quality of the compressed video data.

22. An apparatus for manipulating compressed digital video, the manipulated compressed digital video being a manipulation of data from at least one of a plurality of compressed digital video sources, the apparatus comprising:

at least one video input module for receiving compressed video input data from at least one source of said plurality of sources, the video input module comprising a generalized decoder operative to decode the compressed video input data and generate a primary video data stream and to process said primary video data stream to generate a secondary data stream; and

at least one video output module for receiving primary video data stream and secondary data stream from at least one input module of said plurality of input modules, and operative to encode the primary video data stream with referring to the secondary data stream into manipulated compressed video output data,

whereby the use of said secondary data stream by the output module improves the speed of encoding and the quality of the compressed video data.

24. The compressed video combiner unit of claim 23, wherein the association between the secondary data stream and the primary video data stream is that the secondary data stream includes side information.

25. The compressed video combiner unit of claim 24, wherein said side information includes at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter usage indication, quantizer identifications, coded/uncoded decisions, the amount of information within each macroblock, image segmentation indication, scene cut off indication, camera zoom identification, camera pan identification, camera movements identification and statistical information.

26. The compressed video combiner unit of claim 23, wherein the compressed video input data includes at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter indication, DCT coefficients and quantizer values.

27. The compressed video combiner unit of claim 23, wherein said rate control unit comprises:

means to read said secondary data stream;

means to process said secondary data stream; and

means to control a generalized encoder based upon said processed secondary data stream.

28. The compressed video combiner unit of claim 27, wherein said rate control unit comprises:

means to read feedback data from a generalized encoder;

means to process said secondary data stream with said feedback data; and

means to control said generalized encoder based upon said processed secondary data stream and said feedback.

29. The compressed video combiner unit of claim 23, wherein said means to route said primary video data stream includes a common interface selected from a group consisting of:

shared memory, ATM bus, TDM bus, switching and direct connection.

30. The compressed video combiner unit of claim 23, wherein said means to route the secondary data stream includes a common interface selected from a group consisting of:

shared memory, ATM bus, TDM bus, switching and direct connection.

31. The compressed video combiner unit of claim 23, wherein said primary video data stream includes information in the DCT domain.

32. The multipoint control unit of claim 23, wherein said primary video data stream includes information in spatial domain.

33. The compressed video combiner unit of claim 23, wherein said video output module receives at least one of said primary video data streams and it'"'"'s associated said secondary data stream and control information from an external device.

34. The compressed video combiner unit of claim 33, wherein said rate control unit of said video output module comprises:

means to read said secondary data stream;

means to read said control information;

means to process said secondary data stream;

means to process the control information; and

means to control said generalized encoder based upon the processed secondary data stream and the processed control information.

35. The compressed video combiner unit of claim 34, wherein said rate control unit of said video output module comprises:

means to read feedback data from a generalized encoder; and

means to process said secondary data stream with said control information and said feedback; and

means to control said generalized encoder based upon the results.

36. The compressed video combiner unit of claim 33, wherein the control information includes at least one type of information selected from a group consisting of:

region of interest indication, screen layout requirements, user quality preferences and special effects.

37. The compressed video combiner unit of claim 33 wherein the control information is bidirectional information.

38. The compressed video combiner unit of claim 23 wherein said video output module receives said primary video data stream and said secondary data stream, and said rate control unit of said video output module comprises:

means to read said secondary data stream;

means to read data on the actual amount of bits used after variable length coding;

means to process the respective secondary data stream with the variable length coding information; and

means to control said generalized encoder based on the said processed information, whereby the use of said variable length coding information and the secondary data stream by generalized encoder improves the speed of encoding and the quality of the compressed video output signal by improving the output bits allocation.

39. The compressed video combiner unit of claim 38 wherein said video output module receives said primary video data stream and said secondary data stream, and said rate control unit of said video output module comprises:

means to read feedback data from a generalized encoder;

means to process the respective secondary data stream with the variable length coding information and said feedback data; and

means to control said generalized encoder based on the said processed information.

23. A compressed video combiner unit for generating a compressed digital video, which is a composition of plurality of compressed digital video sources, the compressed video combiner unit comprising:

at least one video input module for receiving a compressed video input data from at least one source of said plurality of sources, the video input module comprising;

a generalized decoder operative to decode the compressed video input data and generate a primary video data stream, the generalized decoder comprising;

a data processing unit operative to process said compressed video input data and said primary video data stream to generate a secondary data stream, said secondary data stream being associated with said primary video stream; and

at least one video output module operative to receive at least one of said primary video data stream and said secondary data stream, the output module comprising;

a rate control unit; and

a generalized encoder, in communication with said rate control unit and operative to receive said primary video data from said at least one input module and encode said primary video data into a compressed video output data;

means to route said primary video data from at least one input module to at least one output module; and

means to route said secondary data stream from at least one input module to the at lest one output module;

whereby the use of said secondary data stream by the output module improves the speed of encoding and the quality of the compressed video output data.

40. An apparatus for manipulating compressed digital video, the manipulated compressed digital video being a manipulation of data from at least one of a plurality of compressed digital video sources and destinations, the apparatus comprising:

at least one video input module, each video input module being operative to receive a compressed video input signal belongs to one of said sources, depending on the required manipulation, to decode said compressed video input signal for generating a decoded video data stream and to transfer the decoded video data stream to a common interface; and

at least one video output module, each output module being operative to grab at least one of said decoded video data streams from the common interface, to encode said decoded video data stream into a compressed video output stream and to transfer said compressed video output stream to at least one destination of said plurality of destinations; and

a common interface for routing said data streams from at least one input module to at least one output module;

wherein there is no permanent logical relation or connection between said input port and said output port and the temporary logical connection is depending on the current needs of a current manipulation, where by using of said configuration improves resources allocation of said apparatus.

41. The apparatus of claim 40, wherein the manipulation of the compressed video input data includes at least one type of manipulations selected from a group consisting of:

transcoding and compositing.