Method and system for multimedia communication control

• US 6,496,216 B2
• Filed: 09/14/2001
• Issued: 12/17/2002
• Est. Priority Date: 01/13/2000
• Status: Expired due to Term

• Current Assignee: Polycom Israel Limited (HP Inc.)
• Original Assignee: Polycom Israel Limited (HP Inc.)
• Inventors: Eisenberg, Aviv, Eshkoli, Noam, Elbaz, Moshe, Feder, Meir, Yona, Ilan
• Primary Examiner(s): CHAN, WING F
• Application Number:
  US09/952,340
• Publication Number:
  US 20020015092A1
• Time in Patent Office: 459 Days
• Field of Search: 348/14-7-, 379/202.01
• US Class Current: 348/14.09
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A multipoint control unit (MCU) or other digital video-processing apparatus operates to manipulate compressed digital video from several compressed digital video sources. The apparatus has a plurality of video input modules and a plurality of video output module. Each of the video input modules receives a compressed video signal from one of the sources and generally decodes the data into a primary data stream and a secondary data stream. The video output module receives the primary and secondary data streams, from at least one of the input module for generally encoding to a compressed output stream for transmission.

Claims

1. An apparatus for manipulating compressed digital video information to form manipulated compressed video information, the manipulated compressed video information being a manipulation of data from at least one of a plurality of compressed digital video sources, the apparatus comprising:

at least one video input module for receiving compressed video input data from at least one source of the plurality of compressed digital video sources, the at least one video input module comprising a generalized decoder operative to decode the compressed video input data, generate a primary video data stream, and process the compressed video input data and the primary video data stream to generate a secondary data stream; and

at least one video output module for receiving the primary video data stream and the secondary data stream from the at least one video input module, and being operative to encode the primary video data stream with references to the secondary data stream to form manipulated compressed video output data,

whereby the use of the secondary data stream by the at least one video output module improves a speed of encoding and the manipulated compressed video output data'"'"'s quality.

2. The apparatus of claim 1, wherein the video output module comprises:

a rate control unit; and

a generalized encoder, in communication with the rate control unit and operative to receive the primary video data stream, having primary video data, from the at least one video input module and encode the primary video data into the manipulated compressed video output data.

3. The apparatus of claim 2, wherein the rate control unit comprises:

means to read the secondary data stream;

means to process the secondary data stream; and

means to control the generalized encoder based upon results of processing the secondary data stream.

4. The apparatus of claim 2, wherein the rate control unit comprises:

means to read feedback data from the generalized encoder;

means to process the secondary data stream with the feedback data; and

means to control the generalized encoder based upon results of processing the secondary data stream with the feedback data.

5. The apparatus of claim 4, wherein the secondary data stream comprises side information which further comprises at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter usage indication, quantizer identifications, coded/uncoded decisions, the amount of information within each macroblock, image segmentation indication, scene cut off indication, camera zoom identification, camera pan identification, camera movements identification, and statistical information.

6. The apparatus of claim 2, wherein the secondary data stream is associated with a primary data stream to form a n associated secondary data stream, and the at least one video output module receives the primary video data stream, the associated secondary data stream and control information from an external device.

7. The apparatus of claim 6, wherein the rate control unit of the video output module comprises:

means to read the secondary data stream;

means to read the control information;

means to process the secondary data stream;

means to process the control information; and

means to control the generalized encoder based upon results of processing the secondary data stream and results of processing the control information.

8. The apparatus of claim 7, wherein the rate control unit of the at least one video output module comprises:

means to read feedback data from the generalized encoder;

means to process the secondary data stream with the control information and the feedback data; and

means to control the generalized encoder based upon results of processing the secondary data stream with the control information and the feedback data.

9. The apparatus of claim 6, wherein the control information includes at least one type of information selected from a group consisting of:

region of interest indications, screen layout requirements, user quality preferences, and special effects.

10. The apparatus of claim 6, wherein the control information is bi-directional information.

11. The apparatus of claim 2, wherein the at least one video output module receives the primary video data stream and the secondary data stream, and the rate control unit of the at least one video output module comprises:

means to read the secondary data stream;

means to read data related to how many bits are used after variable length coding;

means to process the secondary data stream with variable length coding information; and

means to control the generalized encoder based on results of processing the variable length coding information, whereby the use of the variable length coding information and the secondary data stream by the generalized encoder improves a speed of encoding and the compressed video output signal'"'"'s quality by improving an output bit allocation.

12. The apparatus of claim 11 wherein the at least one video output module receives the primary video data stream and the secondary data stream, and the rate control unit of the at least one video output module comprises:

means to read feedback data from the generalized encoder;

means to process the secondary data stream with the variable length coding information and the feedback data; and

means to control the generalized encoder based on results of processing the secondary data stream with the variable length coding information and the feedback data.

13. The apparatus of claim 1, further comprising:

means to route the primary video data from the at least one video input module to the at least one video output module; and

means to route the secondary data stream from the at least one video input module to the at least one video output module.

14. The apparatus of claim 13, wherein the means to route the primary video data stream includes a common interface selected from a group consisting of:

shared memory, an ATM bus, a TDM bus, switching, and a direct connection.

15. The apparatus of claim 13, wherein the means to route the secondary data stream includes a common interface selected from a group consisting of:

shared memory, an ATM bus, a TDM bus, switching, and a direct connection.

16. The apparatus of claim 1, wherein the manipulation of the compressed video input data includes at least one type of manipulation selected from a group consisting of:

transcoding and compositing.

17. The apparatus of claim 1, wherein the secondary data stream is associated with the primary video data stream in that the secondary data stream includes side information.

18. The apparatus of claim 1, wherein the compressed video input data includes at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter indication, DCT coefficients and quantizer values.

19. The apparatus of claim 1, wherein the primary video data stream includes information in a DCT domain.

20. The apparatus of claim 1, wherein the primary video data stream includes information in a spatial domain.

21. An apparatus for manipulating compressed digital video forming manipulated compressed digital video, the manipulated compressed digital video being a manipulation of data from at least one of a plurality of compressed digital video sources and destinations, the apparatus comprising:

at least one video input module, each video input module of the at least one video input module being operative to receive a compressed video input signal that belongs to one of the compressed digital video sources depending on the required manipulation, to decode the compressed video input signal for generating a decoded video data stream and to transfer the decoded video data stream to a common interface;

at least one video output module, each video output module of the at least one video output module being operative to grab the decoded video data stream from the common interface, to encode the decoded video data stream into a compressed video output stream, and to transfer the compressed video output stream to at least one destination of the plurality of destinations; and

a common interface forming a temporary logical connection for routing the decoded video data stream from at least one input module to at least one output module;

wherein there is no permanent logical relation or connection between the at least one video input module and the at least one video output module, and the apparatus has a configuration in which the temporary logical connection depends on the current needs of a current manipulation, whereby use of the configuration improves resources allocation of the apparatus.

22. The apparatus of claim 21, wherein the manipulation of the compressed video input data includes at least one type of manipulations selected from a group consisting of:

transcoding and compositing.

23. A compressed video combiner unit for generating a compressed digital video signal, which is a composition of plurality of compressed digital video sources, the compressed video combiner unit comprising:

at least one video input module for receiving compressed video input data from at least one source of the plurality of compressed digital video sources, the at least one video input module further comprising

a generalized decoder operative to decode the compressed video input data and generate a primary video data stream, the generalized decoder further comprising

a data processing unit operative to process the compressed video input data and the primary video data stream to generate a secondary data stream, the secondary data stream having an association with the primary video stream forming associated secondary data;

at least one video output module operative to receive at least one of the primary video data stream and the secondary data stream, the at least one video output module further comprising

a rate control unit, and

a generalized encoder, in communication with the rate control unit and operative to receive the primary video data from the at least one video input module and encode the primary video data into compressed video output data;

means to route the primary video data from the at least one video input module to the at least one video output module; and

means to route the secondary data stream from the at least one video input module to the at least one video output module;

whereby the use of the secondary data stream by the at least one video output module improves a speed of encoding and the compressed video output data'"'"'s quality.

24. The compressed video combiner unit of claim 23, wherein the association between the secondary data stream and the primary video data stream is that the secondary data stream includes side information.

25. The compressed video combiner unit of claim 24, wherein the side information includes at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter usage indication, quantizer identifications, coded/uncoded decisions, an amount of information within each macroblock, image segmentation indication, scene cut off indication, camera zoom identification, camera pan identification, camera movements identification, and statistical information.

26. The compressed video combiner unit of claim 23, wherein the compressed video input data includes at least one type of information selected from a group consisting of:

frame type, resolution, motion vectors, filter indication, DCT coefficients, and quantizer values.

27. The compressed video combiner unit of claim 23, wherein the rate control unit comprises:

means to read the secondary data stream;

means to process the secondary data stream; and

means to control the generalized encoder based upon results of processing the secondary data stream.

28. The compressed video combiner unit of claim 23, wherein the rate control unit comprises:

means to read feedback data from the generalized encoder;

means to process the secondary data stream with the feedback data; and

means to control the generalized encoder based upon results of processing the secondary data stream with the feedback data.

29. The compressed video combiner unit of claim 23, wherein the means to route the primary video data stream includes a common interface selected from a group consisting of:

shared memory, an ATM bus, a TDM bus, switching, and a direct connection.

30. The compressed video combiner unit of claim 23, wherein the means to route the secondary data stream includes a common interface selected from a group consisting of:

shared memory, an ATM bus, a TDM bus, switching, and a direct connection.

31. The compressed video combiner unit of claim 23, wherein the primary video data stream includes information in a DCT domain.

32. The compressed video combiner of claim 23, wherein the primary video data stream includes information in a spatial domain.

33. The compressed video combiner unit of claim 23, wherein the video output module receives at least one of the primary video data streams, the associated secondary data stream, and control information from an external device.

34. The compressed video combiner unit of claim 33, wherein the rate control unit of the video output module comprises:

means to read the secondary data stream;

means to read the control information;

means to process the secondary data stream;

means to process the control information; and

means to control the generalized encoder based upon results of processing secondary data stream with results of processing control information.

35. The compressed video combiner unit of claim 34, wherein the rate control unit of the video output module comprises:

means to read feedback data from the generalized encoder;

means to process the secondary data stream with the control information and the feedback data; and

means to control the generalized encoder based upon results of processing the secondary data stream with the control information and the feedback data.

36. The compressed video combiner unit of claim 33, wherein the control information includes at least one type of information selected from a group consisting of:

a region of interest indication, screen layout requirements, user quality preferences, and special effects.

37. The compressed video combiner unit of claim 33, wherein the control information is bi-directional information.

38. The compressed video combiner unit of claim 23 wherein the at least one video output module receives the primary video data stream and the secondary data stream, and the rate control unit of the at least one video output module comprises:

means to read the secondary data stream;

means to read data related to how many bits are in use after variable length coding;

means to process the secondary data stream with the variable length coding information; and

means to control the generalized encoder based on results of processing the secondary data stream with the variable length coding information, whereby the use of the variable length coding information and the secondary data stream by the generalized encoder improves a speed of encoding and the compressed video output signal'"'"'s quality by improving an output bit allocation.

39. The compressed video combiner unit of claim 38 wherein the at least one video output module receives the primary video data stream and the secondary data stream, and the rate control unit of the at least one video output module comprises:

means to read feedback data from the generalized encoder;

means to process the secondary data stream with the variable length coding information and the feedback data; and

means to control the generalized encoder based on results of processing the secondary data stream with the variable length coding information and the feedback data.

40. An apparatus for manipulating compressed digital video forming manipulated compressed digital video, the manipulated compressed digital video being a manipulation of data from at least one of a plurality of compressed digital video sources and destinations, the apparatus comprising:

at least one video input module, each video input module of the at least one video input module being operative to receive a compressed video input signal that belongs to one of the compressed digital video sources depending on the required manipulation, to decode the compressed video input signal for generating a decoded video data stream and to transfer the decoded video data stream to a common interface;

at least one video output module, each video output module of the at least one video output module being operative to grab the decoded video data stream from the common interface, to encode the decoded video data stream into a compressed video output stream, and to transfer the compressed video output stream to at least one destination of the plurality of destinations; and

a common interface forming a non-dedicated connection for routing the decoded video data stream from at least one video input module to at least one video output module;

wherein there is no dedicated logical relation or connection between the at least one video input module, and the at least one video output module and the apparatus has a configuration in which the non-dedicated logical connection depends on the current needs of a current manipulation, whereby use of the configuration improves resources allocation of the apparatus.