Method and system for multimedia video processing

• US 6,757,005 B1
• Filed: 09/14/2001
• Issued: 06/29/2004
• Est. Priority Date: 01/13/2000
• Status: Expired due to Term

• Current Assignee: Polycom Israel Limited (HP Inc.)
• Original Assignee: Polycom Israel Limited (HP Inc.)
• Inventors: Eshkoli, Noam, Eiesenberg, Aviv, Elbaz, Moshe, Yona, Ilan
• Primary Examiner(s): CHAN, WING F
• Application Number:
  US09/952,339
• Time in Patent Office: 1,019 Days
• Field of Search: 348/14-7-, 379/202.01, 370/259-262, 370/395.21
• US Class Current: 348/14.09
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

An improved method and a system of utilizing the decoding/encoding video resources of a Video Processing Device (VPD) by offering a distributed architecture. A conventional VPD comprises a plurality of video ports, each video port is dedicated to a user and comprises at least one decoder and one encoder. The distributed VPD comprises a plurality of input ports and a plurality of output ports. Each input port includes an input module. The input module is operative to receive a compressed video input stream, manipulate the compressed video stream into a primary stream and optionally generate a secondary data stream associated with the primary data stream. A variety of levels of service for a session can be offered. A client may select the number of ports that will be used by the session. For example, a single port may multicast its compressed output video stream to all the destinations within a session, or to all destinations within a plurality of ports. The multicasting may be such that there is anywhere from one port for each group of destinations using the same compressed video stream to one port for each user.

Claims

1. An apparatus for manipulating compressed digital video messages received from at least one compressed video source of a plurality of compressed video sources to form manipulated compressed video output and for sending the manipulated compressed video output to at least one of a plurality of compressed video destinations, the apparatus comprising:

a common interface;

at least one video input port being operative to grab a compressed video input signal from the at least one compressed video source, to decode the compressed video input signal for generating a video data stream, and to transfer the video data stream to the common interface; and

at least one video output port being operative to grab the video data stream from the common interface, to encode the video data stream forming a compressed video output stream, and to transfer the compressed video output stream to at least one destination of the plurality of compressed video destinations;

wherein no permanent logical relation or connection exists between the at least one video input port and the at least one video output port, and the at least one video input port and the at least one video output port are arranged in a distributed configuration having a temporary logical connection based on current requirements of a current session;

whereby use of the distributed configuration improves resource allocation of the apparatus.

2. The apparatus of claim 1 wherein the at least one video output port grabs the video data stream form the common interface based on a layout associated with a conference.

3. The apparatus of claim 1 wherein the at least one video input port grabs the compressed video input signal from a currently visible video source in at least one layout that is currently being used in a session.

34. The apparatus of claim 1, wherein each video output module composes more than one video data stream into one video data stream before being encoded and transferred to the at least one destination.

4. A method of offering a level of service to a video conference, the method comprising:

offering levels of service to a conference moderator, different levels of the levels of service requiring different amounts of video resources; and

assigning one or more video resources to the video conference based on a selected level of service.

5. The method of claim 4 wherein the offering is based on processing a profile associated with the video conference.

6. The method of claim 4 wherein the one or more video resources is selected from the group consisting of input port, output port, and fat port.

7. The method of claim 4 further comprising:

routing a compressed video input signal of an input endpoint to an input module;

processing the compressed video input signal using the input module;

routing decoded video data of a set of input modules to output modules;

processing the decoded video data to form a single compressed video output signal using one output module of the output modules; and

transferring the single compressed video output signal to a receiving endpoint.

8. The method of claim 5, wherein the offering of levels of service takes into consideration at least one parameter selected from a group of parameters consisting of:

how many different compression standards are involved in the video conference, how many different resolution are involved in the video conference, which type of layout is used in the video conference, how many types of layouts are in use in the video conference, how many different bit rates are in use in the video conference, and how many different frame rates are in use in the video conference.

9. The method of claim 6, wherein the assigning further comprises assigning one output port for each requested compression standard thereby binding a resources allocation associated with a video processing device with needs of the video conference and not with how many participants are in the video conference.

10. The method of claim 6, wherein the assigning further comprises assigning one output port for each requested bit rate.

11. The method of claim 6, wherein the assigning further comprises assigning one output port for each requested layout.

12. The method of claim 6, wherein the assigning further comprises assigning one output port for each requested resolution.

13. The method of claim 6, wherein the assigning further comprises assigning one output port for each requested frame rate.

14. The method of claim 7, wherein the receiving endpoint uses a compression standard that is used for the single compressed video output signal.

15. The method of claim 7, wherein the receiving endpoint uses a bit rate that is used for the single compressed video output signal.

16. The method of claim 7, wherein the receiving endpoint uses a resolution that is used for the single compressed video output signal.

17. The method of claim 7, wherein the receiving endpoint uses a layout that is used for the single compressed video output signal.

18. The method of claim 7, wherein the receiving endpoint uses a frame rate that is used for the single compressed video output signal.

19. The method of claim 6, wherein the assigning further comprises assigning one input port for each currently visible source thereby binding a resources allocation associated with a video processing device with layouts used by the video conference and not with how many participants are in the video conference.

20. The method of claim 7, wherein the output module is dedicated to a layout in which the input module participates.

35. The method of claim 4 further comprising:

routing multiple compressed video input signals from a first set of endpoints to multiple input modules, each compressed video input signal being routed from one of the first set of endpoints to one of the multiple input modules;

the multiple compressed video input signals, the multiple input modules and the fist set of endpoints having a one to one correspondence;

processing the multiple compressed video input signals using the multiple input modules thereby forming decoded video data, each input module processing a different one of the multiple compressed input signals;

routing the decoded video data from the input modules to multiple output modules, each output module may have decoded video data from multiple input modules routed thereto;

processing the decoded video data to form compressed video output signals, each output module forming one compressed video output signal; and

transferring the compressed video output signals from the multiple output modules to a second set of endpoints, each video output module having the single compressed video output signal transferred to one or more endpoints of the second set of endpoints.

21. A method of offering a level of service to a video conference, the method comprising:

a conference moderator selecting a level of service, the video conference using a number of fat ports including at least one fat port, a different level of service requiting a different number of fat ports;

assigning one or more fat ports selected from the number of fat ports to the video conference based on the conference moderator selecting;

routing compressed video input signas of at least one in endpoint to an input module of the one or more fat ports; and

processing the compressed video input signals using the one or more fat ports to form a compressed video output stream; and

transferring the compressed video output stream to at least one receiving endpoint.

22. The method of claim 21, wherein the assigning further comprises assigning one fat port for each requested compression standard.

23. The method of claim 21, wherein the assigning further comprises assigning one fat port for each requested layout.

24. The method of claim 21, wherein the assigning further comprises assigning one fat port for each requested bit rate.

25. The method of claim 21, wherein the assigning further comprises assigning one fat port for each requested resolution.

26. The method of claim 21, wherein the at least one receiving endpoint uses a compression standard that is used by the compressed video output stream.

27. The method of claim 21, wherein the at least one receiving endpoint uses a layout that is associated with the compressed video output stream.

28. The method of claim 21, wherein the at least one receiving endpoint uses a bit rate associated with the compressed video output stream.

29. The method of claim 21, wherein the at least one receiving endpoint has a resolution that is associated with the compressed video output stream.

30. The method of claim 21, wherein the one or more fat ports are assigned to a layout associated with the at least one input endpoint.

31. The method of claim 21, wherein the at least one receiving endpoint is associated with a layout that is generated by the one or more fat ports.

32. The method of claim 21, wherein the assigning further comprises assign one fit port for each requested frame rate.

33. The method of claim 21, wherein the at least one receiving endpoint uses a frame rate generated by the one or more fat ports.

37. The method of claim 21, wherein each fat port is associated with multiple input modules and each input module is associated with no more than one video input signal.

36. The method of clam 35 wherein the first set of endpoints and second set of endpoints are identical as to which endpoints are included.

38. A processor-based video conference system comprising a medium storing instructions for causing the processor to:

offer levels of service to a moderator of a video conference, different levels of the levels of service requiring different amounts of video resources; and

assign one or more video resources selected from the group consisting of input port, output port and fat port to the video conference based on a selected level of service.

39. A processor-based video conference system comprising a medium storing instructions for causing the processor to:

allow a conference moderator of a video conference to select a level of service;

use a number of fat ports including at least one fat port associated with the level of service selected;

route compressed video input signals of at least one endpoint to an input module of an assigned fat port selected from the number of fat ports;

process the compressed video input signals of the at least one endpoint using the assigned fat port forming a compressed video output steam; and

transfer the compressed video output s of each fat port to at least one endpoint.