Method and apparatus for compositing compressed video data

• US 5,408,274 A
• Filed: 03/11/1993
• Issued: 04/18/1995
• Est. Priority Date: 03/11/1993
• Status: Expired due to Fees

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Chang, Shih-Fu, Chen, Wen-Lung, Messerschmitt, David
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Le, Vu
• Application Number:
  US08/029,435
• Time in Patent Office: 768 Days
• Field of Search: 358/183, 358/136, 358/450, 358/451, 358/181, 11/160, 11/93, 11/95, 348/580, 348/581, 348/584, 348/586, 348/578, 348/407, 348/699, 348/700
• US Class Current: 348/700
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

Inverse motion compensating devices are used to transform motion compensated transform domain video signals to corresponding transform domain video signals. A compositing mechanism receives the transform domain video signals from the inverse motion compensating devices and performs transform domain video compositing operations directly on the transform domain video signals. The compositing operations result in a single composited signal incorporating each of the transform domain video signals. The single composited signal may be encoded for motion compensation to generate a single motion compensated transform domain video signal suitable for transmission. After transmission, inverse transform domain operations and inverse motion compensation operations may be performed to reconstruct the composited signal. The composited signal may then be projected on a video display device.

Claims

1. An apparatus for processing a first motion compensated transform domain video signal and a second motion compensated transform domain video signal, said apparatus comprising:

an inverse motion compensating device performing inverse motion compensation on said first motion compensated transform domain video signal and said second motion compensated transform domain video signal to obtain a first transform domain video signal and a second transform domain video signal; and

a compositing mechanism coupled to said inverse motion compensating device, said compositing mechanism performing transform domain video compositing operations on said first transform domain video signal and on said second transform domain video signal to obtain a transform domain composited video signal.

2. The apparatus of claim 1 wherein said inverse motion compensating device includes a reference block calculator for generating an inverse motion compensated transform domain signal corresponding to said first motion compensated transform domain video signal.

3. The apparatus of claim 2 further comprising:

a transform domain object translator for calculating non-grid aligned reference blocks within a grid of a first video frame of said first transform domain video signal to obtain a grid aligned current block in a grid of a second video frame of said first transform domain video signal.

4. The apparatus of claim 1 further comprising:

a motion compensator for transforming said transform domain composited video signal to a motion compensated transform domain composited video signal.

5. The apparatus of claim 1 wherein said compositing mechanism includes:

a compositing unit object translator for matching a grid aligned block of a first video frame with a block of a second video frame.

6. The apparatus of claim 1 wherein said compositing mechanism includes:

an overlap execution unit for positioning a first video object from a frame of-said first transform domain video signal over a second video object from a frame of said second transform domain video signal.

7. The apparatus of claim 1 wherein said compositing mechanism includes:

a pixel-wise multiplier to execute pixel multiplication between pixels within a frame of said first transform domain video signal and pixels within a frame of said second transform domain video signal.

8. The apparatus of claim 1 wherein said compositing mechanism includes:

a filter for performing filtering operations on said first transform domain video signal and on said second transform domain video signal.

9. The apparatus of claim 1 wherein said compositing mechanism includes:

a scaling unit for adjusting the size of image objects in said transform domain composited video signal.

10. A method of processing a first video signal and a second video signal to form a single composited video signal, said method comprising the steps of:

encoding said first video signal at a first location to form a first motion compensated transform domain video signal;

encoding said second video signal at a second location to form a second motion compensated transform domain video signal;

transmitting said first motion compensated transform domain video signal to a third location;

transmitting said second motion compensated transform domain video signal to said third location;

executing an inverse motion compensating operation at said third location to obtain a first transform domain video signal and a second transform domain video signal; and

performing transform domain video compositing operations on said first transform domain video signal and on said second transform domain video signal to obtain a transform domain composited video signal.

11. The method of claim 10 further comprising the step of:

encoding said transform domain composited video signal to form a motion compensated transform domain composed video signal; and

transmitting said motion compensated transform domain composited video signal to a fourth location.

12. The method of claim 11 further comprising the step of

effecting an inverse transform domain operation and an inverse motion compensation operation on said motion compensated transform domain composited video signal to yield a reconstructed composited video signal;

and

displaying said reconstructed composited video signal.

13. An apparatus for processing a first motion compensated transform domain video signal and a second motion compensated transform domain video signal, said apparatus comprising:

an inverse motion compensating device performing inverse motion compensation on said first motion compensated transform domain video signal and said second motion compensated transform domain video signal to obtain a first transform domain video signal and a second transform domain video signal, said inverse motion compensating device including a transform domain object translator for calculating non-grid aligned reference blocks within a grid of a first video frame of said first transform domain video signal to render a grid aligned current block in a grid of a second video frame of said first transform domain video signal; and

a compositing mechanism coupled to said inverse motion compensating device, said compositing mechanism performing transform domain video compositing operations on said first transform domain video signal and on said second transform domain video signal to obtain a transform domain composited video signal.

14. The apparatus of claim 13 wherein said first motion compensated transform domain video signal is a first Motion Compensated/Discrete Cosine Transform (MC/DCT) video signal and said second motion compensated transform domain video signal is a second MC/DCT video signal.

15. The apparatus of claim 14 wherein said transform domain object translator generates DCT coefficients for said non-grid aligned reference blocks based upon DCT coefficients of corresponding adjacent grid-aligned blocks encompassing each of said non-grid aligned reference blocks.

16. The apparatus of claim 13 further comprising:

a motion compensator for transforming said transform domain composited video signal to a motion compensated transform domain composited video signal.

17. The apparatus of claim 16 wherein said motion compensator includes a transform domain object translator for calculating non-grid aligned reference blocks within a grid of a first video frame of said first transform domain video signal to obtain a grid aligned current block in a grid of a second video frame of said first transform domain video signal.

18. The apparatus of claim 13 wherein said compositing mechanism includes:

a compositing unit object translator for matching a grid aligned block of a first video frame with a block of a second video frame.

19. The apparatus of claim 13 wherein said compositing mechanism includes:

an overlap execution unit for positioning a first video object from a frame of said first transform domain video signal over a second video object from a frame of said second transform domain video signal.

20. The apparatus of claim 13 wherein said compositing mechanism includes:

a pixel-wise multiplier to execute pixel multiplication between pixels within a frame of said first transform domain video signal and pixels within a frame of said second transform domain video signal.

21. The apparatus of claim 13 wherein said compositing mechanism includes:

a filter for performing filtering operations on said first transform domain video signal and on said second transform domain video signal.

22. The apparatus of claim 13 wherein said compositing mechanism includes:

a scaling unit for adjusting the size of image objects in said transform domain composited video signal.