Methods and apparatus for decoding and displaying multiple digital images in parallel

• US 20020176508A1
• Filed: 04/30/2002
• Published: 11/28/2002
• Est. Priority Date: 10/11/1994
• Status: Active Grant

• Current Assignee: Jill Macdonald Boyce, Larry Pearlstein
• Original Assignee: Jill Macdonald Boyce, Larry Pearlstein
• Inventors: Boyce, Jill MacDonald, Pearlstein, Larry
• Granted Patent: US 7,173,970 B2
• Time in Patent Office: Days
• US Class Current: 375/240.25
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/4084 in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
• CPC Class Codes: G11B 15/125 conditioned by the operating function of the apparatus
• CPC Class Codes: G11B 15/1875 adaptations for special effects or editing signal processing or indexing therefor G11B27/00
• CPC Class Codes: G11B 15/4673 by controlling the speed of the tape while the head is rotating
• CPC Class Codes: G11B 5/0086 using cyclically driven heads providing segmented tracks
• CPC Class Codes: H03M 7/00 Conversion of a code where information is represented by a given sequence or number of digits to a code where the same , similar or subset of information is represented by a different sequence or number of digits
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 : the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/184 : the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/192 : the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/30 : using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/40 : using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/42 : characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/423 : characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/427 : Display on the fly, e.g. simultaneous writing to and reading from decoding memory
• CPC Class Codes: H04N 19/428 : Recompression, e.g. by spatial or temporal decimation
• CPC Class Codes: H04N 19/44 : Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 : Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/48 : using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/51 : Motion estimation or motion compensation
• CPC Class Codes: H04N 19/513 : Processing of motion vectors
• CPC Class Codes: H04N 19/587 : involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/59 : involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/70 : characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 : Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 : involving filtering within a prediction loop
• CPC Class Codes: H04N 19/90 : using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 19/91 : Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 21/23106 : involving caching operations prefetching while addressing of a memory level in which the access to the desired data or data block requires associative addressing means within memory systems or architectures G06F12/0862; caching at an intermediate stage in a data network H04L67/568
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/236 : Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/2662 : Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/426 : Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4325 : by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/434 : Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/4382 : Demodulation or channel decoding, e.g. QPSK demodulation analog front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/4402 : involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/440254 : by altering signal-to-noise parameters, e.g. requantization
• CPC Class Codes: H04N 21/440263 : by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 21/4621 : Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 5/45 : Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 5/46 : for receiving on more than one standard at will deflecting circuits of multi-standard receivers H04N3/27
• CPC Class Codes: H04N 5/78263 : for recording on tracks inclined relative to the direction of movement of the tape
• CPC Class Codes: H04N 9/8042 : involving data reduction
• CPC Class Codes: H04N 9/8227 : the additional signal being at least another television signal

Abstract

Methods and apparatus for implementing a reduced cost HDTV/SDTV video decoder are disclosed. The described joint video decoder is capable of decoding HDTV pictures at approximately the resolution of standard definition television pictures and can be used to decode HDTV and/or SDTV pictures. The described video decoder may be used as part of a picture-in-picture decoder circuit for providing picture-in-picture capability without providing multiple full resolution video decoders. The reduction in decoder circuit complexity is achieved through the use of a plurality of data reduction techniques including the use of a preparser, downsampling, and truncating pixel values.

Claims

1. A decoder circuit for decoding digital video signals including one or a high definition and a standard definition digital video signal, wherein the high definition video signal includes digital video data representing high definition pictures and the standard definition digital video signal includes digital video data representing standard definition pictures, the decoder circuit comprising:

a channel buffer for temporarily storing the digital video data;

a downsampler module coupled to the channel buffer, the downsampler module being adapted for receiving a code control signal having one of a first state and a second state, the downsampler module downsampling and outputting the digital video data representing the pictures when the mode control signal is in the first state and outputting the digital video data representing the pictures without performing downsampling when the mode control signal is in the second state; and

a picture memory coupled to the downsampler module for storing the digital video data representing the pictures output by the downsampler module.

2. The decoder circuit of claim 1, wherein

the first state of the mode control signal indicates a high definition video mode of decoder operation and the second state of the mode control signal indicates a standard definition video mode of decoder operation.

3. The decoder circuit of claim 2, wherein the channel buffer and the picture memory are implemented using a common memory device including first, second and third memory circuits, and wherein:

during the standard definition mode of decoder operation, the first memory circuit operates as the channel buffer, and the second and third memory circuits operate as the picture memory; and

during the high definition mode of decoder operation, the first and second memory circuits operate as the channel buffer, and the third memory circuit operates as the frame buffer.

4. The decoder circuit of claim 3, further comprising:

an interpolation module coupled to the picture memory for receiving the data representing the pictures stored therein, the interpolation module including;

a half pel interpolation circuit for performing half pel interpolation on the data representing the video pictures;

an upsampler circuit for upsampling the data representing the video pictures;

a downsampler circuit for downsampling the upsampled data representing the video pictures; and

coupling means for selectively coupling the picture memory in series with the upsampler circuit, the half pel interpolation circuit and the downsampler circuit in response to the first state of the mode control signal and for coupling the picture memory in series with the half pel interpolation circuit such that the upsampler circuit and the downsampler circuit are bypassed in response to the second state of the mode control signal.

5. The decoder circuit of claim 4, wherein each of the high definition pictures and the standard definition pictures include a plurality of blocks, each block including a plurality of DCT coefficients, the decoder circuit comprising:

preparser means coupled to the channel buffer for removing from each block of a high definition picture DCT coefficients teat exceed a preselected maximum number of DCT coefficients per block and to pass blocks of standard definition pictures without removing DCT coefficients.

6. The decoder circuit of claim 4, wherein the data representing each of the high definition pictures and the standard definition pictures includes a plurality of blocks, each block including a plurality of DCT coefficients, and wherein the standard definition pictures are transmitted at a variable data rate, the decoder circuit comprising:

preparser means coupled to the channel buffer for removing from each block of a high or standard definition picture, DCT coefficients that exceed a preselected maximum number of DCT coefficients per block.

7. The decoder circuit of claim 6, wherein the channel buffer is coupled to the preparser means by a feedback line and wherein the channel buffer signals the preparser means to control the output data rate of the preparser means as a function of the amount of data stored in the channel buffer and the mode of decoder operation as indicated by the state of the mode control signal.

8. The decoder circuit of claim 7, wherein the number of DCT coefficients included in the blocks of the data representing standard definition pictures is less than or equal to a first number of DCT coefficients and wherein the number of DCT coefficients included in the blocks of the data representing high definition pictures is less than or equal to a second number of DCT coefficients, the decoder circuit further comprising:

a syntax parser and variable length decoder circuit coupled to the channel buffer; and

an inverse quantization circuit coupled to the syntax parser and variable length decoder circuit for inverse quantizing a third number of DCT coefficients per block, the third number of DCT coefficients being less than the second number of DCT coefficients and greater than or equal to the first number of DCT coefficients.

9. The decoder circuit of claim 8, wherein the first number of DCT coefficients per block is less than 11 and wherein the second number of DCT coefficients per block is less than 65.

10. The decoder circuit of claim 8, wherein the DCT coefficients of each high definition or standard definition picture represent a 2 dimensional array of blocks including a series of columns of DCT coefficients, the decoder circuit comprising:

an IDCT engine comprising one dimensional 8 point IDCT circuits for performing IDCT operations on a subset of the columns of DCT coefficients included in each high definition picture and all of the columns of DCT coefficients of each block of each standard definition picture, the same one dimensional 8 point IDCT circuits used to process the subset of columns of DCT coefficients included in the high definition picture being used to perform the IDCT operation on all the columns of DCT coefficients of each block of a standard definition picture.

11. The decoder circuit of claim 10, wherein the subset of subset of the columns of DCT coefficients included in each high definition picture consists of three or less columns of DCT coefficients.

12. The decoder circuit of claim 8, wherein the DCT coefficients of each high definition or standard definition picture represent a 2 dimensional array of blocks including a series of rows of DCT coefficients, the decoder circuit comprising:

an IDCT engine comprising one dimensional 8 point IDCT circuits for performing IDCT operations on a limited subset of the rows of DCT coefficients included in a high definition picture and all of the rows of DCT coefficients of each block of a standard definition picture.

13. The decoder circuit of claim 6, wherein the number of DCT coefficients included in the blocks of the data representing standard definition pictures is less than or equal to a first number of DCT coefficients and wherein the number of DCT coefficients included in the blocks of the data representing high definition pictures is less than or equal to a second number of DCT coefficients, the decoder circuit further comprising:

a syntax parser and variable length decoder circuit coupled to the channel buffer; and

an inverse quantization circuit coupled to the syntax parser and variable length decoder circuit for inverse quantizing a third number of DCT coefficients per block, the third number of DCT coefficients being less than the second number of DCT coefficients and greater than or equal to the first number of DCT coefficients.

14. The decoder circuit of claim 13, wherein the first number of DCT coefficients per block is less than 11 and wherein the second number of DCT coefficients per block is less than 65.

15. The decoder circuit of claim 14, further comprising:

mode control signal generator means including means for detecting the presence of data representing high definition video signals in the video signal, the mode control signal generator outputting the mode control signal having a first state upon detection of the presence of data representing high definition video pictures and outputting the mode control signal having a second state at other times.

16. The decoder circuit of claim 9, further comprising:

mode control signal generator means including means for detecting the presence of data representing high definition video signals in the video signal, the mode control signal generator outputting the mode control signal having a first state upon detection of the presence of data representing high definition video pictures and outputting the mode control signal having a second state at other times.

18. The IDCT engine of claim 18, wherein M is three.

20. The method of claim 19, wherein each high definition picture includes a plurality of blocks, each block including a plurality of DCT coefficients, and wherein the step of processing the received picture for a first time includes the step of:

using a preparser to remove from each block of the received picture, DCT coefficients that exceed a preselected maximum number of DCT coefficients per block.

21. The method of claim 20, wherein the DCT coefficients of a high definition picture correspond to N columns of DCT coefficients, where N is a positive integer, the method further comprising the step of:

generating a mode control signal indicating whether the picture stored in the received picture is a high definition picture or a standard definition picture;

operating an inverse discrete cosine transform circuit responsive to the mode control signal to perform an inverse discrete cosine transform on upto but not exceeding M columns of DCT coefficients of each received high definition picture, where M is a positive integer less than N, and on all the columns of DCT coefficients of each received standard definition picture.

22. The method of claim 21, wherein M is three and N is eight.

23. The method of claim 21, further comprising the step of:

operating a downsampler circuit in response to the mode control signal to downsample each high definition picture upon which an inverse discrete cosine operation is performed and to pass each standard definition picture, without downsampling, upon which an inverse discrete cosine operation is performed.

25. The method of claim 23, wherein the first degree of resolution corresponds to the degree of resolution of a high definition television picture and the second degree of resolution corresponds to the degree of resolution of a standard definition television picture.

17. An IDCT engine for performing an IDCT operation on a block of digital video data corresponding to a high definition picture and for performing an IDCT operation on a block of digital video data corresponding to a standard definition picture, each block of video data corresponding to a high definition picture including a plurality of DCT coefficients representing a series of N rows and M columns of DCT coefficients, each block of video data corresponding to a standard definition picture including a series of rows and columns of DCT coefficients, where N and M are positive integers, the IDCT engine comprising:

a plurality of one dimensional 8 point IDCT circuits for performing IDCT operations on a subset of the N rows and M columns of DCT coefficients, the same plurality of one dimensional 8 point IDCT circuits performing IDCT operations on all rows and columns of DCT coefficients of the standard definition block of video data.

19. A method of decoding a digital video signal including a video picture, the video picture being one of a high definition picture and a standard definition picture, the method comprising the steps of:

receiving the digital video signal including the picture, the picture being represented by digital video data;

detecting whether the received picture is a high definition picture or a standard definition picture;

upon detecting that the received picture is a high definition picture, processing the received picture for a first time to reduce the amount of digital video data used to represent the received picture;

storing the received picture in a memory device, the stored picture corresponding to a processed high definition picture or an unprocessed standard definition picture.

24. A method of decoding a digital video signal including a picture of one of a first and a second degree of resolution, the first degree of resolution being a higher degree of resolution than the second degree of resolution, the picture including digital video data, the method comprising the steps of:

providing a picture buffer for storing pictures;

receiving the digital video signal including the picture;

monitoring the digital video signal to determine whether the received picture is of the first or second degree of resolution;

upon determining that the received picture is of the first degree of resolution performing the steps of;

i. downsampling the picture to reduce the amount of video data included in the picture; and

ii storing the picture in the picture buffer;

upon determining that the received picture is of the second degree of resolution performing the step of storing the picture in the picture buffer without downsampling.