Classifying image areas of a video signal

• US 7,715,477 B2
• Filed: 05/28/2003
• Issued: 05/11/2010
• Est. Priority Date: 05/29/2002
• Status: Expired due to Fees

• Current Assignee: Video 264 Innovations, LLC
• Original Assignee: Innovation Management Sciences, LLC
• Inventors: Fogg, Chad, Butler, Simon, Webb, Richard, Garrido, Diego
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US10/447,296
• Publication Number:
  US 20070230914A1
• Time in Patent Office: 2,540 Days
• Field of Search: 375/240.25, 375/240.26, 375/240.12, 375/240.13, 375/240.23, 375/240.24, 375/240.28
• US Class Current: 375/240.11
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/94 : Vector quantisation
• CPC Class Codes: H04N 7/0125 : one of the standards being a high definition standard
• CPC Class Codes: H04N 7/0137 : dependent on presence/absence of motion, e.g. of motion zones H04N7/014 takes precedence; movement detection in television signals H04N5/144
• CPC Class Codes: H04N 7/014 : involving the use of motion vectors motion estimation and compensation in video coding H04N19/51
• CPC Class Codes: H04N 7/0145 : the interpolation being class adaptive, i.e. it uses the information of class which is determined for a pixel based upon certain characteristics of the neighbouring pixels
• CPC Class Codes: H04N 9/8042 : involving data reduction

Abstract

A method of enhancing picture quality of a video signal is disclosed. The method comprises the steps of receiving base layer images of standard definition pictures from a base layer decoder; defining image areas of the standard definition pictures; classifying image areas into image types by assigning a class number; and generating enhanced pictures based upon the standard definition pictures and the classification of the image areas. A circuit for enhancing picture quality of a video signal is also disclosed. The circuit comprising a base layer decoder; a classifier coupled to the base layer decoder, the classifier generating a class number for image areas of a standard definition picture; a summing circuit coupled to the classifier; an exchange stream decoder coupled to the summing circuit, the exchange stream decoder generating an index; and a codebook table coupled to the summing circuit. The codebook table preferably stores a plurality of codevectors based upon the class number and the index.

Claims

1. A method of enhancing picture quality of a video signal, said method comprising the steps of:

receiving base layer of standard definition pictures from a base layer decoder;

generating interpolated images via an interpolator in accordance with said standard definition pictures and predefined parameters;

defining image areas of said standard definition pictures;

classifying said defined image areas into image types by assigning a class number;

using said class number to obtain from memory at least one enhancement step, said enhancement step corresponding to a specific one of said image areas; and

generating enhanced pictures in response to at least one class number and an least one index, and based upon said interpolated images and said enhancement step as applied to said corresponding image area.

2. The method of claim 1 wherein said step of generating enhanced images comprises generating a difference image from said standard definition pictures and said enhancement step, and adding base images and difference images.

3. The method of claim 2 further comprising a step of generating said difference images containing information that is present only in said high-definition pictures.

4. The method of claim 3 further comprising a step of prioritizing image areas that need to be enhanced.

5. The method of claim 1 wherein said step of classifying image areas comprises using different coding tools for different classes of picture data.

6. The method of claim 1 wherein said step of receiving base layer images of standard definition pictures comprises receiving standard definition pictures derived from high-definition pictures through filtering and subsampling.

7. The method of enhancing picture quality of a video signal of claim 1 wherein said step of receiving base images of standard definition pictures comprises receiving base images coded with a transform coder.

8. The method of enhancing picture quality of a video signal of claim 1 wherein said step of receiving base images of standard definition pictures comprises receiving base images coded with MPEG.

9. The method of enhancing picture quality of a video signal of claim 8 wherein said step of receiving base images of standard definition pictures comprises receiving base images coded based upon a standard stream on a DVD.

10. The method of enhancing picture quality of a video signal of claim 1 wherein said step of classifying image areas comprises using block-based frequency and spatial domain operations.

11. A method of enhancing picture quality of a video signal comprising the steps of:

receiving base images of standard definition pictures from a base layer decoder;

defining image areas of said base images of standard definition pictures;

classifying said image areas into image types based upon at least in part an index;

receiving a partitioned codebook table based upon said classes of image types;

generating first enhanced pictures based upon entries in said partitioned codebook identified by the classification of image areas and an enhancement stream vector;

activating a run-time process to generate second enhanced pictures in response to pre-processed high definition signals and base layer signals; and

generating a packetized elementary stream in response to said first enhanced pictures and said second enhanced pictures.

12. The method of claim 11 wherein said step of receiving base layer images of standard definition pictures comprises receiving standard definition pictures derived from high definition pictures Through filtering and subsampling.

13. The method of claim 12 further comprising a step of generating differenced data based upon said high definition pictures.

14. The method of claim 13 further comprises a step of generating a vector quantization codebook based upon said high definition pictures.

15. The method of claim 11 wherein said step of classifying image areas into image types comprising assigning a class number to each image area.

16. The method of claim 15 wherein said step of classifying image areas into image types comprises assigning a separate sub-table codevector index to each image area.

17. The method of claim 16 further comprising a step of prioritizing image areas that need to be enhanced.

18. A circuit for enhancing picture quality of a video signal, said circuit comprising:

a base layer decoder capable of outputting a standard definition picture;

a classifier coupled to said base layer decoder, said classifier generating a class number for image areas of said standard definition picture;

an exchange stream decoder coupled to said classifier, said exchange stream decoder generating an index;

a codebook table coupled to said exchange stream decoder, said codebook table storing a plurality of codevectors based upon said class number and said index; and

a summing circuit coupled to said codebook table and configured to generate a reconstruction signal in response to one or more codevectors and said standard definition picture.

19. The circuit of claim 18 further comprising an interpolator coupled to said base layer decoder.

20. The circuit of claim 19 wherein said interpolator comprises a temporal predictive interpolator.

21. The circuit of claim 19 wherein said interpolator comprises a circuit for providing motion compensation.

22. The circuit of claim 19 wherein said interpolator generates an interpolated image based upon said base image of standard definition pictures.

23. The circuit of claim 22 further comprising a difference block based upon a codevector in said codebook table.

24. The circuit of claim 19 further comprising an enhanced block based upon said interpolated block and said difference block.

25. The circuit of claim 18 wherein said codebook tables comprises classes of codevectors.

26. The circuit of claim 18 wherein said decoder further comprises an encoder.

27. The circuit of claim 26 wherein said classes are based upon common properties measured on base images and previously enhanced images on the decoder.

28. The circuit of claim 27 wherein said classes are identified in enhancement stream headers.

29. A circuit for enhancing picture quality of a video signal, said circuit comprising:

base layer decoder means capable of outputting a standard definition picture;

classifier means coupled to said base layer decoder means, said classifier means generating a class number for image areas of said standard definition picture;

exchange stream decoder means coupled to said classifier means, said exchange stream decoder means generating an index;

a codebook table coupled to said classifier means, said codebook table storing a plurality of codevectors based upon said class number and said index; and

a summing circuit means coupled to said codebook table and configured to generate a reconstruction signal in response to one or more codevectors and said standard definition picture.

30. The circuit of claim 29 further comprising a checksum sent in an enhancement stream header.

31. The circuit of claim 30 wherein said checksum indicates for predetermined sets of blocks corresponding to said enhancement stream header whether a receiver has derived the same set of classes derived by an encoder.

32. The circuit of claim 31 wherein said enhancement stream headers comprise thresholds and ranges for classification.