Video interpolation coding

• US 20040022318A1
• Filed: 05/28/2003
• Published: 02/05/2004
• Est. Priority Date: 05/29/2002
• Status: Active Grant

• Current Assignee: Video 264 Innovations, LLC
• Original Assignee: Innovation Management Sciences, LLC
• Inventors: Fogg, Chad, Butler, Simon, Webb, Richard, Garrido, Diego
• Granted Patent: US 7,656,950 B2
• Time in Patent Office: Days
• US Class Current: 375/240.11
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/94 : Vector quantisation
• CPC Class Codes: H04N 7/0125 : one of the standards being a high definition standard
• CPC Class Codes: H04N 7/0137 : dependent on presence/absence of motion, e.g. of motion zones H04N7/014 takes precedence; movement detection in television signals H04N5/144
• CPC Class Codes: H04N 7/014 : involving the use of motion vectors motion estimation and compensation in video coding H04N19/51
• CPC Class Codes: H04N 7/0145 : the interpolation being class adaptive, i.e. it uses the information of class which is determined for a pixel based upon certain characteristics of the neighbouring pixels
• CPC Class Codes: H04N 9/8042 : involving data reduction

Abstract

A method of enhancing picture quality of a video signal is described. The method comprising steps of receiving base images of pictures having a first definition from a base layer decoder; coding the differences between the base images of pictures and pictures having a second definition using vector quantization; creating a database of codebooks based upon the differences; and generating enhanced images based upon the base images and enhancement stream data. A circuit for enhancing picture quality of a video signal is also described. The circuit comprises a base layer decoder generating a base image of a standard definition picture; an interpolator coupled to the base layer decoder and generating an interpolated block; a classifier coupled to the base layer decoder and generating a class number; and a summing circuit coupled to the interpolator and the classifier. The summing circuit preferably adds the interpolated block and a difference block.

Claims

1. A method of enhancing picture quality of a video signal, said method comprising the steps of:

receiving base images of pictures having a first definition from a base layer decoder;

coding the differences between said base images of pictures having a first definition and pictures having a second definition using vector quantization;

creating a database of codebooks based upon said differences between said base images of pictures having a first definition and pictures having a second definition; and

generating enhanced images based upon said base images of said pictures having a first definition and enhancement stream data.

2. The method of claim 1 further comprising a step of generating interpolated blocks based upon said base images.

3. The method of claim 1 further comprising a step of classifying said base images.

4. The method of claim 3 wherein said step of classifying said base image comprises assigning a class number and a codevector index to each region of said base image.

5. The method of claim 1 wherein said step of creating a database of codebooks comprises generating a codebook table.

6. The method of claim 5 wherein said step of generating a codebook table comprises a step of classifying image areas having common codevectors.

7. The method of claim 2 further comprising a step of generating a difference block.

8. The method of claim 7 wherein said step of generating enhanced images comprises adding interpolated blocks and said difference blocks.

9. The method of claim 1 further comprising a step of generating an enhancement stream containing enhancement data.

10. The method of claim 1 wherein said step of receiving base images of pictures having a first definition comprises receiving base images coded with a transform coder.

11. The method of claim 1 wherein said step of receiving base images of pictures having a first definition comprises receiving base images coded with MPEG on a DVD.

12. The method of claim 1 wherein said step of analyzing the differences between said base images of pictures having a first definition and pictures having a second definition using vector quantization comprises a step of using a Generalized Lloyd Algorithm.

13. The method of claim 1 wherein said step of analyzing the differences between said base images of pictures having a first definition and pictures having a second definition using vector quantization comprises a step of using a Pair-wise Nearest Neighbor Algorithm.

14. The method of claim 1 wherein said step of analyzing the differences between said base images of pictures having a first definition and pictures having a second definition using vector quantization comprises a step of using a BFOS algorithm.

15. The method of claim 1 wherein said step of analyzing the differences between said base images of pictures having a first definition and pictures having a second definition using vector quantization comprises a step of using a combination of a Generalized Lloyd Algorithm, a Pair-wise Nearest Neighbor Algorithm, and BFOS algorithm continuously applied to each screen of said video signal.

16. The method of claim 1 wherein said step of receiving base images of pictures having a first definition comprises receiving standard definition picture having a resolution from a group consisting of:

720×

480;

704×

480

704×

576; and

720×

576.

17. The method of claim 18 wherein said step of coding the differences between said base images of pictures having a first definition and pictures having a second definition comprises coding the differences between said base images of pictures having a first definition and pictures having a resolution from a group consisting of:

1920×

1080;

1440×

960

1440×

1152; and

1920×

1152.

18. The method of claim 1 wherein said step of generating enhanced images based upon said base images of standard definition pictures and enhancement stream data comprises a step of generating enhanced images based upon said base images of standard definition pictures, codebook data and codevector indexes.

19. A method of enhancing picture quality of a video signal, said method comprising the steps of:

analyzing the differences between said image of standard definition pictures and high definition pictures;

creating a database of codebooks based upon said differences between said images of standard definition pictures and high definition pictures;

receiving base images of standard definition pictures from a base layer decoder;

generating an interpolated block based upon said base images of standard definition pictures;

generating a difference block based upon said codebook; and

generating enhanced images based upon said standard definition images by adding said interpolated block and said difference block.

20. The method of claim 19 further comprising a step of classifying said base images.

21. The method of claim 20 wherein said step of classifying said base image comprises assigning a class number to each region of said base image.

22. The method of claim 19 wherein said step of creating a database of codebooks comprises generating a codebook table.

23. The method of claim 22 wherein said step of generating a codebook table comprises a step of classifying images having common codevectors.

24. The method of claim 22 further comprising a step of encoding a codevector index.

25. A circuit for enhancing picture quality of a video signal, said circuit comprising:

a base layer decoder generating a base image of a standard definition picture;

an interpolator coupled to said base layer decoder and generating an interpolated block;

a classifier coupled to said base layer decoder and generating a class number; and

a summing circuit coupled to said interpolator and said classifier, said summing circuit adding said interpolated block and a difference block.

26. The circuit of claim 25 wherein said interpolator comprises a temporal predictive interpolator.

27. The circuit of claim 25 wherein said interpolator comprises a circuit for providing motion compensation.

28. The circuit of claim 25 further comprising a second summing circuit coupled to said classifier and an index from an enhance stream decoder.

29. The circuit of claim 25 further comprising a codebook table coupled to said second summary circuit.

30. The circuit of claim 29 wherein said codebook tables comprise classes of codevectors.

31. The circuit of claim 30 wherein said classes are based upon properties measured on base images and previously enhanced images in the decoder.

32. The circuit of claim 25 further comprising an enhanced picture based upon said base image of a standard definition picture.

33. A circuit for enhancing picture quality of a video signal, said circuit comprising:

base layer decoder means generating a base image of a standard definition picture;

temporal predictive interpolator means coupled to said base layer decoder means and generating an interpolated block;

classifier means coupled to said base layer decoder means and generating a class number; and

summing circuit means coupled to said temporal predictive interpolator means and classifier means, said summing circuit means adding said interpolated block and a difference block.