Apparatus and method of building an electronic database for resolution synthesis

• US 6,466,702 B1
• Filed: 04/21/1998
• Issued: 10/15/2002
• Est. Priority Date: 04/21/1997
• Status: Expired due to Fees

• Current Assignee: Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University))
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Atkins, C. Brian, Bouman, Charles A., Allebach, Jan P.
• Primary Examiner(s): Mancusco, Joseph
• Assistant Examiner(s): MILLER, MARTIN E
• Application Number:
  US09/064,638
• Time in Patent Office: 1,638 Days
• Field of Search: 358/1.2, 382/205, 382/225, 382/260, 382/299, 382/300
• US Class Current: 382/300
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403

Abstract

An electronic database for image interpolation is generated by a computer. The computer generates a low-resolution image from a training image, a plurality of representative vectors from the low-resolution image, and a plurality of interpolation filters corresponding to each of the representative vectors. The interpolation filters and the representative vectors are generated off-line and can be used to perform image interpolation on an image other than the training image. The database can be stored in a device such as computer or a printer.

Claims

1. A method of building an electronic database for data resolution synthesis from at least one training file, the method comprising:

generating a low-resolution file from each training file;

generating a plurality of representative vectors from each low-resolution file, the representative vectors generated by computing a number of cluster vectors from each low-resolution file and using the cluster vectors to compute the representative vectors; and

generating a set of interpolation filters for each of the representative vectors;

wherein low-resolution observation vectors, the cluster vectors, the representative vectors and a high-resolution file corresponding to each low-resolution file are used to compute the interpolation filters, whereby a high resolution file may be a training file.

2. The method of claim 1, further comprising the step of generating a sharpened high-resolution file, the sharpened high-resolution file being used to compute the interpolation filters.

3. The method of claim 1, wherein the representative vectors are generated by using a maximum likelihood estimate.

4. The method of claim 3, wherein the representative vectors are generated by using an expectation maximization technique.

5. The method of claim 3, wherein a classifier including the representative vectors is computed by initializing the classifier and updating the classifier until optimal values for the classifier have been obtained.

6. The method of claim 5, wherein the classifier further includes a variance and a number of class weights, and wherein the representative vectors, the class weights and the variance are computed simultaneously.

7. The method of claim 1, wherein each cluster vector is generated by forming an observation window about sampled data in a low resolution file, extracting a vector including neighboring data of the sampled data, and scaling the vector.

8. The method of claim 1, wherein coefficients for the interpolation filters are computed by:

computing a number of filter design triplets from data in the low-resolution file, each filter design triplet corresponding to sampled data in the low-resolution file, each filter design triplet including an observation vector for the sampled data, a cluster vector for the sampled data, and a vector of high resolution data from a high-resolution file;

computing training statistics from the filter design triplets; and

computing the coefficients from the training statistics.

9. The method of claim 1, wherein the steps are run off-line in a computer.

10. The method of claim 1, wherein the interpolation filters are linear filters.

11. The method of claim 1, wherein the representative vectors are generated by using a parameter optimization technique.

12. A method of using a computer to compute a plurality of resolution synthesis parameters from a training image, the method comprising the steps of:

computing a low-resolution image from the training image;

computing a plurality of cluster vectors for a number of pixels in the low-resolution image;

using the cluster vectors to compute a number of representative vectors for the low resolution image, where that the number of representative vectors is less than the number of cluster vectors; and

using low-resolution observation vectors, the cluster vectors, the representative vectors and vectors from a high-resolution image to compute sets of interpolation filter coefficients corresponding to each of the representative vectors;

whereby the high-resolution image may be the training image; and

whereby the interpolation filter coefficients and the representative vectors are stored in the database for later interpolation of an image other than the training image.

13. The method of claim 12, wherein between 25,000 and 100,000 cluster vectors are computed.

14. The method of claim 12, wherein each cluster vector for a non-border pixel is computed by extracting a first vector from a square observation window centered about a sampled pixel in the low-resolution image, and scaling the first vector.

15. The method of claim 12, where the number of representative vectors is between 50 and 100.

16. The method of claim 12, wherein the representative vectors are computed using a maximum likelihood estimate.

17. The method of claim 16, wherein a classifier including the representative vectors is computed by initializing the classifier and updating the classifier until optimal values for the classifier have been obtained.

18. The method of claim 12, wherein the representative vectors are computed using an expectation-maximization algorithm.

19. The method of claim 18, wherein the representative vectors are computed by:

setting initial values for a classifier including a number of class weights, a variance and the number of representative vectors;

computing a quality measure of how well the cluster vectors are represented by the initial values for the classifier;

updating the classifier;

recomputing the quality measure for the updated classifier; and

determining whether the cluster vectors are suitably represented by the updated classifier, the classifier being updated until the cluster vectors are suitably represented.

20. The method of claim 12, further comprising the step of computing a sharpened high-resolution image from the training image, wherein the sharpened image is used along with low-resolution observation vectors, the cluster vectors and the representative vectors to compute the interpolation filter coefficients.

21. The method of claim 12, wherein the interpolation filter coefficients are computed by:

computing a number of filter design triplets from the low-resolution image, each filter design triplet corresponding to a sampled pixel in the low-resolution image, each filter design triplet including an observation vector for the sampled pixel, a cluster vector for the sampled pixel, and a vector of high-resolution pixels corresponding to the sampled pixel, the high-resolution pixels being taken from the high-resolution image;

computing training statistics from the filter design triplets; and

computing the coefficients from the training statistics.

22. The method of claim 21, wherein between 500,000 and 1,000,000 filter design triplets are computed.

23. The method of claim 21, wherein the interpolation filter coefficients are computed for linear interpolation filters.

24. The method of claim 12, wherein the steps are run off-line in the computer.

25. The method of claim 24, wherein the database is stored for transfer to a second computer, whereby the second computer can access the database to perform image interpolation on images other than the training images.

26. The method of claim 24, wherein the database is stored in memory of a printer, whereby the printer can access the database to perform image interpolation on images other than the training images.

27. The method of claim 12, wherein the representative vectors are generated by using a parameter optimization technique.

28. Apparatus comprising:

a processor; and

memory for storing an electronic database, the memory programmed to cause the processor to access a training file;

generate a low-resolution file from the training file;

generate a plurality of representative vectors from the low-resolution file;

generate a set of interpolation filters for each of the representative vectors; and

store the interpolation filters and the representative vectors in the memory as part of the database;

the processor generating the representative vectors by computing a number of cluster vectors from the low-resolution file, and using the cluster vectors to generate the representative vectors;

the processor generating the interpolation filters from low-resolution observation vectors, the cluster vectors, the representative vectors and a plurality of vectors from a high-resolution file corresponding to the low-resolution file.

29. The apparatus of claim 28, wherein the memory is further programmed to cause the processor to generate a sharpened high-resolution file from the training file, the sharpened high-resolution file being used to compute the interpolation filters.

30. The apparatus of claim 28, wherein the memory is further programmed to cause the processor to generate a classifier including the representative vectors by initializing the classifier and updating the classifier until optimal values for the classifier have been obtained.

31. The apparatus of claim 28, wherein the processor generates each cluster vector by forming an observation window about sampled data in the low-resolution file, extracting a vector including neighboring data of the sampled data, subtracting a value of the sampled data from values of the data in the vector;

and scaling the vector.

32. The apparatus of claim 28, wherein the processor computes coefficients for the interpolation filters by:

computing a number of filter design triplets from data in the low-resolution file, each filter design triplet corresponding to sampled data in the low-resolution file, each filter design triplet including an observation vector for the sampled data, a cluster vector for the sampled data, and a vector of high resolution data from a high-resolution file, the high resolution data corresponding to the sampled data;

computing training statistics from the filter design triplets; and

computing the coefficients from the training statistics.

33. The apparatus of claim 28, wherein the interpolation filters are linear filters.

34. An article for instructing a processor to compute a resolution synthesis database from a training image, the article comprising:

computer memory; and

a plurality of executable instructions stored in the computer memory, the instructions, when executed, instructing the processor to compute a low-resolution image from the training image;

compute a plurality of representative vectors from the low-resolution image; and

compute a set of interpolation filters for each of the representative vectors;

the processor computing the representative vectors by computing a number of cluster vectors from the low-resolution image, and using the cluster vectors to compute the representative vectors;

the processor computing the interpolation filters from low-resolution observation vectors, the cluster vectors, the representative vectors and vectors from a high-resolution image corresponding to the low-resolution image.

35. The article of claim 34, wherein the instructions further instruct the processor to compute a sharpened high-resolution image from the training image, the sharpened high-resolution file being used to compute the interpolation filters.

36. The article of claim 34, wherein the instructions instruct the processor to compute a classifier including the representative vectors by initializing the classifier and updating the classifier until optimal values for the classifier have been obtained.

37. The article of claim 34, wherein the instructions instruct the processor to compute each cluster vector by forming an observation window about a sampled pixel in the low-resolution image, extracting a vector including neighboring pixels of the sampled pixel, subtracting a value of the sampled pixel from values of the pixels in the vector;

and scaling the vector.

38. The article of claim 34, wherein the instructions instruct the processor to compute coefficients for the interpolation filters by:

computing a number of filter design triplets from pixels in the low-resolution image, each filter design triplet corresponding to a sampled pixel in the low-resolution image, each filter design triplet including an observation vector for the sampled pixel, a cluster vector for the sampled pixel, and a vector of high resolution pixels from a high-resolution image, the high resolution pixels corresponding to the sampled pixel;

computing training statistics from the filter design triplets; and

computing the coefficients from the training statistics.

39. The article of claim 34, wherein the representative vectors are generated by using a parameter optimization technique.