Upsampling and signal enhancement

• US 9,830,682 B2
• Filed: 10/26/2015
• Issued: 11/28/2017
• Est. Priority Date: 04/26/2013
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Lakshman, Haricharan, Schwarz, Heiko, Marpe, Detlev, Wiegand, Thomas, Kutyniok, Gitta, Wang-Q, Lim
• Primary Examiner(s): Lee, Jonathan S
• Application Number:
  US14/922,797
• Publication Number:
  US 20160048947A1
• Time in Patent Office: 764 Days
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 3/4076 using the original low-resolution images to iteratively correct the high-resolution images

Abstract

A signal which is to be quality-improved often suffers from the quality degradation in the spatial high frequency region more than compared to the spatial low frequency region. Accordingly a quality improvement is performed efficiently by combining the signal to be quality improved with a high frequency portion extracted from a sparse approximation of the signal to be quality improved.

Claims

1. An apparatus for upsampling a first signal comprising a first resolution, the apparatus comprising a processor or electronic circuit configured to

apply an upsampling process onto the first signal so as to acquire a second signal comprising a second resolution higher than the first resolution;

generate, using sparse approximation, an approximated signal approximating the second signal and comprising the second resolution;

extract a high frequency portion from the approximated signal

using a high pass filter or

by estimating a low frequency portion of the approximated signal and subtracting the low frequency portion from the approximated signal; and

combine the second signal and the extracted high frequency portion.

2. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to perform the upsampling process using a FIR filter, II R filter, or a combination of both.

3. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to generate the approximated signal by applying a sparse approximation to the second signal.

4. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to perform the sparse approximation by sequentially performing a forward transformation to acquire transform coefficients, thresholding the transform coefficients to acquire thresholded transform coefficients and performing an inverse transform onto the thresholded transform coefficients.

5. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to perform the sparse approximation by optimizing a cost function.

6. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to perform the sparse approximation by successively building-up the approximated signal by sequentially optimizing coefficients of a parameterized function.

7. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to estimate the low frequency portion by downsampling and then upsampling the approximated signal.

8. The apparatus according to claim 7, wherein the processor or electronic circuit is configured to, in upsampling the downsampled approximated signal, use the upsampling process of the upsampler.

9. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to, in combining the second signal and the high frequency portion of the approximated signal, add same to one another.

10. The apparatus according to claim 1, when the processor or electronic circuit is configured to perform the generation, the extraction and the combination iteratively in iterations with the sparse approximation being applied onto a combined signal acquired by the combination.

11. The apparatus according to claim 1, wherein the first signal, the second signal and the approximated signal are two-dimensional signals.

12. The apparatus according to claim 1, wherein the processor or electronic circuit is configured to upsample a two-dimensional signal by using a series of one-dimensional row-wise and column-wise upsampling steps.

13. An apparatus for enhancing an input signal, the apparatus comprising a processor or electronic circuit configured to

generate, using sparse approximation, an approximated signal approximating the input signal;

extract a high frequency portion from the approximated signal

using a high pass filter or

estimating a low frequency portion of the approximated signal and subtracting the low frequency portion from the approximated signal; and

combine the input signal and the extracted high frequency portion extracted from the approximated signal.

14. The apparatus according to claim 13, wherein the processor or electronic circuit is configured to generate the approximated signal by applying a sparse approximation to the input signal.

15. The apparatus according to claim 13, wherein the processor or electronic circuit is configured to perform the sparse approximation by sequentially performing a forward transformation to acquire transform coefficients, thresholding the transform coefficients to acquire thresholded transform coefficients and performing an inverse transform onto the thresholded transform coefficients.

16. The apparatus according to claim 13, wherein the processor or electronic circuit is configured to perform the sparse approximation by optimizing a cost function.

17. The apparatus according to claim 13, wherein the processor or electronic circuit is configured to perform the sparse approximation by successively building-up the approximated signal by sequentially optimizing coefficients of a parameterized function.

18. The apparatus according to claim 13, wherein the processor or electronic circuit is configured to estimate the low frequency portion by downsampling and then upsampling the approximated signal.

19. The apparatus according to claim 13, wherein the processor or electronic circuit is configured to, in combining the input signal and the high frequency portion of the approximated signal, add same to one another.

20. The apparatus according to claim 13, when the processor or electronic circuit is configured to perform the generation, the extraction and the combination iteratively in iterations with the sparse approximation applied onto a combined signal acquired by the combination.

21. The apparatus according to claim 13, wherein the input signal and the approximated signal are two-dimensional signals.

22. The apparatus according to claim 13, wherein the processor or electronic circuit is configured to enhance a two-dimensional signal by using a series of one-dimensional row-wise and column-wise upsampling steps.

23. A method for upsampling a first signal comprising a first resolution, comprising

applying an upsampling process onto the first signal so as to acquire a second signal comprising a second resolution higher than the first resolution;

generating, using sparse approximation, an approximated signal approximating the second signal and comprising the second resolution;

extracting a high frequency portion from the approximated signal using a high pass filter or by estimating a low frequency portion of the approximated signal and subtracting the low frequency portion from the approximated signal; and

combining the second signal and the extracted high frequency portion.

25. A non-transitory computer readable medium storing a computer program or computer program product for performing, when executed on a computer, a method according to claim 23.

24. A method for enhancing an input signal, comprising

generating, using sparse approximation, an approximated signal approximating the input signal;

extracting a high frequency portion from the approximated signal using a high pass filter or by estimating a low frequency portion of the approximated signal and subtracting the low frequency portion from the approximated signal; and

combining the input signal and the extracted high frequency portion extracted from the approximated signal.

26. A non-transitory computer readable medium storing a computer program or computer program product for performing, when executed on a computer, a method according to claim 24.