Computing a higher resolution image from multiple lower resolution images using model-base, robust Bayesian estimation

• US 7,809,155 B2
• Filed: 06/29/2006
• Issued: 10/05/2010
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Nestares, Oscar, Ettinger, Scott M., Mulgaonkar, Prasanna G., Haussecker, Horst W., Dierks, Michael P.
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/479,999
• Publication Number:
  US 20070019887A1
• Time in Patent Office: 1,559 Days
• Field of Search: 382/100, 382/269, 382/276, 382/284, 382/299, 382/148, 382/260, 382/275, 382/294, 358/1.2, 358/3.01, 358/3.07, 358/3.26, 358/3.7, 358/463, 375/340.21, 375/240.27, 706/20, 706/934, 707/100, 707/101, 348/14.14, 348/207.1, 348/211.6, 348/14.12, 348/208.13, 348/333.11, 370/414, 370/416, 370/418, 370/455
• US Class Current: 382/100
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

A result higher resolution (HR) image of a scene given multiple, observed lower resolution (LR) images of the scene is computed using a Bayesian estimation image reconstruction methodology. The methodology yields the result HR image based on a Likelihood probability function that implements a model for the formation of LR images in the presence of noise. This noise is modeled by a probabilistic, non-Gaussian, robust function. The image reconstruction methodology may be used to enhance the image quality of images or video captured using a low resolution image capture device. Other embodiments are also described and claimed.

Claims

1. A method comprising:

acquiring an image sequence;

computing with an image processing component (1) a likelihood gradient using a probabilistic, non-Gaussian, robust function and (2) a prior gradient; and

using the likelihood gradient and the prior gradient to update a high resolution image.

2. The method of claim 1, wherein the image sequence is a sequence of infrared images.

3. The method of claim 1, wherein the image sequence is a sequence of images obtained from a camera having a CMOS sensor.

4. The method of claim 1, wherein the image sequence is a sequence of images obtained from a camera having a vibrating sensor.

5. The method of claim 1, wherein the image sequence is a sequence of images obtained from a scanning imaging device.

6. The method of claim 1, wherein the image sequence is a sequence of images obtained from a medical imaging device.

7. The method of claim 1, wherein the image sequence is a sequence of images obtained from a surveillance camera.

8. The method of claim 7, further comprising selecting a desired portion of the image sequence before computing the likelihood gradient.

9. The method of claim 8, wherein the desired portion is a portion containing a face.

10. The method of claim 8, wherein the desired portion is a portion containing text.

11. A method comprising:

performing with a video processing component a super resolution operation on low resolution video to create high resolution video using a probabilistic, non-Gaussian, robust function; and

broadcasting the high resolution video.

12. The method of claim 11, wherein performing the super resolution operation occurs online at the time of broadcasting.

13. The method of claim 11, further comprising storing the high resolution video prior to broadcasting the high resolution video.

14. The method of claim 13, wherein the low resolution video comprises National Television Standards Committee (NTSC) video.

15. The method of claim 13, wherein the high resolution video comprises High Definition Television (HDTV) video.

16. The method of claim 11, wherein performing a super resolution operation on low resolution video to create high resolution video comprises measuring and storing motion vectors offline and subsequently using the motion vectors to create high resolution video online at the time of broadcasting.

17. The method of claim 11, further comprising receiving low resolution video from a user before performing a super resolution operation on the low resolution video, wherein broadcasting the high resolution video comprises sending the high resolution video to the user.

18. An apparatus comprising:

a first interface to receive low resolution video content;

an image processing component coupled to the first interface, wherein the image processing component is dedicated to running a super resolution algorithm to convert the low resolution video content into high resolution video content using a probablistic, non-Gaussian, robust function; and

a second interface coupled to the image processing component to output the high resolution video content.

19. The apparatus of claim 18, wherein the first interface is a satellite television receiver.

20. The apparatus of claim 18, wherein the first interface is a cable television receiver.

21. The apparatus of claim 18, wherein the high resolution video content is output to a display.

22. The apparatus of claim 21, wherein the display is a television.

23. The apparatus of claim 18, wherein the second interface is a network interface.

24. The apparatus of claim 18, wherein the second interface is an interface to a hard disk drive.

25. The apparatus of claim 18, wherein the image processing component includes at least one microprocessor.

26. The apparatus of claim 25, wherein the image processing component further includes memory coupled to the microprocessor.

27. A method comprising:

acquiring a plurality of one dimensional signals, wherein each of the plurality of signals is phase shifted from another;

computing with a signal processing component (i) a likelihood gradient using the phase shifted signals and a probablistic, non-Gaussian, robust function, and (ii) a prior gradient; and

using the likelihood gradient and the prior gradient to update a high resolution one dimensional signal.

28. The method of claim 27, wherein the one dimensional signals are audio signals acquired using an array of microphones.

29. The method of claim 28, wherein the high resolution one dimensional signal is a high resolution audio signal.

30. The method of claim 27, wherein the one dimensional signals are radio signals acquired using an array of antennas.

31. The method of claim 30, wherein the high resolution one dimensional signal is a high resolution radio signal.