Method and apparatus for super-resolution imaging using digital imaging devices

• US 8,553,093 B2
• Filed: 09/30/2008
• Issued: 10/08/2013
• Est. Priority Date: 09/30/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Wong, Earl Quong, Murakami, Yoshihiro, Li, Pingshan, Edwards, Eric D., Triteyaprasert, Soroj, Kushida, Hidenori, Nakamura, Makibi, Sugiura, Mamoru
• Primary Examiner(s): Ho, Tuan
• Assistant Examiner(s): Garces-Rivera, Angel L
• Application Number:
  US12/242,809
• Publication Number:
  US 20100079608A1
• Time in Patent Office: 1,834 Days
• Field of Search: 348/208.13
• US Class Current: 348/208.13
• CPC Class Codes: G06T 3/4069 by subpixel displacements
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A super-resolution image is generated from a sequence of low resolution images. In one embodiment, the image shift information is measured for each of the low resolution images using an image stabilization component of an imaging device. The shift information is used to generate the super-resolution image. In another embodiment, the blurs are calculated for each of the low resolution images and are used to generate the super-resolution image.

Claims

1. A computerized method comprising:

acquiring a plurality of low resolution images, wherein the plurality of acquired low resolution images is acquired at different focus positions;

calculating a blur for each of the plurality of acquired low resolution images using a blur difference between different ones of the plurality of acquired low resolution images; and

generating a super-resolution image from the plurality of acquired low resolution images and the calculated blurs.

2. The computerized method of claim 1, further comprising:

calculating a blur difference between a pair of the plurality of acquired low resolution images.

3. The computerized method of claim 2, wherein the calculating the blur difference further comprising convolving one image of the image pair with a blur kernel.

4. The computerized method of claim 2, further comprising:

re-registering one image of the pair so the image pair has the same shift position.

5. The computerized method of claim 1, wherein the generating the super-resolution image comprises:

generating an initial guess of the super-resolution image by interpolating each of the plurality of low resolution images onto a finer grid.

6. The computerized method of claim 1, wherein the generating the super-resolution image comprises:

iteratively computing the super-resolution image by,

computing a plurality of approximation low resolution images using a previous candidate super-resolution image,

computing a plurality of error images based on the difference between the plurality of approximation low resolution images and the plurality of low resolution images, and

computing a new candidate super-resolution from the plurality of error images and the previous candidate super-resolution image.

7. The computerized method of claim 6, wherein if one of more of the plurality of error images are less than a threshold, the iteratively computing of the super-resolution image has converged.

8. A non-transitory machine readable medium having executable instructions to cause a processor to perform a method comprising:

acquiring a plurality of low resolution images, wherein the plurality of acquired low resolution images is acquired at different focus positions;

calculating a blur for each of the plurality of acquired low resolution images using a blur difference between different ones of the plurality of acquired low resolution images; and

generating a super-resolution image from the plurality of acquired low resolution images and the calculated blurs.

9. The non-transitory machine readable medium of claim 8, further comprising:

calculating a blur difference between a pair of the plurality of acquired low resolution images.

10. The non-transitory machine readable medium of claim 9, wherein the calculating the blur difference further comprising convolving one image of the image pair with a blur kernel.

11. The non-transitory machine readable medium of claim 8, further comprising:

re-registering one image of the pair so the image pair has the same shift position.

12. The non-transitory machine readable medium of claim 8, wherein the generating the super-resolution image comprises:

generating an initial guess of the super-resolution image by interpolating each of the plurality of low resolution images onto a finer grid.

13. The non-transitory machine readable medium of claim 8, wherein the generating the super-resolution image comprises:

iteratively computing the super-resolution image by,

computing a plurality of approximation low resolution images using a previous candidate super-resolution image,

computing a plurality of error images based on the difference between the plurality of approximation low resolution images and the plurality of low resolution images, and

computing a new candidate super-resolution from the plurality of error images and the previous candidate super-resolution image.

14. The non-transitory machine readable medium of claim 13, wherein if one of more of the plurality of error images are less than a threshold, the iteratively computing of the super-resolution image has converged.

15. A system comprising:

a processor;

a memory coupled to the processor though a bus; and

a process executed from the memory by the processor to cause the processor to acquire a plurality of low resolution images, wherein the plurality of acquired low resolution images is acquired at different focus positions, calculate a blur for each of the plurality of acquired low resolution images using a blur difference between different ones of the acquired plurality of low resolution images and to generate a super-resolution image from the acquired plurality of low resolution images and the calculated blurs.

16. The system of claim 15, wherein the process further causes the processor to calculate a blur difference between a pair of the plurality of acquired low resolution images.

17. The system of claim 16, wherein a process to calculate of the blur difference causes the processor to convolve one image of the image pair with a blur kernel.

18. The system of claim 16, wherein the process further causes the processor to re-registering one image of the pair so the image pair has the same shift position.

19. The system of claim 15, wherein the process further causes the processor to generate an initial guess of the super-resolution image by interpolating each of the plurality of low resolution images onto a finer grid.

20. The system of claim 19, wherein the process further causes the processor to iteratively compute the super-resolution image, wherein a process to iteratively compute the super-resolution image causes the processor to compute a plurality of approximation low resolution images using a previous candidate super-resolution image, to compute a plurality of error images based on the difference between the plurality of approximation low resolution images and the plurality of low resolution images, and to compute a new candidate super-resolution from the plurality of error images and the previous candidate super-resolution image.