Resolution enhancement for images stored in a database

• US 20030108252A1
• Filed: 12/11/2001
• Published: 06/12/2003
• Est. Priority Date: 12/11/2001
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Carrig, James J.
• Granted Patent: US 7,123,780 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

A method and apparatus for the enhancement of images stored in a database are disclosed. A first image at a first resolution and a second image at second resolution are used to generate an enhanced image at a third resolution.

Claims

1. A method for image enhancement comprising:

receiving an input image;

matching regions of the input image to other available data;

forming a combined image containing some pixels spaced more closely than the input image, and

generating an output image at a resolution finer than the input image resolution.

2. The method according to claim 1, wherein the output image resolution is greater than the input image resolution and less than or equal to the resolution of the combined image.

3. The method of claim 1 wherein generating an output image at a resolution finer than the input image resolution further comprises applying a least squares filter to generate each output pixel.

4. The method of claim 1 wherein generating an output image at a resolution finer than the input image resolution further comprises operating on at least one of input image pixels and combined image pixels to generate each output image pixel at a resolution finer than the input image resolution.

5. The method according to claim 4, wherein operating further comprises applying a filter to generate each output pixel.

6. The method according to claim 5, wherein the filter is an optimal least squares filter for each output pixel.

7. The method of claim 6 wherein the optimal least squares filter for each output pixel is based on an irregular sample grid.

8. The method of claim 1 wherein other available data may change over time.

9. The method of claim 1 wherein the image and other available data are video images in a home networking database.

10. The method of claim 1 further comprising the transfer of a payment before the output image is viewed by a user.

11. A processing system comprising a processor, which when executing a set of instructions performs the method of claim 1.

12. A machine-readable medium having stored thereon instructions, which when executed performs the method of claim 1.

13. The machine-readable medium of claim 12 wherein the input image is retrieved from and the output image is stored to a home networked database.

14. An apparatus for image enhancement comprising:

means for receiving an input image;

means for matching regions of the input image to other available data;

means for forming a combined image containing some pixels spaced more closely than the input image, and

means for generating an output image at a resolution finer than the input image resolution.

15. The apparatus according to claim 14, wherein the output image resolution is greater than the input image resolution and less than or equal to the resolution of the combined image.

16. The apparatus of claim 14 wherein means for generating an output image at a resolution finer than the input image resolution further comprises applying a filter to generate each output pixel.

17. The apparatus of claim 14 wherein means for generating an output image at a resolution finer than the input image resolution further comprises operating on at least

one of input image pixels and combined image pixels to generate each output image pixel at a resolution finer than the input image resolution.

18. The apparatus according to claim 17, wherein means for operating further comprises solving a least squares problem to generate each output pixel.

19. The apparatus according to claim 18, wherein the solution to the least squares problem is an optimal least squares filter for each output pixel.

20. The apparatus of claim 19 wherein the optimal least squares filter for each output pixel is based on an irregular sample grid.

21. The apparatus of claim 14 wherein other available data may change over time.

22. The apparatus of claim 14 wherein the image and other available data are video images in a home networking database.

23. A system comprising a processor, which when executing a set of instructions, performs the following:

retrieves a first video image at a first resolution;

forms a second video image at a second resolution; and

generates a third video image at a third resolution.

24. The system of claim 23 wherein the third resolution is greater than the first resolution, and less than or equal to the second resolution.

25. The system of claim 23 wherein the images may have missing pixels.

26. The system of claim 23 wherein the second and third video image may change over time.

27. The system of claim 23 wherein generating a third video image at a third resolution further comprises applying an optimal least squares filter for each output pixel.

28. The system of claim 23 wherein the video images are located on a home networking database.

29. An apparatus comprising:

means for receiving a input image having pixels at a first resolution;

means for receiving other available data having pixels at a second resolution;

means for forming a combined image containing some pixels spaced more closely than the input image, and;

means for generating an output image at a resolution finer than the input image resolution by applying a filter to the combined image pixels.

30. The apparatus of claim 29 wherein the filter is a least squares filter.

31. The apparatus of claim 30 wherein the least squares filter is optimal for each output image pixel.

32. The apparatus of claim 29 wherein applying a filter to the combined image pixels is means for applying the filter by a numerical tap method.

33. The apparatus of claim 29 where the means for forming a combined image is means for motion compensation.

34. A machine-readable medium having stored thereon information representing the apparatus of claim 29.

35. An apparatus for image enhancement comprising:

a first device having an input and an output, the input coupled to receive a first image to be enhanced;

a second device having an input an output, the input coupled to receive a second image;

a third device having a first input, a second input, and an output, the first input coupled to receive the first device output, and the second input coupled to receive the second device output; and

a fourth device having an input and an output, the input coupled to receive the third device output and the fourth device output coupled to send a third enhanced image.

36. The apparatus of claim 35 wherein the first device input and second device input are coupled to a home network.

37. The apparatus of claim 35 wherein the fourth device output is coupled to a home network.

38. The apparatus of claim 35 wherein the third device further comprises a least squares filtering device having an input and an output, the input coupled to receive an image, the output coupled to send a filtered image.