Generating super resolution digital images

• US 20030002707A1
• Filed: 06/29/2001
• Published: 01/02/2003
• Est. Priority Date: 06/29/2001
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Reed, Alastair M., Hannigan, Brett T.
• Granted Patent: US 7,218,751 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/4015 Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns

Abstract

Slight camera movement between when successive images are captured is advantageously utilized to minimize or eliminate the need to interpolate in order to fill in the “holes” in a Bayer pattern. The captured color values from multiple appropriately positioned images are used to fill in the “holes” in a Bayer pattern. Fore example, instead of interpolating the value of red for the second pixel position on the first row of a Bayer pattern, an image is selected which is positioned one pixel to the right of the first image, and the red vales from this image are used for the red values of the second pixel on the first line. The value of the pixels in multiple images which are appropriately aligned to each pixel position are averaged to generate a better value for each pixel position. Information carried by a digital watermark (either alone or together with other techniques) is used to determine the alignment of the images. Images are selected which are positioned so that corresponding pixels fall within a specified tolerance from a location in a Bayer pattern. The pixel values of the images which fall within the specified tolerance of each pixel position in a Bayer pattern are selected and used for the alignment.

Claims

1. ) the method of generating a high resolution image from a series of nearly identical low resolution images, each of which contain a reference signal by reading said reference signal from each of said low resolution images to determine the alignment of the pixels in the image,

selecting the images whose pixels are within a specified tolerance from specified positions,

combining the selected images to generate a high resolution image.

2. ) The method recited in claim 1 wherein the alignment is determined relative to the positions in a Bayer square.

3. ) The method in claim 1 wherein the pixels in said low resolution images have colors in accordance with a Bayer square.

4. ) The method recited in claim 1 wherein said reference signal is a watermark signal.

5. ) The method recited in claim 1 wherein said reference signal is a watermark grid signal.

6. ) A method of generating a high resolution images from a plurality of low resolution images whose pixel values are in a Bayer square,

determining which of said low resolution images align with each pixel hole position of a Bayer square,

combining said multiple low resolution images to fill in the holes in a Bayer square.

7. ) The method in claim 6 wherein the pixels values in multiple images which are aligned with each pixel position are averaged and the averaged images a combined to fill the holes in a Bayer square.

8. ) A method combining a generating a high resolution image from a plurality of low resolution images comprising,

producing a physical image which includes a hidden reference signal,

capturing a plurality of low resolution electronic images of said physical image,

using said reference signal to align a plurality of said low resolution images,

combining said aligned low resolution images into a high resolution image.

9. ) The method recited in claim 8 wherein said low resolution images are aligned in accordance with the holes in a Bayer square.

10. ) The method recited in claim 8 wherein a plurality of low resolution images are captured and only those low resolution images which align to within a specified tolerance with the holes in a Bayer square are used to form said composite image.

11. ) A system for generating a high resolution image from a series of nearly identical relatively low resolution images including

a watermark reading program for reading a watermark grid signal from each of said low resolution images to determine the alignment of the pixels in the low resolution images relative to the positions in a Bayer square,

an image selection program for selecting the low resolution images whose pixels are within a specified tolerance from each position in the Bayer square,

an image combination program for combining the selected low resolution images to generate a high resolution image.

14. ) The method recited in claim 13 wherein each of said low resolution images are combined to fill holes in a Bayer square.

15. ) The method recited in claim 13 wherein a plurality of low resolution images are captured and only a selected number of said low resolution images are used to form said high resolution image.

16. ) The method recited in claim 13 wherein said reference signal is a watermark grid signal.

18. ) The method recited in claim 17 wherein said reference signal is a watermark signal.

19. ) The method recited in claim 17 wherein said reference signal is a watermark grid signal.

12. ) A system for generating a high resolution images from a plurality of relatively low resolution images whose pixel values are in a Bayer square configuration, means determining which of said images align with each pixel position of a Bayer square to within a specified tolerance,

means combining multiple aligned low resolution images to fill in the holes in a Bayer square.

13. ) A method combining a generating a high resolution image from a plurality of relatively low resolution images comprising,

producing a physical image which includes a hidden reference signal,

capturing a plurality of low resolution electronic images of said physical image,

using said reference signal to align a plurality of said images,

combining said aligned images into a high resolution image.

17. ) The method of generating a high resolution image from a plurality of low resolution images comprising the steps of,

capturing a series of low resolution images, each of which contain a reference signal,

reading said reference signal from each of said low resolution images,

aligning said low resolution images in accordance the location of said reference signal,

combining said aligned low resolution images into a high resolution image.

20. ) The method of aligning multiple low resolution images to form a high resolution image wherein both a hidden reference signal embedded in the low resolution images and visible image content are used to align said images.