Method and apparatus for synthesis of higher resolution images

• US 10,043,242 B2
• Filed: 07/30/2014
• Issued: 08/07/2018
• Est. Priority Date: 07/31/2013
• Status: Active Grant

• Current Assignee: MBDA UK Ltd. (MBDA SAS)
• Original Assignee: MBDA UK Ltd. (MBDA SAS)
• Inventors: Matson, Gary Paul, Sherriff, Andrew John, Middleton, Robert James
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US14/908,625
• Publication Number:
  US 20160171658A1
• Time in Patent Office: 1,469 Days
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 3/4069 by subpixel displacements
• CPC Class Codes: G06T 3/4076 using the original low-resolution images to iteratively correct the high-resolution images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/11 Region-based segmentation

Abstract

An image-processing method includes obtaining an image including a target object, the image being formed by an array of pixels. A current frame portion is extracted from the image, the frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object. A previously calculated current super-resolved frame portion is provided, corresponding to the region of interest in the image. An updated super-resolved frame portion is calculated from the current frame portion and the current super-resolved frame portion.

Claims

1. An image-processing method comprising the steps of:

(i) obtaining a stream of images including a target object, each image of the stream of images being formed by an array of pixels;

(ii) extracting a current frame portion from an image from the stream of images, the current frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object;

(iii) providing a previously calculated super-resolved frame portion, which has been calculated in a previous iteration of steps (ii) to (iv) of the method, corresponding to the region of interest in the image;

(iv) calculating an updated super-resolved frame portion from the current frame portion and the previously calculated super-resolved frame portion, both the updated super-resolved frame portion and the previously calculated super-resolved frame portion being weighted with respective weightings in the calculation, the weightings being applied on a frame portion basis such that for each frame portion the same weighting is applied to all pixels in the frame portion; and

(v) repeating for a plurality of times steps (ii) to (iv) of the method for successive images, wherein the respective weightings used in the calculation of each of the successive updated super-resolved frame portions are such that the effective contribution made to the super-resolved frame portion by a given frame portion progressively decays with each successive calculated super-resolved frame portion.

2. A method as claimed in claim 1, in which the previously calculated super-resolved frame portion is decayed by giving it a higher weighting in the calculation of the updated super-resolved frame portion than a weighting given to the current frame portion.

3. A method as claimed in claim 1, in which the frame portion extracted in each iteration of step (ii) after the first iteration corresponds to a region of interest identified in the image used in the first iteration of step (ii).

4. A method as claimed in claim 1, in which the region of interest is assumed to be in the current image at the position of the region of interest in a previous image.

5. A method as claimed in claim 1, in which the target is tracked between images.

6. A method as claimed in claim 1, in which a translation shift between the current frame portion and the previously calculated super-resolved frame portion is calculated by a correlation in the spatial domain, phase correlation in the frequency domain, or by an iterative method in which correlation in the spatial domain or phase correlation in the frequency domain is used as an initial guess to constructed an estimated super resolved frame portion and registration is iterated to produce the optimal super resolved frame portion.

7. A method as claimed in claim 1, in which the position of the target in the image is determined using cross-correlation with a de-resolved version of a super resolution frame portion calculated in a previous iteration of the method.

8. A method as claimed in claim 1, in which a working frame portion is extracted from the image and used to identify the location of the region of interest.

9. A method as claimed in claim 1, in which an image deblurring step is performed.

10. A method as claimed in claim 9, in which deblurring of the summed image is carried out using equations formulated within the image domain.

11. A method as claimed in claim 1, in which the super resolution algorithm is carried out only on the pixels of the current frame portion that overlap with the pixels of the previously calculated current super-resolved frame portion, or on all pixels of both the current frame portion and the previously calculated super-resolved frame portion, or using only the pixels corresponding to the pixels of the frame portion of the first pass of the method.

12. A method as claimed in claim 1, wherein the region of interest comprises a portion of the pixels less than the number of pixels forming the image.

13. An image-processing apparatus comprising:

(i) an imager for obtaining a stream of images including a target object, each image of the stream of images being formed by an array of pixels;

(ii) an image processor including at least one microprocessor configured to

a. extract a current frame portion from an image from the stream of images, the current frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object;

b. obtain a previously calculated super-resolved frame portion, which has been calculated in a previous iteration of steps (a) to (d), corresponding to the region of interest in the image;

c. apply a weighting to the current frame portion and the previously calculated super-resolved frame portion, relative to each other in a predetermined manner, the weightings being applied on a frame portion basis such that for each frame portion the same weighting is applied to all pixels in the frame portion;

d. calculate an updated super-resolved frame portion from the weighted current frame portion and the previously calculated super-resolved frame portion; and

e. repeat for a plurality of times steps a to d for successive images, the respective weightings used in the calculation of each of the successive updated super-resolved frame portions being such that the effective contribution made to the super-resolved frame portion by a given frame portion progressively decays with each successive calculated super-resolved frame portion.

14. A missile seeker including an image processing apparatus according to claim 13.

15. A computer program product embodied on non-transitory, computer-readable medium and configured to cause, when the computer program is executed, data-processing apparatus to:

(i) receive a stream of images including a target object, each image of the stream of images being formed by an array of pixels;

(ii) extract a current frame portion from an image from the stream of images, the current frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object;

(iii) provide a previously calculated super-resolved frame portion, which has been calculated in a previous iteration of steps (ii) to (v), corresponding to the region of interest in the image;

(iv) apply a weighting to the current frame portion and the previously calculated super-resolved frame portion, relative to each other in a predetermined manner, the weightings being applied on a frame portion basis such that for each frame portion the same weighting is applied to all pixels in the frame portion;

(v) calculate an updated super-resolved frame portion from the weighted current frame portion and the previously calculated super-resolved frame portion; and

(vi) repeat for a plurality of times steps (ii) to (v) for successive images, the respective weightings used in the calculation of each of the successive updated super-resolved frame portions being such that the effective contribution made to the super-resolved frame portion by a given frame portion progressively decays with each successive calculated super-resolved frame portion.