Method and apparatus for synthesis of higher resolution images

• US 9,792,669 B2
• Filed: 07/30/2014
• Issued: 10/17/2017
• Est. Priority Date: 07/31/2013
• Status: Active Grant

• Current Assignee: MBDA UK Ltd. (MBDA SAS)
• Original Assignee: MBDA UK Ltd. (MBDA SAS)
• Inventors: Matson, Gary Paul, Sherriff, Andrew John, Middleton, Robert James
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US14/908,624
• Publication Number:
  US 20160171657A1
• Time in Patent Office: 1,175 Days
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 3/4069 by subpixel displacements
• CPC Class Codes: G06T 3/4076 using the original low-resolution images to iteratively correct the high-resolution images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

In an image-processing method, a stack is provided for storing a predetermined number of frame portions. An image including a target object is obtained, the image being formed by an array of pixels. A frame portion is extracted from the image, the frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object. The frame portion is stored in the stack, the storing including discarding an oldest previously stored frame portion from the stack if the number of frame portions stored in the stack has reached the predetermined number. The steps of the method are repeated a plurality of times. Frame portions in the stack having a phase substantially equal to a given phase are averaged. A super-resolved image is calculated from the plurality of stored frame portions.

Claims

1. An image-processing method comprising the steps of:

(i) obtaining an image including a target object, the image being formed by an array of pixels;

(ii) extracting a frame portion from the image, the frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object and the frame portion having an associated phase, representative of a sub-pixel translation shift of the region of interest defined by the frame portion relative to the region of interest defined by another frame portion;

(iii) storing the frame portion in the stack, the storing including discarding an oldest previously stored frame portion from the stack if the number of frame portions stored in the stack has reached the predetermined number;

(iv) repeating steps (i) to (iii) a plurality of times; and

(v) calculating a super-resolved image from a plurality of phase averages, wherein each phase average is the average of frame portions in the stack having the same phase, wherein the frame portions of each phase average have an associated phase representative of a sub-pixel translation shift of the regions of interest defined by those frame portions relative to the region of interest defined by another frame portion.

2. A method as claimed in claim 1, in which the target object is identified in the first image by a user or a target recognition algorithm.

3. A method as claimed in claim 1, in which the region of interest is defined by a user or by an algorithm in the first image and then calculated in subsequent images.

4. A method as claimed in claim 1, in which a shift in the position of the target in successive images is calculated, and the integer part of the calculated shift is used to shift the region of interest in the later image relative to the position of the region of interest in the earlier image.

5. A method as claimed in claim 1, in which a working frame portion is extracted from each subsequent image.

6. A method as claimed in claim 1, in which the translation shift between a pair of images is calculated by a correlation.

7. A method as claimed in claim 1, in which a translation shift between a first pair of frame portions is calculated and the resulting calculated translation shift is used in calculating a first super-resolved image derived from a first set of frame portions including said first pair of frame portions, and then subsequently a second super-resolved image is calculated from a second set of frame portions, different from the first set, but still including said first pair of frame portions, wherein the second super-resolved image is calculated using the previously calculated translation shift.

8. A method as claimed in claim 1, in which the calculation of the super-resolved image includes updating a super-resolved image calculated in a previous iteration by changing in the calculation only the phases which have changed in a new frame.

9. A method as claimed in claim 1, in which the calculation of a super-resolved image from the plurality of stored frame portions includes updating a super-resolved image calculated in a previous iteration by removing the oldest frame from its corresponding phase average, adding the new frame to its corresponding phase average, and updating the phase average over phases with the two modified phases.

10. A method as claimed in claim 1, in which the calculation of the super-resolved image includes a deblurring step.

11. A method as claimed in claim 1, in which the calculation of the super-resolved image is bypassed on each iteration until a predefined minimum number of frame portions have been stored on the stack.

12. An image-processing apparatus comprising:

(i) an imager for obtaining an image including a target object, the image being formed by an array of pixels;

(ii) a stack of for storing a predetermined number of frame portions;

(iii) an image processor configured to

a. extract a frame portion from the image, the frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object, the frame portion having an associated phase, representative of a sub-pixel translation shift of the region of interest defined by the frame portion relative to the region of interest defined by another frame portion;

b. store the frame portion in the stack, the storing including discarding an oldest previously stored frame portion from the stack if the number of frame portions stored in the stack has reached the predetermined number; and

c. calculating a super-resolved image from a plurality of phase averages, wherein each phase average is the average of frame portions in the stack having the same phase, wherein the frame portions of each phase average have an associated phase representative of a sub-pixel translation shift of the regions of interest defined by those frame portions relative to the region of interest defined by another frame portion.

13. A missile seeker including an image processing apparatus according to claim 12.

14. A computer program product embodied on non-transitory, computer-readable medium and configured to cause, when the computer program is executed, data-processing apparatus to:

(i) receive an image including a target object, the image being formed by an array of pixels;

(ii) extract a frame portion from the image, the frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object, the frame portion having an associated phase, representative of a sub-pixel translation shift of the region of interest defined by the frame portion relative to the region of interest defined by another frame portion;

(iii) store the frame portion in a stack for storing a predetermined number of frame portions, the storing including discarding an oldest previously stored frame portion from the stack if the number of frame portions stored in the stack has reached the predetermined number;

(iv) repeat steps (i) to (iii) a plurality of times; and

(v) calculating a super-resolved image from a plurality of phase averages, wherein each phase average is the average of frame portions in the stack having the same phase, wherein the frame portions of each phase average have an associated phase representative of a sub-pixel translation shift of the regions of interest defined by those frame portions relative to the region of interest defined by another frame portion.