Image data processing techniques for highly undersampled images

• US 20080212895A1
• Filed: 01/09/2008
• Published: 09/04/2008
• Est. Priority Date: 01/09/2007
• Status: Abandoned Application

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Mattox, Barry G.
• Application Number:
  US12/007,358
• Publication Number:
  US 20080212895A1
• Time in Patent Office: Days
• US Class Current: 382/294
• CPC Class Codes: H04N 2209/046 Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values

Abstract

An exemplary method for processing undersampled image data includes: aligning an undersampled frame comprising image data to a reference frame; accumulating pixel values for pixel locations in the aligned undersampled frame; repeating the aligning and the accumulating for a plurality of undersampled frames; assigning the pixel values accumulated for the pixel locations in the aligned undersampled frames to closest corresponding pixel locations in an upsampled reference frame; and populating the upsampled frame with a combination of the assigned pixel values to produce a resulting frame of image data.

Claims

1. A method for processing undersampled image data, comprising:

aligning an undersampled frame comprising image data to a reference frame;

accumulating pixel values for pixel locations in the aligned undersampled frame;

repeating the aligning and the accumulating for a plurality of undersampled frames;

assigning the pixel values accumulated for the pixel locations in the aligned undersampled frames to closest corresponding pixel locations in an upsampled reference frame; and

populating the upsampled frame with a combination of the assigned pixel values to produce a resulting frame of image data.

2. The method of claim 1, wherein the image data includes dithered image data.

3. The method of claim 1, wherein, if more than one pixel value is assigned to a given upsampled pixel location, the populating includes combining the assigned pixel values.

4. The method of claim 1, wherein, if unpopulated pixel locations exist in the upsampled frame after the populating, then the populating includes interpolating nearest populated upsampled pixel locations to generate pixel values for the unpopulated upsampled pixel locations.

5. The method of claim 1, wherein the image capture device includes a plurality of different types of detector elements.

6. The method of claim 5, wherein each undersampled frame comprises image data captured by a given type of the detector elements, and wherein the resulting frame comprises image data for the given type of detector element.

7. The method of claim 5, wherein the different types of detector elements include detector elements having different wavelength sensitivities or different polarization sensitivities.

8. The method of claim 5, wherein the different types of detector elements are arranged in an array according to a repeating pattern.

9. The method of claim 8, wherein the repeating pattern is selected according to motion characteristics of the image data being captured.

10. The method of claim 5, wherein the method is performed in parallel for each of the different types of detector elements of the image capture device to produce resulting frames for each of the different types of detector elements.

11. The method of claim 10, comprising:

combining the resulting frames for each of the different types of detector elements to produce a composite frame.

12. A method for processing undersampled image data, comprising:

aligning an undersampled frame comprising image data to a reference frame;

assigning pixel values for pixel locations in the aligned undersampled frame to closest corresponding pixel locations in an upsampled reference frame;

combining, for each upsampled pixel location, the pixel value or values assigned to the upsampled pixel location with a previously combined pixel value for the upsampled pixel location and incrementing a count of the number of pixel values assigned to the upsampled pixel location;

repeating the aligning, the assigning, and the combining for a plurality of undersampled frames; and

normalizing, for each upsampled pixel location, the combined pixel value by the count of the number of pixel values assigned to the upsampled pixel location to produce a resulting frame of image data.

13. The method of claim 12, wherein the image data includes dithered image data.

14. The method of claim 12, wherein the image capture device includes a plurality of different types of detector elements.

15. The method of claim 14, wherein each undersampled frame comprises image data captured by a given type of the detector elements, and wherein the resulting frame comprises image data for the given type of detector element.

16. The method of claim 14, wherein the different types of detector elements include detector elements having different wavelength sensitivities or different polarization sensitivities.

17. The method of claim 14, wherein the different types of detector elements are arranged in an array according to a repeating pattern.

18. The method of claim 17, wherein the repeating pattern is selected according to motion characteristics of the image data being captured.

19. The method of claim 14, wherein the method is performed in parallel for each of the different types of detector elements of the image capture device to produce resulting frames for each of the different types of detector elements.

20. The method of claim 19, comprising:

combining the resulting frames for each of the different types of detector elements to produce a composite frame.

21. A system for processing undersampled image data, comprising:

an image capture device; and

a processing device configured to process a plurality of undersampled frames comprising image data captured by the image capture device,

wherein the processing device is configured to process the undersampled frames by aligning each undersampled frame to a reference frame, accumulating pixel values for pixel locations in the aligned undersampled frames, assigning the pixel values accumulated for the pixel locations in the aligned undersampled frames to closest corresponding pixel locations in an upsampled reference frame, and populating the upsampled frame with a combination of the assigned pixel values to produce a resulting frame of image data.

22. The system of claim 21, wherein the image data includes dithered image data.

23. The system of claim 21, wherein the image capture device comprises:

a focal place array.

24. The system of claim 21, wherein the image capture device includes a plurality of different types of detector elements.

25. The system of claim 24, wherein each undersampled frame comprises image data captured by a given type of the detector elements, and wherein the resulting frame comprises image data for the given type of detector element.

26. The system of claim 24, wherein the different types of detector elements include detector elements having different wavelength sensitivities or different polarization sensitivities.

27. The system of claim 24, wherein the different types of detector elements are arranged in an array according to a repeating pattern.

28. The system of claim 27, wherein the repeating pattern is selected according to motion characteristics of the image data being captured.

29. The system of claim 24, wherein the processing device is configured to process undersampled frames for each of the different types of detector elements in parallel to produce resulting frames for each of the different types of detector elements.

30. The system of claim 29, wherein the processing device is configured to combine the resulting frames for each of the different types of detector elements to produce a composite frame.

31. The system of claim 30, comprising:

a display device configured to display the composite image.

32. The system of claim 30, wherein the processing device is configured to process the composite image in accordance with a target recognition or target tracking application.

33. The system of claim 21, wherein, if more than one pixel value is assigned to a given upsampled pixel location, the processing device is configured to combine the assigned pixel values.

34. The system of claim 21, wherein, if unpopulated pixel locations exist in the upsampled frame after the populating, the processing device is configured to interpolate nearest populated upsampled pixel locations to generate pixel values for the unpopulated upsampled pixel locations.

35. A system for processing undersampled image data, comprising:

an image capture device; and

a processing device configured to align an undersampled frame comprising image data captured by the image capture device to a reference frame, assign pixel values for pixel locations in the aligned undersampled frame to closest corresponding pixel locations in an upsampled reference frame, and combine, for each upsampled pixel location, the pixel value or values assigned to the upsampled pixel location with a previously combined pixel value for the upsampled pixel location and increment a count of the number of pixel values assigned to the upsampled pixel location,

wherein the processing device is configured to repeat the aligning, the assigning, and the combining for a plurality of undersampled frames and, for each upsampled pixel location, normalize the combined pixel value by the count of the number of pixel values assigned to the upsampled pixel location to produce a resulting frame of image data.

36. The system of claim 35, wherein the image data includes dithered image data.

37. The system of claim 35, wherein the image capture device comprises:

a focal place array.

38. The system of claim 35, wherein the image capture device includes a plurality of different types of detector elements.

39. The system of claim 38, wherein each undersampled frame comprises image data captured by a given type of the detector elements, and wherein the resulting frame comprises image data for the given type of detector element.

40. The system of claim 38, wherein the different types of detector elements include detector elements having different wavelength sensitivities or different polarization sensitivities.

41. The system of claim 38, wherein the different types of detector elements are arranged in an array according to a repeating pattern.

42. The system of claim 41, wherein the repeating pattern is selected according to motion characteristics of the image data being captured.

43. The system of claim 38, wherein the processing device is configured to process undersampled frames for each of the different types of detector elements in parallel to produce resulting frames for each of the different types of detector elements.

44. The system of claim 43, wherein the processing device is configured to combine the resulting frames for each of the different types of detector elements to produce a composite frame.

45. The system of claim 44, comprising:

a display device configured to display the composite image.

46. The system of claim 44, wherein the processing device is configured to process the composite image in accordance with a target recognition or target tracking application.