Method and apparatus for automatic linear shift parallax correction for multi-array image systems

• US 20080239116A1
• Filed: 03/27/2007
• Published: 10/02/2008
• Est. Priority Date: 03/27/2007
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Micron Technology, Inc.
• Inventors: Smith, Scott
• Granted Patent: US 7,738,017 B2
• Time in Patent Office: Days
• US Class Current: 348/250
• CPC Class Codes: G06T 2207/20068 Projection on vertical or horizontal image axis
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: H01L 27/14603 Special geometry or disposition of pixel-elements, address-lines or gate-electrodes
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H04N 23/15 Image signal generation with circuitry for avoiding or correcting image misregistration
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

A method and apparatus for correcting linear shift parallax error in multi-array image systems. Average pixel cell signal values for pixel cells within a summing window of each column or row of at least two sub-arrays are computed during read-out. The images of the sub-arrays are correlated based on the averages, then shifted based on a result of the correlation function to correct the exhibited parallax error. The summation, correlation, and shifting can be performed by a pixel pipeline processing circuit.

Claims

1. A method of correcting parallax error in images acquired from a plurality of pixel cell arrays, said method comprising:

computing a plurality of either column sums or row sums of pixel signal values from each of the plurality of pixel cell arrays;

computing an average pixel signal value corresponding to each of said plurality of sums;

correlating the average pixel signal values of the plurality of pixel cell arrays; and

shifting an image formed from at least one of the plurality of pixel cell arrays based on a result of the correlation.

2. The method of claim 1, wherein the plurality of pixel cell arrays comprise three pixel cell arrays.

3. The method of claim 2, wherein a first pixel cell array is sensitive to red light, a second pixel cell array is sensitive to green light, a third pixel cell array is sensitive to blue light.

4. The method of claim 2, wherein average pixel signal values from a first pixel cell array and a second pixel cell array are correlated, average pixel signal values from the second pixel cell array and a third pixel cell array are correlated, and at least two images formed from the plurality of pixel cell arrays are shifted based on a result of the correlations.

5. The method of claim 1, wherein the plurality of sums is computed in conjunction with a pixel signal read-out operation of the pixel cell arrays.

6. The method of claim 1, wherein each of said plurality of sums is a column sum.

7. The method of claim 1, wherein each of said plurality of sums is a row sum.

8. The method of claim 6, wherein fewer than all of the pixel signal values in a column are included in each column sum.

9. The method of claim 8, wherein a first and last group of pixel signal values in each column are excluded from the respective column sum and a middle group of pixel signal values are included in the respective column sum.

10. The method of claim 9, wherein the middle group of pixel signal values comprises 2^k pixel signal values where k is an integer.

11. The method of claim 10, wherein each average pixel signal value is computed by shifting the corresponding column sum by k bits.

12. The method of claim 1, wherein the correlating step comprises determining a maximum of a correlation function.

13. The method of claim 12, wherein the correlation function is

14. The method of claim 1, wherein the shifting step comprises shifting an image along a single axis.

15. A method of correcting linear shift parallax error in an imager having a plurality of linearly-arranged sub-arrays, said method comprising:

reading pixel cell values from a plurality of sub-arrays;

computing sums of pixel values for columns of pixels as said pixel cell values are read-out;

computing average pixel cell values for each of the columns;

correlating images formed from the plurality of sub-arrays based on the average pixel cell values; and

shifting an image formed from at least one of the plurality of sub-arrays based on a result of the correlation.

16. The method of claim 15, further comprising combining the images formed from the plurality of sub-arrays after at least one of the images has been shifted.

17. The method of claim 15, wherein the sums and the average pixel cell values are computed by an image processing circuit.

18. The method of claim 17, wherein the image processing circuit is on the same chip as the plurality of sub-arrays.

19. The method of claim 15, wherein the correlating and shifting steps are performed by an image processing circuit.

20. The method of claim 17, wherein the correlating and shifting steps are performed by a central processing unit coupled to the image processing circuit.

21. A method of correcting parallax error for images captured by a plurality of pixel cell arrays, said method comprising:

correlating data from pixel values of images respectively captured by said pixel cell arrays;

determining from said correlated data the amount of parallax error between said captured images; and

shifting at least one of said images relative to another of said images to reduce said parallax error.

22. The method of claim 21, wherein said data from pixel values of imagers comprises averages of column sums of pixel values computed during read-out of said pixel cell arrays.

23. An imager, comprising:

a plurality of sub-arrays arranged in a linear configuration, each sub-array comprising pixel cells organized in rows and columns;

a read-out circuit for reading out pixel cell values from said plurality of sub-arrays; and

a pixel signal processing circuit configured to compute a sum of the pixel cell values for either columns or rows in each of said plurality of sub-arrays, to compute an average associated with each sum, to correlate images formed from the plurality of sub-arrays based on the averages, and to shift an image formed from at least one of the plurality of sub-arrays in accordance with a result of the correlation.

24. The imager of claim 23, wherein the pixel signal processing circuit is configured to compute a sum comprising only values of pixel cells within a summing window, wherein the summing window comprises fewer than all of the pixel cells within each of either a column or a row.

25. The imager of claim 23, wherein the pixel signal processing circuit is configured to compute a sum by adding a pixel cell value from the read-out circuit to a partial sum stored in a memory and write the result back to the memory.

26. The imager of claim 23, wherein the memory is a line buffer memory.

27. The imager of claim 23, wherein the pixel signal processing circuit is a pixel pipeline processor.

28. The imager of claim 23, wherein the pixel signal processing circuit is configured to operate at no less than the pixel read-out clock rate.

29. The imager of claim 23, wherein the result of the correlation is a maximum of a correlation function.

30. The method of claim 29, wherein the correlation function is

31. The imager of claim 23, wherein the plurality of sub-arrays comprises:

a sub-array configured to detect red light;

a sub-array configured to detect green light; and

a sub-array configured to detect blue light.

32. The imager of claim 23, wherein each of the plurality of sub-arrays is spaced between approximately 2 millimeters and approximately 3 millimeters from an adjacent sub-array.

33. The imager of claim 23, further comprising an output port configured to output correlated images to a video device.

34. An imager, comprising:

a red sub-array, a green sub-array, and a blue sub-array arranged in a linear configuration, wherein one of the sub-arrays is a center sub-array;

a read-out circuit for reading out pixel cell values from the sub-arrays; and

a pixel processing circuit, comprising;

a summing circuit configured to compute a sum of the pixel cell values for columns in the sub-arrays; and

a buffer memory associated with the summing circuit for storing sum values,

wherein the processor is configured to correlate images from a first sub-array and the center sub-array using said sum values, shift an image formed from the first sub-array based on a result of the correlation, correlate images formed from a second sub-array and the center sub-array using said sum values, shift the image formed from the second sub-array based on a result of the correlation, and combine the shifted images from said first and second sub-arrays with the image from the center sub-array.

35. The imager of claim 34, wherein the green sub-array is the center sub-array.

36. The imager of claim 34, further comprising a lens for focusing an image on said sub-arrays.

37. The imager of claim 36, further comprising a color filter respectively associated with each sub-array such that each sub-array detects respective wavelengths of light.

38. The imager of claim 34, wherein the pixel processing circuit is a pixel pipeline processor.

39. A camera system, comprising:

an imager, comprising;

a plurality of sub-arrays arranged in a linear configuration, each sub-array comprising pixel cells organized into rows and columns and each sub-array responsive to a respective band of wavelengths of light;

a read-out circuit for reading out pixel cell values from said plurality of sub-arrays;

a lens for focusing an image on said sub-arrays; and

a pixel signal processing circuit configured to compute a sum of the pixel cell values for each column in said sub-arrays, to average the sums of the pixel cell values, to correlate images formed from the plurality of sub-arrays using said averages, to shift an image from at least one of the plurality of sub-arrays based on a result of the correlation, and to combine shifted and unshifted images corresponding to said plurality of sub-arrays.

40. The imager of claim 39, further comprising an output port configured to output combined images.