Method and apparatus providing perspective correction and/or image dewarping

• US 8,411,998 B2
• Filed: 08/20/2008
• Issued: 04/02/2013
• Est. Priority Date: 07/17/2008
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Huggett, Anthony, Kirsch, Graham
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US12/222,950
• Publication Number:
  US 20100014770A1
• Time in Patent Office: 1,686 Days
• Field of Search: 382275-277, 382/293, 382/295, 382/296, 382/298, 345/427, 348/580, 348/581, 348/583
• US Class Current: 382/293
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image

Abstract

Methods and apparatuses for providing dewarping and/or perspective correction of an input image are disclosed. Described embodiments include processing that provides dewarping and/or perspective correction by associating pixel values identified by input pixel addresses corresponding to an input image with output pixel addresses corresponding to an output image. An image processor having a storage circuit and an address mapping unit for determining a corresponding input pixel address from an output pixel address is also disclosed.

Claims

1. An image processor that creates an output image from an input image, the image processor comprising:

a storage circuit that stores a plurality of lines of pixel values from the input image, each pixel value having an associated input pixel address, and outputs pixel values as an output image;

an output address generator that specifies an output address for a pixel value in the output image; and

an address mapping unit that receives the output address and determines at least one input pixel value from the input image stored in the storage circuit which the storage circuit should associate with the output address, wherein the address mapping unit provides dewarping of the input image by applying vertical scaling to a vertical output index of the output address and horizontal scaling to a horizontal output index of the output address to determine dewarped horizontal and vertical indexes corresponding to an input pixel address, wherein the address mapping unit further comprises;

a first polynomial function generator that determines a radial gain for the horizontal and vertical scaling of dewarping by using a squared horizontal offset value and a squared vertical offset value.

2. The image processor of claim 1, wherein the storage circuit further comprises:

a buffer circuit that receives the plurality of lines of pixel values from the input image and outputs at least one of the pixel values in accordance with the address mapping unit.

3. The image processor of claim 2, wherein the storage circuit further comprises:

an interpolation filter that receives the at least one of the pixel values that is output from the buffer and uses interpolation to produce a corresponding output pixel value.

4. The image processor of claim 1, wherein the address mapping unit provides perspective correction of the input image by applying vertical shifting to a vertical output index of the output address and horizontal scaling to a horizontal output index of the output address, and outputs perspective-corrected horizontal and vertical indexes corresponding to an input pixel address.

5. The image processor of claim 4, wherein the address mapping unit further comprises:

a second polynomial function generator that provides the vertical shifting for the perspective correction; and

third polynomial function generator that provides a horizontal gain for the horizontal scaling for the perspective correction.

6. The image processor of claim 1, wherein the address mapping unit is further configured to perform:

perspective correction of the input image by applying vertical shifting to a vertical output index of the output address and horizontal scaling to a horizontal output index of the output address to determine perspective-corrected horizontal and vertical indexes and

wherein the dewarping is perfomed on the perspective-corrected vertical index and the perspective-corrected horizontal index.

7. The image processor of claim 1, wherein the image processor is part of a system for capturing and processing the input image, the system further comprising:

an imaging device having a pixel array; and

an imager control circuit for controlling capture of the input image.

8. The image processor of claim 7, wherein the system is contained on a single chip.

9. An image processor that creates an output image from an input image, the image processor comprising:

a storage circuit that stores a plurality of lines of pixel values from the input image, each pixel value having an associated input pixel address, and outputs pixel values as an output image;

an output address generator that specifies an output address for a pixel value in the output image; and

an address mapping unit that receives the output address and determines at least one input pixel value from the input image stored in the storage circuit which the storage circuit should associate with the output address, wherein the address mapping unit provides dewarping of the input image, wherein the address mapping unit is configured to provide;

perspective correction of the input image by applying vertical shifting to a vertical output index of the output address and horizontal scaling to a horizontal output index of the output address to determine perspective-corrected horizontal and vertical indexes; and

dewarping of the input image by applying vertical scaling to the perspective corrected vertical output index and horizontal scaling to the perspective-corrected horizontal index to determine horizontal and vertical input indexes corresponding to an input pixel address, the address mapping unit further comprising;

a first polynomial function generator that provides a value for the vertical shifting for the perspective correction;

a second polynomial function generator that provides a horizontal gain for the horizontal scaling for the perspective correction; and

a third polynomial function generator that determines a radial gain for the horizontal scaling and the vertical scaling of the dewarping by using a squared horizontal offset value and a squared vertical offset value.

10. The image processor of claim 9, wherein the address mapping unit receives:

the vertical output index and horizontal output index of the output address provided by the output address generator;

a first set of coefficients used by the first polynomial function generator for determining a vertical offset for the vertical shifting for perspective correction;

a second set of coefficients used by the second polynomial function generator for determining a horizontal gain for the horizontal scaling for perspective correction;

an output horizontal perspective center index representing the horizontal center of the output image;

an optical horizontal perspective center index representing the location of the horizontal center for the desired perspective of the input image;

a third set of coefficients used by the third polynomial function generator to determine the radial gain for the horizontal and vertical scaling of dewarping; and

horizontal and vertical optical center indexes representing the center pixel address of the input image.

11. The image processor of claim 10, wherein the first, second, and third sets of input coefficients are received from an external source.

12. The image processor of claim 10, wherein the first, second, and third sets of input coefficients are stored in the address mapping unit.

13. The image processor of claim 9, wherein the address mapping unit further comprises a reciprocal element that receives an output of the third polynomial function generator and outputs a reciprocal of the output of the third polynomial function generator.

14. An image processor that creates an output image from an input image, the image processor comprising:

a storage circuit that stores a plurality of lines of pixel values from the input image, each pixel value having an associated input pixel address, and outputs pixel values as an output image;

an output address generator that specifies an output address for a pixel value in the output image; and

an address mapping unit that receives the output address and determines at least one input pixel value from the input image stored in the storage circuit which the storage circuit should associate with the output address, wherein the address mapping unit provides dewarping of the input image, wherein the address mapping unit is configured to provide;

perspective correction of the input image by applying vertical shifting to a vertical output index of the output address and horizontal scaling to a horizontal output index of the output address to determine perspective-corrected horizontal and vertical indexes; and

dewarping of the input image by applying vertical scaling to the perspective corrected vertical output index and horizontal scaling to the perspective-corrected horizontal index to determine horizontal and vertical input indexes corresponding to an input pixel address, wherein the address mapping unit further comprises;

a first multiplexer that receives and outputs one of squared horizontal and vertical optical offset values computed from pre-programmed perspective-correction coefficients and horizontal and vertical output indexes from an output address generator;

a second multiplexer that receives and outputs one of a squared value of the vertical output index from the output address generator and a sum of the squared horizontal and vertical optical offset values output by the first multiplexer;

a third multiplexer that receives and outputs one of;

a first set of even and a first set of odd coefficients used for determining a value for the vertical shifting for the perspective correction;

a second set of even and a second set of odd coefficients used for determining a horizontal gain for the horizontal scaling for the perspective correction; and

a fifth set of coefficients used for determining the radial gain for the horizontal scaling and the vertical scaling of the dewarping by using a squared horizontal offset value and a squared vertical offset value;

a polynomial function generator receiving input from the second and third multiplexers; and

a control unit that controls the first, second, and third multiplexers.

15. The image processor of claim 14,

further comprising a reciprocal element that receives the radial gain and outputs a reciprocal of the radial gain.

16. A method of processing an input image comprising:

storing a plurality of pixel values of the input image in a buffer, each pixel of the input image having a corresponding input pixel address;

designating an output pixel address having a horizontal output index and a vertical output index;

determining an input pixel address corresponding to the designated output pixel address by calculating a horizontal input index and a vertical input index, wherein calculating the horizontal and vertical input indexes provides dewarping of the input image;

determining an output pixel value associated with the output pixel address based on the pixel value at the corresponding at least one input pixel address; and

outputting the output pixel value, wherein calculating the horizontal and vertical input indexes provides for;

perspective correction of the input image by applying vertical shifting to the vertical output index and horizontal scaling to the horizontal output index to determine perspective-corrected horizontal and vertical indexes; and

dewarping of the input image by applying vertical scaling to the perspective-corrected vertical index and horizontal scaling to the perspective-corrected horizontal index to determine horizontal and vertical input indexes corresponding to an input pixel address which the storage circuit should associate with the output address, wherein calculating the horizontal and vertical input indexes comprises;

generating a perspective-corrected vertical index from the vertical output index and a first set of coefficients for determining the amount of vertical shifting;

generating perspective-corrected gain from the vertical output index and a second set of coefficients for determining the amount of horizontal scaling for dewarping;

subtracting an output horizontal perspective center index from the horizontal output index to produce a horizontal offset, wherein the output horizontal perspective center index represents the horizontal center of the output image;

multiplying the perspective-corrected gain and the horizontal offset, and

adding a desired new horizontal perspective center to the product to generate a perspective-corrected horizontal index;

subtracting a vertical optical center index from the vertical output index to produce a vertical optical offset value, and subtracting a horizontal optical center index from the horizontal output index to produce a horizontal optical offset value, wherein the vertical and horizontal optical center indexes represent the center pixel address of the input image;

calculating the radial gain by;

squaring the horizontal optical offset value;

squaring the vertical optical offset value;

generating a radial gain from a sum of the squared horizontal and vertical optical offset values and a third set of coefficients for determining the horizontal scaling and the vertical scaling for the dewarping; and

outputting the radial gain;

multiplying the vertical optical offset and the radial gain, and adding the vertical optical center index to the product to generate the dewarped vertical index; and

multiplying the horizontal optical offset and the radial gain, and adding the horizontal optical center index to the product to generate the dewarped horizontal index.

17. The method of claim 16, wherein the horizontal and vertical input indexes are calculated as integer values.

18. The method of claim 16, wherein the horizontal and vertical input indexes are calculated as scaled floating point values, each having an integer component and a fractional component.

19. The method of claim 16, wherein the horizontal and vertical input indexes are equal to the perspective-corrected horizontal and vertical indexes, respectively.

20. The method of claim 16, wherein the horizontal and vertical input indexes are equal to the dewarped horizontal and vertical indexes, respectively.

21. The method of claim 16, wherein calculating the radial gain further comprises calculating a reciprocal value of the generated radial gain, and outputting the reciprocal value as the radial gain.

22. The method of claim 16, wherein the steps of storing a plurality of pixel values, designating an output pixel address, determining an input pixel address, determining an output pixel value, and outputting the output pixel value are implemented on a single microchip.

23. The method of claim 16, wherein the steps of storing a plurality of pixel values, designating an output pixel address, determining an input pixel address, determining an output pixel value, and outputting the output pixel value are implemented by a single processing module in a computer processor.

24. A method of processing an input image, the method comprising:

storing a plurality of rows of pixel values of the input image, each pixel of the input image having an input pixel address, each input pixel address having a horizontal index and a vertical index; and

generating a row of output pixel values, each output pixel value having an output pixel address having a horizontal output index and a vertical output index, wherein each of the output pixel values is determined by;

designating the output pixel address corresponding to the output pixel value;

determining an input pixel address corresponding to the designated output pixel address, wherein the determination includes calculating a horizontal input index and a vertical input index, wherein the determination provides for at least one of perspective correction and dewarping;

determining the output pixel value associated with the output pixel address based on the stored pixel value corresponding to the determined input pixel address; and

outputting the output pixel value, wherein generating a row of output pixel values further comprises;

generating a vertical offset perspective-corrected index and storing it in a first register; and

generating a perspective-corrected horizontal-gain and storing it in a second register,

and wherein determining an input pixel address corresponding to the designated output pixel address for each output pixel address in the row of output pixel values further comprises;

inputting the horizontal output index and an output horizontal perspective center in a first subtraction means;

inputting the result of the first subtraction means and the perspective-corrected horizontal gain stored in the second register into a second product means to generate a perspective-corrected horizontal index;

entering the perspective-corrected horizontal index and a difference of an optical horizontal perspective center and an optical center horizontal index into a second summing means to generate a horizontal offset perspective-corrected index;

generating a radial gain based on a square of the horizontal offset perspective-corrected index and a square of the vertical offset perspective-corrected index;

inputting the vertical offset perspective-corrected index and the radial gain into a third product means;

inputting the horizontal offset perspective-corrected index and the radial gain into a fourth product means;

inputting a result of the third product means and the optical center vertical index to a summing means to generate the input vertical index; and

inputting a result of the fourth product means and the optical center horizontal index to a summing means to generate the input horizontal index.

25. The method of claim 24, wherein generating a radial gain further comprises calculating a reciprocal value of the generated radial gain, and inputting the reciprocal value as the radial gain into the third product means.

26. The method of claim 24, wherein the steps of storing a plurality of rows of pixel values of the input image and generating a row of output pixel values are performed on a single microchip.

27. The method of claim 24 wherein the steps of storing a plurality of rows of pixel values of the input image and generating a row of output pixel values are performed by a single processing module on a computer processor.