Methods and apparatus for selective pixel readout for image transformation

• US 10,438,332 B2
• Filed: 05/18/2017
• Issued: 10/08/2019
• Est. Priority Date: 05/18/2017
• Status: Active Grant

• Current Assignee: Semiconductor Components Industries LLC (ON Semiconductor Corporation)
• Original Assignee: Semiconductor Components Industries LLC (ON Semiconductor Corporation)
• Inventors: Clark, II, Airell Richard, Brown, Timothy, Perez, Ignacio J., Hernandez, Agustin B.
• Primary Examiner(s): Jerabek, Kelly L
• Application Number:
  US15/599,305
• Publication Number:
  US 20180336667A1
• Time in Patent Office: 873 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/003 for viewing trailer hitches
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/306 using a re-scaling of images
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 25/443 by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming

Abstract

Various embodiments of the present technology may comprise a method and apparatus for pixel readout. The method and apparatus may comprise a pixel array capable of reading out one or more non-rectilinear subsets of the pixel array to form a non-rectilinear primary image. The one or more non-rectilinear subsets may be selected according to a desired rectilinear output image, wherein the rectilinear output image is formed using only the one or more subsets. The primary image may then be transformed to form the rectilinear output image.

Claims

1. An imaging apparatus, comprising:

an image signal processor comprising a spatial transform engine, wherein the spatial transform engine is configured to transform a primary image into a desired secondary image utilizing a spatial transformation algorithm;

a camera module, comprising a plurality of pixels forming rows and columns, coupled to the image signal processor and configured to;

generate a pixel signal for each pixel from the plurality of pixels;

define a first subset of pixels from the plurality of pixels;

wherein;

the first subset is defined according to the desired secondary image; and

the first subset comprises a non-rectilinear outer perimeter;

define a second subset of pixel from the plurality of pixels;

read out a first plurality of pixel signals corresponding to the first subset of pixels to the spatial transform engine; and

discard a second plurality of pixel signals corresponding to the second subset of pixels prior to being readout from the respective pixels.

2. The imaging apparatus according to claim 1, further comprising a wide-angle lens positioned adjacent to the plurality of pixels.

3. The imaging apparatus according to claim 1, wherein:

the primary image is non-rectilinear; and

the desired secondary image is rectilinear.

4. The imaging apparatus according to claim 1, wherein:

the first plurality of pixel signals forms the primary image; and

the first plurality of pixel signals are transformed into the desired secondary image.

5. The imaging apparatus according to claim 1, wherein the non-rectilinear outer perimeter of the first subset is related to the spatial transformation algorithm.

6. The imaging apparatus according to claim 1, wherein:

the first subset corresponds to the spatial transformation algorithm; and

the spatial transformation algorithm is selected according to the desired secondary image.

7. The imaging apparatus according to claim 1, wherein the desired secondary image comprises at least one of:

a backup view, a trailer hitch view, a triptych view, and a split side view.

8. The imaging apparatus according to claim 1, wherein:

the first subset comprises a plurality of rows, wherein each row comprises a starting pixel and an ending pixel; and

at least one edge of the starting pixel and at least one edge of the ending pixel forms a portion of the outer perimeter of the first subset.

9. A method for forming an image with an imaging apparatus having a lens, comprising:

collecting charge with a pixel array;

selecting a desired output image;

identifying a spatial transformation algorithm based on the desired output image;

identifying a first pixel subset of the pixel array based on the desired output image, wherein the pixel subset forms a non-rectilinear shape;

reading out a first plurality of pixel signals from the first pixel subset;

discarding a second plurality of pixel signals from a second pixel subset prior to being readout from the pixel array;

forming a primary image utilizing the first plurality of pixel signals from the first pixel subset; and

transforming the primary image to the desired output image utilizing the spatial transformation algorithm.

10. The method according to claim 9, wherein:

the primary image is non-rectilinear; and

the desired output image is rectilinear.

11. The method according to claim 9, wherein the desired output image comprises at least one of:

a backup view, a trailer hitch view, a triptych view, and a split side view.

12. The method according to claim 9, wherein the non-rectilinear shape of the first pixel subset is based on the spatial transformation algorithm and a shape of the lens.

13. The method according to claim 9, wherein:

the spatial transformation algorithm corresponds to a plurality of pixel subsets from the pixel array; and

two of the pixel subsets from the plurality of pixel subsets are mirror images of each other.

14. The method according to claim 9, wherein:

reading out the first plurality of pixel signals from the first pixel subset comprises sequentially reading out each row of the first pixel subset;

each row comprises a starting pixel and ending at an ending pixel; and

at least one edge of the starting pixel and at least one edge of the ending pixel forms a portion of an outer perimeter of the non-rectilinear shape.

15. A system, comprising:

an imaging apparatus, comprising;

a camera module, coupled to an image signal processor, comprising;

a plurality of pixels forming rows and columns; and

a lens positioned adjacent to the plurality of pixels; and

the image signal processor, coupled to the camera module, comprising a spatial transform engine configured to;

receive first pixel data from a predetermined pixel subset of the plurality of pixels, wherein;

the pixel subset is selected according to a desired output image;

the pixel subset forms a non-rectilinear shape having an outer perimeter;

construct a primary image utilizing the first pixel data from the predetermined pixel subset; and

forming the desired output image utilizing the first pixel data from the primary image and a spatial transformation algorithm;

a host processor coupled to the imaging apparatus and configured to;

receive a signal relating to the desired output image;

transmit a readout operation instruction, corresponding to the desired output image, to the imaging apparatus; and

transmit a reset operation instruction to discard the second pixel data, wherein the second pixel data is discarded prior to being readout from the respective pixels; and

a user interface coupled to the host processor and configured to display the desired output image.

16. The system according to claim 15, wherein:

the primary image is non-rectilinear; and

the desired output image is rectilinear.

17. The system according to claim 15, wherein the non-rectilinear shape of the predetermined pixel subset is related to the spatial transformation algorithm.

18. The system according to claim 15, wherein:

the predetermined pixel subset corresponds to the spatial transformation algorithm; and

the spatial transformation algorithm is selected according to the desired output image.

19. The system according to claim 15, wherein the desired output image comprises at least one of:

a backup view, a trailer hitch view, a triptych view, and a split side view.

20. The system according to claim 15, wherein:

each row of pixels in the pixel subset comprises a starting pixel and an ending pixel; and

at least one edge of the starting pixel and at least one edge of the ending pixel forms a portion of the outer perimeter of the non-rectilinear shape.