Image sensor method and apparatus having hardware implemented edge detection processing

• US 7,151,844 B2
• Filed: 12/06/2001
• Issued: 12/19/2006
• Est. Priority Date: 12/06/2001
• Status: Expired due to Fees

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: General Motors Corporation (Motors Liquidation Co. GUC Trust)
• Inventors: Lesperance, Ronald Michael, Schauerte, Frank J., Stevenson, Charles Neil, Troxell, John Richard
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Strege, John B.
• Application Number:
  US10/011,172
• Publication Number:
  US 20030108221A1
• Time in Patent Office: 1,839 Days
• Field of Search: 382103-104, 382/199, 382/205, 382260-269, 701/1, 701200-226, 701300-302, 280/81.6
• US Class Current: 382/104
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H01L 27/14621 Colour filter arrangements

Abstract

An image sensor comprising an integrated circuit chip that includes both a pixel array and edge detection circuitry hardwired together on the chip. The pixel array provides an addressable, non-destructive readout of the pixel data, with the edge detection circuit providing a hardwired implementation of a pair of n×n masks that are convolved with the pixel array data to thereby generate horizontal and vertical edge data that is combined together and outputted by the image sensor. The masks are implemented as a pair of one-dimensional filters, with one of the filters being a one-dimensional Gaussian filter used to smooth the pixel data along one axis and the other filter providing a first derivative Gaussian function to detect edges along the other, orthogonal axis. The non-destructibility of the readout from the pixel array eliminates the need for buffering large amounts of pixel data and the use of separable, one-dimensional filters permits each mask to be implemented using only n simultaneous outputs from the array, 2n weights, and n−1 buffers for temporary storage of the smoothed pixel data.

Claims

1. A method for detecting edges in an image using an image sensor that comprises a two-dimensional array of addressable pixels, with each of said pixels providing a data value indicative of a portion of the image sensed by that pixel, the method comprising the steps of:

(a) sensing the image using the image sensor; and

(b) carrying out the following steps for each of a number of selected pixels using a two-dimensional localized group of pixels surrounding each selected pixel, wherein each of the localized groups of pixels comprises a plurality of one-dimensional sets of pixels, each one-dimensional set being orthogonally offset from the other one-dimensional sets in the group;

(b1) accessing a one-dimensional set of pixels contained within the localized group of pixels;

(b2) reading the data values associated with the pixels accessed in step (b1);

(b3) determining an intermediate data value using a smoothing function that is applied to the data values read in step (b2);

(b4) repeating steps (b1) through (b3) for each of the one-dimensional sets of pixels contained within the localized group to thereby produce a set of said intermediate data values; and

(b5) determining a final data value for the selected pixel using a differencing function that is applied to the set of intermediate data values;

wherein step (b) further comprises processing the selected pixels sequentially such that, for each selected pixel, steps (b1) through (b5) are carried out for that selected pixel prior to carrying out any of the steps (b1) through (b5) for the next selected pixel.

2. The method of claim 1, wherein step (b) further comprises detecting edges in a first one of the dimensions of said array and wherein the one-dimensional set of pixels used in step (b1) comprises a set of pixels extending in the second dimension of said array.

3. The method of claim 2, further comprising to step of(c) detecting edges in the second dimension of said array by carrying out the following steps for each of a number of selected pixels using a localized group of pixels surrounding each selected pixel:

(c1) accessing a one-dimensional set of pixels that are contained within the localized group of pixels and that extend in the first dimension;

(c2) reading the data values associated with the pixels accessed in step (c1);

(c3) determining an intermediate data value using a smoothing function that is applied to the data values read in step (c2);

(c4) repeating steps (c1) through (c3) for each of the one-dimensional sets of pixels contained within the localized group to thereby produce a set of said intermediate data values;

(c5) determining a final data value for the selected pixel using a differencing function that is applied to the set of intermediate data values.

4. The method of claim 1, wherein said smoothing function comprises a one-dimensional Gaussian function.

5. The method of claim 1, wherein said differencing function comprises a one-dimensional, first derivative Gaussian function.

6. The method of claim 1, wherein the smoothing function comprises a set of weights and wherein step (b3) further comprises:

determining a number of multiplication products by multiplying each data value wit an associated one of said weights, and

determining the intermediate data value by summing together the multiplication products.

7. The method of claim 1, further comprising carrying out steps (b1) through (b5) using hardwired circuitry formed on a single chip that includes the image sensor.

8. The method of claim 1, wherein step (a) further comprises sensing and recording the image using an image sensor having a non-destructive readout, and wherein step (b) further comprises carrying out steps (b1) through (b5) for each of a number of adjacent pixels, whereby the data value recorded by at least some of said pixels is read out more than once.

9. The method of claim 1, wherein step (b) further comprises carrying out steps (b1) through (b5) by buffering of at least some of said intermediate data values and without buffering of the data values read in step (b2).

10. An image sensor for use in sensing an image and outputting image data that identifies edges contained within the image, comprising:

an array of addressable pixels, each of which provides a non-destructive readout of an analog signal indicative of a portion of the image sensed by that pixel; and

an edge detection circuit comprising;

a smoothing filter circuit connected to said array, said smoothing filter circuit having an output for supplying filtered output data and having one or more inputs for receiving a plurality of the analog signals from said pixels; and

a differencing filter circuit having an input coupled to the output of said smoothing filter circuit to receive the filtered output data, said differencing filter circuit having an output for supplying image data that identifies edges contained within the image.

11. An image sensor as defined in claim 10, wherein at least one of said filter circuits comprises one or more switched-capacitor multiplying circuits.

12. An image sensor as defined in claim 11, wherein said smoothing filter circuit includes n switched-capacitor multiplying circuits, each of which has an input connected to said array to simultaneous receive n analog signals from n different pixels in said array, and wherein said switched-capacitor multiplying circuits include one or more binary weights.

13. An image sensor as defined in claim 10, wherein said differencing filter circuit includes a plurality of buffers for temporary storage of the filtered output data provided by said smoothing filter circuit.

14. An image sensor as defined in claim 10, wherein said smoothing filter circuit is connected to said array via a multiplexor circuit that accesses different sets of pixels from said array and provides the analog signals from the accessed pixels to said smoothing filter.

15. An image sensor as defined in claim 10, wherein said edge detection circuit comprises a horizontal edge detection circuit having a plurality of said inputs connected to said array to simultaneously receive analog signals from different pixels located in any single row of said array.

16. An image sensor as defined in claim 15, further comprising a control circuit connected to said array to provide said horizontal edge detection circuit with analog signals from different groups of contiguous pixels located within any single row of said array and to sequentially provide said horizontal edge detection circuit with analog signals from the same group of pixels located within contiguous rows of said array.

17. An image sensor as defined in claim 15, further comprising a vertical edge detection circuit having a second plurality of said inputs connected to said array to simultaneously receive analog signals from different pixels located in any single column of said array.

18. An image sensor as defined in claim 17, further comprising a control circuit connected to said array to provide said horizontal edge detection circuit with analog signals from different groups of contiguous pixels located within any single row of said array and to sequentially provide said horizontal edge detection circuit with analog signals from the same group of pixels located within contiguous rows of said array;

and

wherein said control circuit is further operable to provide said vertical edge detection circuit with analog signals from different groups of contiguous pixels located within any single column of said array and to sequentially provide said vertical edge detection circuit with analog signals from the same group of pixels located within contiguous columns of said array.

19. An image sensor as defined in claim 10, further comprising a control circuit for selectively providing analog signals from different pixels to said smoothing filter, wherein said smoothing filter circuit and said differencing filter circuit together comprise a mask and wherein said control circuit is operable to convolve said mask with said array to thereby output the image data for each of number of different pixels in said array.

20. An image sensor as defined in claim 10, wherein said array and edge detection circuit together comprise a single integrated circuit.

21. An image sensor as defined in claim 10, wherein said smoothing filter circuit comprises a one-dimensional Gaussian filter.

22. An image sensor as defined in claim 21, wherein said differencing filter circuit comprises a one-dimensional, first derivative Gaussian filter.

23. A method for detecting edges in an image using an image sensor that comprises a two-dimensional array of pixels, with each of said pixels providing a data value indicative of a portion of the image sensed by that pixel, the method comprising the steps of:

providing a mask that comprises a one-dimensional Gaussian function applied to groups of the image data values in a first direction to thereby produce intermediate data values that each represent a combination of the image data values contained in one of said groups, said mask further comprising a separate one-dimensional, first derivative Gaussian function applied to sets of the intermediate data values; and

convolving the mask with the image to produce edge data for each of a number of selected pixels.

24. The method of claim 23, further comprising the steps of:

providing a second mask that comprises a one-dimensional Gaussian function applied to additional groups of the image data values in the second direction to thereby produce additional intermediate data values that each represent a combination of the image data values contained in one of said additional groups, said mask further comprising a separate one-dimensional, first derivative Gaussian function applied to sets of the additional intermediate data values; and

convolving the second mask with the image to produce edge data for each of a number of selected pixels.

25. The method of claim 24, further comprising the step of combining together the edge data obtained as a result of convolving the first and second masks.

26. The method of claim 24, further comprising carrying out each of the steps using an image sensor having a non-destructive readout and using hardwired circuitry formed on a single chip that includes the image sensor.