System and method for independent image sensor parameter control in regions of interest

• US 8,441,535 B2
• Filed: 03/05/2008
• Issued: 05/14/2013
• Est. Priority Date: 03/05/2008
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Morin, Jeffrey L.
• Primary Examiner(s): Pasiewicz, Daniel M
• Application Number:
  US12/042,998
• Publication Number:
  US 20090225189A1
• Time in Patent Office: 1,896 Days
• Field of Search: 348/148, 348/149, 348/207.1, 348/207.11, 3482114-2116, 348/211.14, 348/216.1, 348/222.1, 348/229.1, 348/230.1, 382104-105
• US Class Current: 348/148
• CPC Class Codes: G03B 7/003 setting of both shutter and diaphragm G03B7/02 - G03B7/22 take precedence
• CPC Class Codes: H04N 25/44 by partially reading an SSIS array
• CPC Class Codes: H04N 25/443 by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 25/46 by combining or binning pixels

Abstract

An image sensing systems supports independently setting sensor parameters in two or more different regions of interest. In one implementation resolution, exposure time, and gain are independently selectable in each region of interest. Independently setting exposure time and gain in different regions of interest improves image recognition when there are large differences in the illumination of objects within an image. Independently setting resolution permits the resolution to be selectively increased in regions of interest requiring high resolution in order to reduce the pixel data bandwidth and processing requirements.

Claims

1. An image sensing system, comprising:

an image sensor having an array of pixels to capture an image;

a controller to independently set image sensor parameters in two or more different portions of the image sensor corresponding to a plurality of regions of interest with the controller selecting default image sensor parameters in other portions of the image outside of the plurality of regions of interest; and

a processor receiving image data from the image sensor, the processor identifying regions of interest, the processor comprising a plurality of sub-modules each supporting a respective region of interest, the plurality of sub-modules providing control signals to the image sensor to selectively adjust image sensor parameters in each of the plurality of regions of interest according to an image recognition criterion,

wherein the captured image comprises;

two or more different portions having two or more set resolutions and at least one additional portion non-overlapping with the two or more different portion and having a lower default resolution,

an average pixel data rate that is lower than that of an image having full high resolution mode,

an amount of pixel data which is less than that of the full high resolution mode image, and

wherein the image sensor parameters comprise resolution, exposure, and gain.

2. The image sensing system of claim 1, wherein

the controller independently sets a local pixel exposure time and pixel resolution in each of the plurality of regions of interest,

the controller selects a minimum resolution and pixel exposure time in other portions of the image outside of the plurality of regions of interest, and

an amount of image data received by the processor from the image sensor for a given image cannot exceed an amount of data contained in the array of pixels.

3. The image sensing system of claim 2, wherein the controller selects resolution by selecting a pixel sub-sampling rate.

4. The image sensing system of claim 2, wherein a minimum pixel resolution is at least a factor of four lower than a maximum image sensor resolution for a region of interest.

5. The image sensing system of claim 2, wherein a minimum pixel resolution is at least a factor of ten lower than a maximum image sensor resolution for a region of interest.

6. The image sensing system of claim 2, wherein for the plurality of regions of interest, each region of interest has the gain, exposure, and resolution independently selected based on an image recognition detection task.

7. The image sensing system of claim 6, wherein each image recognition detection task is a forward looking camera automobile detection task selected from the group consisting of pedestrian detection, headlight detection, stop light detection, road sign detection, license plate detection, brake light detection, and lane detection.

8. The image sensing system of claim 2, the system further comprising an infrared light source illuminating an object for capturing the images.

9. The image sensing system of claim 2, wherein exposure time in regions of interest is based on at least one of an average value or a histogram.

10. The image sensing system of claim 2, wherein resolution in a region of interest is iteratively increased in steps until an image recognition criterion is satisfied.

11. The image sensing system of claim 2, wherein resolution in a region of interest is increased from the minimum image resolution to a maximum image resolution.

22. The image sensing system of claim 1, wherein the image sensor parameters are selectively adjusted iteratively over a sequence of images until the image recognition criterion is satisfied for each region of interest.

12. An image sensing system, comprising:

an image sensor having an array of pixels to capture an image;

a controller to selectively adjust a pixel resolution and exposure time in two or more different portions of the image sensor corresponding to regions of interest of the captured image in response to control signals according to an image recognition criterion; and

an image processor for receiving pixel data from the image sensor and analyzing images, the image processor including a plurality of sub-modules to provide the control signals,

the image sensor having a maximum achievable pixel resolution determined by the array of pixels with the image sensor providing pixel data at a reduced pixel resolution in areas outside of regions of interest to reduce an average data rate of pixel data output from the image sensor for processing,

wherein the captured image comprises;

two or more different portions having two or more set resolutions and at least one additional portion additional portion non-overlapping with the two or more different portion and having a lower default resolution,

an average pixel data rate that is lower than that of an image having full high resolution mode,

an amount of pixel data which is less than that of the full high resolution mode image.

13. The image sensing system of claim 12, wherein said image sensor sub-samples pixels in areas outside of regions of interest.

14. The image sensing system of claim 13, wherein said image sensor increases pixel resolution in a selected region of interest by selectively increasing the sub-sampling rate in the selected region of interest.

15. An image sensing system, comprising:

an image sensor having an array of pixels to capture images and including a controller to independently set a local pixel exposure time and pixel resolution in two or more different portions of the image sensor corresponding to a plurality of regions of interest of a captured image, the image sensor having a maximum achievable pixel resolution determined by the array of pixels with the image sensor providing pixel data at a reduced pixel resolution in areas outside of regions of interest to reduce an average data rate of pixel data output from the image sensor for processing; and

an image processor including image analysis modules to support a plurality of different automobile image recognition detection tasks, the image processor receiving image data from the image sensor, identifying regions of interest for the different image recognition detection tasks, the image processor comprising a plurality of sub-modules for determining adjustments to exposure time, resolution, and gain for each region of interest to perform a selected image recognition detection task, and providing control signals to the image sensor to selectively adjust the resolution and exposure time in each region of interest according to an image recognition criterion,

wherein the captured image comprises;

two or more different portions having two or more set resolutions and at least one additional portion non-overlapping with the two or more different portion and having a lower default resolution,

an average pixel data rate that is lower than that of an image having full high resolution mode,

an amount of pixel data which is less than that of the full high resolution mode image.

16. The image sensing system of claim 15, wherein the automobile image recognition detection tasks are selected from the group consisting of pedestrian detection, headlight detection, stop light detection, road sign detection, license plate detection, brake light detection, and lane detection.

17. An image sensing system to support forward-looking automobile camera applications under illumination conditions in which there is a large range of variation in light intensity from different objects in an image, comprising:

an image sensor having an array of pixels to capture images and including a controller to independently set a local pixel exposure time and gain in two or more different portions of the image sensor corresponding to multiple regions of interest of a captured image; and

an image processor including image analysis modules to support multiple different automobile image recognition detection tasks, the image processor receiving image data from the image sensor, identifying regions of interest for the different image recognition detection tasks, the image processor comprising a plurality of sub-modules for determining adjustments to exposure time and gain for each region of interest to perform a selected image recognition detection task, and providing control signals to the image sensor to selectively adjust the pixel exposure time and gain in each region of interest according to an image recognition criterion,

wherein the captured image comprises;

two or more different portions having two or more set resolutions and at least one additional portion non-overlapping with the two or more different portion and having a lower default resolution,

an average pixel data rate that is lower than that of an image having full high resolution mode.

18. The image sensing system of claim 17, wherein the automobile image recognition detection tasks are selected from the group consisting of pedestrian detection, headlight detection, stop light detection, road sign detection, license plate detection, brake light detection, and lane detection.

19. A method of operating an image sensor coupled to a processor comprising a plurality of sub-modules, comprising:

capturing an image;

identifying regions of interest in the captured image, each region of interest corresponding to an image recognition detection task, each region of interest not-overlapping with each other region of interest;

for each region of interest, selecting a minimum pixel resolution, pixel exposure time, and pixel gain based on the corresponding image recognition detection task; and

adjusting pixel resolution, pixel exposure time, and pixel gain in two or more different portions of the image sensor corresponding to the regions of interest based on control signals from the plurality of sub-modules,

wherein at least two of the regions of interest have at least two set resolutions and at least one additional region of interest is non-overlapping with the at least two of the regions of interest and has a lower default resolution, and

wherein the captured image comprises;

an average pixel data rate that is lower than that of an image having full high resolution mode, and

an amount of pixel data which is less than that of the full high resolution mode image.

20. The method of claim 19, further comprising selecting a pixel resolution in image areas outside of regions of interest to have a reduced pixel resolution.

21. The method of claim 19, wherein said adjusting is performed iteratively over a sequence of images frames until an image recognition criterion is satisfied for each region of interest.