SPATIAL-TEMPORAL MULTI-RESOLUTION IMAGE SENSOR WITH ADAPTIVE FRAME RATES FOR TRACKING MOVEMENT IN A REGION OF INTEREST

• US 20090066782A1
• Filed: 09/07/2007
• Published: 03/12/2009
• Est. Priority Date: 09/07/2007
• Status: Active Grant

• Current Assignee: Regents of The University of Minnesota (University of Minnesota)
• Original Assignee: Regents of The University of Minnesota (University of Minnesota)
• Inventors: Choi, Jaehyuk, Yoon, Euisik
• Granted Patent: US 8,089,522 B2
• Time in Patent Office: Days
• US Class Current: 348/25
• CPC Class Codes: H04N 25/42 by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode
• CPC Class Codes: H04N 25/44 by partially reading an SSIS array
• CPC Class Codes: H04N 25/443 by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 25/47 Image sensors with pixel address output; Event-driven image sensors; Selection of pixels to be read out based on image data
• CPC Class Codes: H04N 25/707 Pixels for event detection
• CPC Class Codes: H04N 25/778 comprising amplifiers shared between a plurality of pixels, i.e. at least one part of the amplifier must be on the sensor array itself
• CPC Class Codes: H04N 3/155 Control of the image-sensor operation, e.g. image processing within the image-sensor

Abstract

A sensor includes an array of pixels to capture an image, the array of pixels arranged as a plurality of pixel groups, each of the pixel groups having a two or more pixels coupled to a shared floating diffusion node for outputting merged image signals from the pixel groups, at least one inter-pixel switch to control transfer of electrical charge from a floating diffusion node for a first one of the pixel groups to a floating diffusion node for a second one of the pixel groups to temporarily store a portion of a previous image frame within the floating diffusion node for the second one of the pixel groups, and a motion comparator to compare an image signal from the first one of the pixel groups with an image signal from the second one of the pixel groups to detect motion between a current frame and the previous frame.

Claims

1. An image sensor comprising:

an array of pixels to capture an image, the array of pixels arranged as a plurality of pixel groups, each of the pixel groups having a two or more pixels coupled to a shared floating diffusion node for outputting merged image signals from the pixel groups;

at least one inter-pixel switch to control transfer of electrical charge from a floating diffusion node for a first one of the pixel groups to a floating diffusion node for a second one of the pixel groups to temporarily store a portion of a previous image frame within the floating diffusion node for the second one of the pixel groups; and

a motion comparator to compare an image signal from the first one of the pixel groups with an image signal from the second one of the pixel groups to detect motion between a current frame and the previous frame.

2. The sensor of claim 1, further comprising:

a first data channel that outputs image data for a background region of the array of pixels; and

a second data channel that outputs image data for a region-of-interest (ROI) of the array of pixels,

wherein the sensor simultaneously generates image readouts from the first channel and the second channel,

wherein the first channel outputs the image data at a first image resolution and a first frame rate, and

wherein the second channel outputs the image data at a second image resolution lower than the first image resolution and at a second frame rate faster than the first frame rate.

3. The sensor of claim 2, further comprising:

a column selector that enables independent pixel control for the ROI of the pixel array and the background region of the pixel array;

a first row decoder that selects an even row within the pixel array;

a second row decoder that selects an odd row within the pixel array;

a top correlated double sampling/analog-to-digital converter (CDS/ADC) block that samples signals from even columns within the pixel array; and

a bottom CDS/ADC block that samples signals from odd columns within the pixel array.

4. The sensor of claim 2, wherein the floating diffusion node for each pixel group merges the plurality of pixels of the group by electrically connecting the pixels to sum charges of each of the pixels such that the sensor generates the image data of the second channel at the second image resolution.

5. The sensor of claim 1, wherein the motion comparator generates a motion information signal that indicates detected motion based on the comparison when a difference between the previous frame and the current frame is above a threshold level.

6. The sensor of claim 1, further comprising a bootstrapping block that generates a control voltage to compensate a threshold voltage drop in a signal path when the sensor enables the inter-pixel switch.

7. The sensor of claim 1, wherein the sensor is a complementary metal-oxide-semiconductor (CMOS) sensor.

8. A video capture device comprising:

a sensor comprising;

an array of pixels to capture an image, the array of pixels arranged as a plurality of pixel groups, each of the pixel groups having a two or more pixels coupled to a shared floating diffusion node for outputting merged image signals from the pixel groups,

at least one inter-pixel switch to control transfer of electrical charge from a floating diffusion node for a first one of the pixel groups to a floating diffusion node for a second one of the pixel groups to temporarily store a portion of a previous image frame within the floating diffusion node for the second one of the pixel groups, and

a motion comparator to output motion information signals based on a comparison of the image signal from the first one of the pixel groups with the image signal from the second one of the pixel groups motion between a current frame and the previous frame; and

a region-of-interest (ROI) decision unit to receive the motion information signals from the motion comparator and output ROI address information to the sensor based on the motion information signals.

9. The video capture device of claim 8, further comprising:

a first data channel that outputs image data for a background region of the array of pixels;

a second data channel that outputs image data for a region-of-interest (ROI) of the array of pixels,

wherein the sensor simultaneously generates image readouts from the first channel and the second channel,

wherein the first channel outputs the image data at a first image resolution and a first frame rate, and

wherein the second channel outputs the image data at a second image resolution lower than the first image resolution and at a second frame rate faster than the first frame rate; and

a signal processing unit that combines the image data from the first data channel with the image data from the second data channel.

10. The video capture device of claim 8, further comprising a plurality of sensors.

11. The video capture device of claim 8, further comprising a display unit to display a merged video output received from the signal processing unit.

12. The video capture device of claim 8, further comprising a video storage unit to store a merged video output received from the signal processing unit.

13. A method comprising:

generating, within an image sensor having an array of pixels, pixelized data from sensed light, wherein the array of pixels are arranged as a plurality of pixel groups, each of the pixel groups having a two or more pixels coupled to a shared floating diffusion node for outputting merged image signals from the pixel groups;

temporarily storing a previous frame signal within a shared floating diffusion node of at least one of the pixel groups, wherein the shared floating diffusion node operates as an analog temporary memory to temporarily store the previous frame signal within the image sensor;

comparing, within the image sensor, the stored previous frame signal to a current frame signal to identify motion.

14. The method of claim 13, further comprising:

identifying a region-of-interest (ROI) of the array of pixels based on the comparison;

simultaneously outputting image data at a first image resolution and a first frame rate from a first data channel for a background region of the array of pixels and outputting image data at a second image resolution lower than the first image resolution and at a second frame rate faster than the first frame rate from a second data channel for the ROI.

15. The method of claim 14, further comprising combining the image data from the first data channel with the image data from the second data channel.

16. The method of claim 13, further comprising merging the plurality of pixels within each of the pixel groups by electrically connecting the pixels with the shared floating diffusion node of the pixel group to sum charges of each of the pixels such that the sensor generates the image readout from the second channel at the second image resolution.

17. The method of claim 13, further comprising generating a motion information signal that indicates detected motion based on the comparison when a difference between the previous frame signal and the current frame signal is above a threshold level.

18. The method of claim 17, further comprising determining an address of a region-of-interest (ROI) based on the motion information signal.

19. The method of claim 13, further comprising connecting a first shared floating diffusion node for a first one of the pixel groups to a second shared floating diffusion node for a second one of the pixel groups with an inter-pixel switch to transfer electrical charge from the first floating diffusion node to the second floating diffusion node.