Motion estimation using intrapixel logic

• US 6,366,317 B1
• Filed: 03/27/1998
• Issued: 04/02/2002
• Est. Priority Date: 03/27/1998
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Fink, Michael J., Morris, Tonia G., Mattison, Phillip E.
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): Vo, Tung T.
• Application Number:
  US09/049,599
• Time in Patent Office: 1,467 Days
• Field of Search: H04/N.964, H04/N.5217, H01/L 2.7148, 348/390, 348/392, 348/240, 348/241, 348/242, 348/243, 375/240, 250/208.1, 250/557, 250/214
• US Class Current: 348/241
• CPC Class Codes: H01L 27/14643 Photodiode arrays; MOS imagers
• CPC Class Codes: H04N 25/63 applied to dark current
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An image sensor array for capturing a sequence of video frames has a plurality of pixels, wherein each one of the pixels has a set of intrapixel logic including a previous register to store a pixel value of a previous frame and a current register to store a pixel value of a current frame, the current registers corresponding to the previous registers. An intrapixel subtractor coupled to the previous and current registers produces a difference between the previous and current registers for all pixels in the video frames in parallel. The differences are accumulated for each block of the image sensor array to form a total divergence for each block. The total divergences, associated motion vectors, and pixel values are used in motion estimation processing by a processor coupled to the image sensor array.

Claims

1. An image sensor array comprising:

a plurality of pixels, each one of the plurality of pixels comprising

a first register to store a first pixel value of a first frame;

a second register to store a second pixel value of a second frame, the second register corresponding to the first register;

a subtractor coupled to the first and second registers to produce a difference between the first pixel value and the second pixel value; and

a third register to store the difference between the first and second registers;

wherein the plurality of pixels are divided into a plurality of N pixel by M pixel blocks, each one of the blocks having N times M differences, each block comprising a fourth register and circuitry for accumulating the N times M differences of each block as a total divergence value into the fourth register.

2. The image sensor array of claim 1, wherein each one of the first registers are coupled with first registers of adjacent pixels to form a two-dimensional network allowing shifting of first pixel values in the left, right, up, and down directions from one first register to another, adjacent first register.

3. The image sensor array of claim 1, wherein the subtractor produces an absolute difference between the first pixel value and the second pixel value.

4. The image sensor array of claim 1, wherein the subtractors of the plurality of pixels operate to produce differences between all pairs of corresponding first and second registers simultaneously in parallel.

5. The image sensor array of claim 1, wherein the circuitry comprises a plurality of accumulators.

6. The image sensor array of claim 1, wherein the first frame is shifted relative to the second frame to create a different correspondence between the first registers and the second registers and wherein each block comprises a register for storing the total divergence value best matching predetermined criteria for the block for all frame shifts.

7. The image sensor array of claim 1, wherein each block comprises motion vector registers for storing motion vector coordinates.

8. The image sensor array of claim 1, wherein the fourth registers are located on edges of the image sensor array.

9. The image sensor array of claim 1, wherein each one of the plurality of pixels comprises a fifth register coupled to the third register to store the difference for the pixel when the fourth register holds the total divergence value matching predetermined criteria for a block.

10. The image sensor array of claim 1, further comprising at least one decoder coupled to the output of the subtractors.

11. The image sensor array of claim 1, wherein each one of the plurality of pixels further comprises a dark register coupled to the subtractor for storing a dark pixel value.

12. The image sensor array of claim 1, wherein each one of the plurality of pixels comprises a photocell.

13. The image sensor array of claim 1, wherein the image sensor array is a CMOS array.

14. A video camera comprising:

an image sensor array having a plurality of pixels, each one of the plurality of pixels comprising a first register to store a first pixel value of a first frame;

a second register to store a second pixel value of a second frame, the second register corresponding to the first register;

a subtractor coupled to the first and second registers to produce a difference between the first pixel value and the second pixel value; and

a third register to store the difference between the first and second registers;

wherein the plurality of pixels are divided into a plurality of N pixel by M pixel blocks, each one of the blocks having N times M differences, each block comprising a fourth register and circuitry for accumulating the N times M differences of each block as a total divergence value into the fourth register.

15. An image sensor array for supporting motion estimation processing comprising:

a plurality of pixels, each one of the plurality of pixels comprising;

first means for storing a first pixel value of a first frame;

second means for storing a second pixel value of a second frame, the second storing means corresponding to the first storing means;

means coupled to the first and second storage means for subtracting the first pixel value from the second pixel value to produce a difference between the first pixel value and the second pixel value; and

third means for storing the difference between the first and second storing means;

wherein the plurality of pixels are divided into a plurality of N pixel by M pixel blocks, each one of the blocks having N times M differences, each block comprising a fourth means for accumulating the N times M differences of each block as a total divergence value.

16. The image sensor array of claim 15, wherein each one of the first storing means are coupled with first storing means of adjacent pixels to form a two-dimensional network allowing shifting of first pixel values in the left, right, up, and down directions from one first storing means to another, adjacent first storing means.

17. The image sensor array of claim 15, wherein the subtracting means of the plurality of pixels operate to produce differences between all pairs of corresponding first and second storing means simultaneously in parallel.

18. The image sensor array of claim 15, wherein each one of the plurality of pixels further comprises a third means for storing the difference between the first and second storing means.

19. The image sensor array of claim 15, wherein each one of the plurality of pixels comprises a fifth storing means coupled to the third storing means for storing the difference for the pixel when the fourth storing means stores the total divergence value matching predetermined criteria for a block.

20. The image sensor array of claim 15, wherein the first frame is shifted relative to the second frame to create a different correspondence between the first storage means and the second storage means and each block comprises means for storing the total divergence value best matching predetermined criteria for the block for all frame shifts.

21. The image sensor array of claim 15, wherein each block comprises means for storing motion vector coordinates.

22. A pixel of an image sensor array, the image sensor array having a plurality of pixels, said pixel comprising:

a first register to store a first pixel value of a first frame;

a second register to store a second pixel value of a second frame, the second register corresponding to the first register;

a subtractor coupled to the first and second registers to produce a difference between the first pixel value and the second pixel value;

a third register to store the difference between the first and second registers; and

a fourth register coupled to the subtractor and the second register to store a dark pixel value.

23. The pixel of claim 22, wherein the subtractor produces an absolute difference between the first pixel value and the second pixel value.

24. The pixel of claim 22, further comprising a fifth register coupled to the third register to store a best difference for the pixel according to predetermined criteria.

25. A method of supporting motion estimation processing in a CMOS image sensor array of a video camera, the CMOS image sensor array having a plurality of pixels wherein the plurality of pixels are divided into a plurality of N pixel by M pixel blocks, comprising:

capturing a first frame and storing pixel values of the first frame in a plurality of first registers, each first register storing a value of one pixel of the CMOS image sensor array within the pixel;

capturing a second frame and storing pixel values of the second frame in a plurality of second registers, each second register storing a value of one pixel of the CMOS image sensor array within the pixel, the second registers corresponding to the first registers;

subtracting the first registers from the second registers in the pixel in parallel to produce a plurality of difference values between the first frame and the second frame;

storing the difference values in third registers, each third register in a pixel for storing the difference in values between the first frame and the second frame for the pixel;

shifting and accumulating the difference values to obtain a total divergence for each block of the CMOS image sensor array; and

comparing the total divergence for each block with a previously stored best match for each block.

26. The method of claim 25, further comprising storing a current total divergence for each block as a best match for each block when the current total divergence is better than the previously stored best match according to predetermined criteria.

27. The method of claim 26, further comprising shifting the first frame relative to the second frame.