Image feature identification and motion compensation apparatus, systems, and methods

• US 7,599,566 B2
• Filed: 06/06/2007
• Issued: 10/06/2009
• Est. Priority Date: 03/14/2007
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Lin, Sheng, Li, Dong, Silverstein, Amnon
• Primary Examiner(s): Couso; Jose L
• Application Number:
  US11/758,815
• Publication Number:
  US 20080226184A1
• Time in Patent Office: 853 Days
• Field of Search: 382/100, 382103-104, 382106-107, 382/173, 382/181, 382/190, 382/197, 382/199, 382/203, 382/209, 382224-228, 382/276, 348/113, 348/116, 348/118, 348/148, 701/26, 701/28, 701/58, 701/65, 701/200, 701300-302, 356/3, 342/29, 342/41, 342/195, 342/350, 342/357.08
• US Class Current: 382/238
• CPC Class Codes: H04N 23/73 by influencing the exposure time

Abstract

Apparatus, systems, and methods disclosed herein may estimate the magnitude of relative motion between a scene and an image capture device used to capture the scene. Some embodiments may utilize discrete cosine transform and/or Sobel gradient techniques to identify one or more blocks of pixels in an originating calibration image frame. Matching blocks of pixels may be located in a successive calibration image frame. Motion vectors originating at one calibration frame and terminating at the other calibration frame may be calculated. The magnitude of relative motion derived thereby may be used to adjust image capture parameters associated with the image capture device, including exposure settings.

Claims

1. Circuitry, comprising:

a gradient vector calculator circuit that calculates a gradient vector for each pixel within a block of pixels excluding pixels located along a periphery of the block of pixels to provide a plurality of gradient vectors; and

gradient vector logic circuitry operatively coupled to the gradient vector calculator circuit to perform a gradient vector analysis on the plurality of gradient vectors to derive a value of an image feature strength metric associated with the block of pixels.

2. The circuitry of claim 1, wherein the gradient vector calculator circuit is configured to perform Sobel operations on the each pixel within the block of pixels to obtain the gradient vector.

3. The circuitry of claim 1, further comprising:

X-Sobel logic circuitry coupled to the gradient vector calculator circuit to mathematically convolve a matrix of pixel luminance values comprising a subject pixel and eight pixels immediately adjacent to the subject pixel by an X-Sobel matrix to obtain an X-Sobel value.

4. The circuitry of claim 3, further comprising:

Y-Sobel logic circuitry coupled to the gradient vector calculator circuit to convolve the matrix of pixel luminance values comprising the subject pixel and the eight pixels immediately adjacent to the subject pixel by a Y-Sobel matrix to obtain a Y-Sobel value.

5. The circuitry of claim 4, further comprising:

pythagorean logic circuitry coupled to the gradient vector calculator circuit to perform a pythagorean operation on the X-Sobel value and the Y-Sobel value to obtain a value of the gradient vector associated with the subject pixel; and

arctangent logic circuitry coupled to the gradient vector calculator circuit to perform an arctangent operation on the X-Sobel value and the Y-Sobel value to obtain an angle associated with the gradient vector.

6. The circuitry of claim 1, further comprising:

a gradient vector buffer circuitry coupled to the gradient vector calculator circuit to store the gradient vector for each pixel within the block of pixels excluding the pixels located along the periphery of the block of pixels.

7. The circuitry of claim 1, further comprising:

gradient vector threshold logic circuitry coupled to the gradient vector logic circuitry to discard ones of the plurality of gradient vectors smaller than a threshold magnitude.

8. The circuitry of claim 7, further comprising:

gradient vector rotational complement logic circuitry coupled to the gradient vector logic circuitry to rotate by 180 degrees remaining ones of the plurality of gradient vectors falling below an X axis.

9. The circuitry of claim 8, further comprising:

a gradient vector quantizer circuit coupled to the gradient vector logic circuitry to quantize the remaining ones of the plurality of gradient vectors into bins comprising ranges of vector angles.

10. The circuitry of claim 9, further comprising:

a base gradient vector filter circuit coupled to the gradient vector logic circuitry to discard remaining ones of the plurality of gradient vectors from a bin containing more of the remaining ones of the plurality of gradient vectors than any other bin and from bins immediately adjacent to the bin containing more of the remaining ones of the plurality of gradient vectors than any other bin to leave a set of final gradient vectors in at least one bin.

11. The circuitry of claim 10, further comprising:

a remaining gradient vector summation module circuit coupled to the gradient vector logic circuitry to count a number gradient vectors in the set of final gradient vectors to obtain the value of the image feature strength metric associated with the block of pixels.

12. A system, comprising:

an image sensor array (ISA) to detect luminance incident to a set of ISA elements and to transform the luminance to create a set of luminance values associated with a pixel representation of an image;

a gradient vector calculator coupled to the ISA to calculate a gradient vector for each pixel within a block of pixels excluding pixels located along a periphery of the block of pixels to provide a plurality of gradient vectors; and

gradient vector logic operatively coupled to the gradient vector calculator to perform a gradient vector analysis on the plurality of gradient vectors to derive a value of an image feature strength metric associated with the block of pixels.

13. The system of claim 12, incorporated into a digital camera.

14. The system of claim 12, incorporated into a digital imaging software product.

15. The system of claim 12, incorporated into a vehicle collision avoidance system.

16. The system of claim 12, further including:

target matching logic operatively coupled to the gradient vector logic to correlate a first image feature block of pixels as selected from a first image of a scene to a second image feature block of pixels as searched for in a second image of the scene, wherein the second image of the scene is captured after capturing the first image of the scene.

17. The system of claim 12, further comprising:

motion priority logic operatively coupled to the gradient vector logic to estimate a magnitude of relative motion between a scene and an image capture device used to capture the scene, the relative motion occurring during a period of time between capturing a first image of the scene and capturing a second image of the scene.

18. The system of claim 17, further comprising:

motion vector logic coupled to the motion priority logic to calculate a magnitude of a motion vector associated with a first image feature block of pixels selected from the first image of the scene, the magnitude calculated as a distance between the first image feature block of pixels and a second image feature block of pixels as found in the second image of the scene, the distance divided by the period of time.

19. A method for using processing circuitry, comprising:

with the processing circuitry, performing a gradient vector analysis on a block of pixels selected from an image that is detected by an image sensor to derive a value of an image feature strength metric associated with the block of pixels; and

with the processing circuitry, calculating a gradient vector for each pixel within the block of pixels excluding pixels located along a periphery of the block of pixels to provide a plurality of gradient vectors.

20. The method of claim 19, further comprising:

with the processing circuitry, discarding ones of the plurality of gradient vectors smaller than a threshold magnitude.

21. The method of claim 20, further comprising:

with the processing circuitry, rotating by 180 degrees remaining ones of the plurality of gradient vectors falling below an X axis; and

with the processing circuitry, quantizing the remaining ones of the plurality of gradient vectors into bins comprising ranges of vector angles.

22. The method of claim 20, further comprising:

with the processing circuitry, discarding all of the remaining ones of the plurality of gradient vectors from a bin containing more of the remaining ones of the plurality of gradient vectors than any other bin and from bins immediately adjacent to the bin containing more of the remaining ones of the plurality of gradient vectors than any other bin to leave a set of final gradient vectors in at least one bin; and

with the processing circuitry, counting a number gradient vectors in the set of final gradient vectors to obtain the value of the image feature strength metric associated with the block of pixels.

23. The method of claim 19, wherein the gradient vector is calculated according to a Sobel technique.

24. The method of claim 23, further comprising:

with the processing circuitry, performing an X-Sobel operation on a subject pixel luminance matrix to obtain an X-Sobel value;

with the processing circuitry, performing a Y-Sobel operation on the subject pixel luminance matrix to obtain a Y-Sobel value;

with the processing circuitry, performing a Pythagorean operation on the X-Sobel value and the Y-Sobel value to obtain a value of a gradient vector associated with the subject pixel; and

with the processing circuitry, performing an arctangent operation on the X-Sobel value and the Y-Sobel value to obtain a direction of the gradient vector associated with the subject pixel.

25. The method of claim 24, further comprising:

with the processing circuitry, configuring the subject pixel luminance matrix to correspond to pixel luminance values associated with a subject pixel and eight pixels immediately adjacent to the subject pixel in a 3×

3 subject pixel matrix.

26. The method of claim 24, wherein the X-Sobel operation comprises:

with the processing circuitry, convolving the subject pixel luminance matrix by an X-Sobel matrix to obtain the X-Sobel value.

27. The method of claim 24, wherein the Y-Sobel operation comprises:

with the processing circuitry, convolving the subject pixel luminance matrix by a Y-Sobel matrix to obtain the Y-Sobel value.