Hardware coprocessor for stripe-based interest point detection

• US 9,020,276 B2
• Filed: 04/24/2013
• Issued: 04/28/2015
• Est. Priority Date: 04/24/2012
• Status: Expired due to Fees

• Current Assignee: STMicroelectronics International N.V. (STMicroelectronics NV)
• Original Assignee: STMicroelectronics SRL (STMicroelectronics NV)
• Inventors: Vigliar, Mario, Licciardo, Gian Domenico
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US13/869,609
• Publication Number:
  US 20130301930A1
• Time in Patent Office: 734 Days
• Field of Search: 382/159, 382/195, 382/170, 382/224, 382/190
• US Class Current: 382/201
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

A hardware coprocessor architecture calculates the Difference-of-Gaussian (DoG) pyramid of an input image and extracts from this the interest points to be used in several image detection algorithms. Advantages of the architecture include the possibility to process the image by stripes, namely by blocks having one dimension coincident with the input image width, in the absence of an input frame buffer and the possibility to avoid RAM memory. The coprocessor is suitable to be tightly coupled with raw image sources like sensors.

Claims

1. An image processor, comprising:

a first stage configured to receive pixels from an image source, the first stage including a plurality of octave elements, each octave element including an octave module and a synchronization buffer to provide pixels to the octave module, wherein the octave module is configured to generate several Difference-of-Gaussian images; and

a second stage including interest point detectors configured to receive the Difference-of-Gaussian images from respective octave modules and to generate interest point values and positions, wherein;

each octave module comprises scale sub-modules that receive pixel data from the synchronization buffer and Difference-of-Gaussian subtractors which receive blurred image data from adjacent scale sub-modules, and

each scale sub-module includes a first Gaussian filter, a second Gaussian filter and a stripe buffer coupled between the first and second Gaussian filters.

2. An image processor as defined in claim 1, wherein the synchronization buffer of the first octave element receives input pixel data from an image sensor and wherein the synchronization buffer of each octave element following the first octave element receives pixel data from a previous octave module.

3. An image processor as defined in claim 1 wherein the first Gaussian filter comprises a plurality of multiplier accumulators configured to multiply pixel data by Gaussian coefficients, and an adder tree configured to add outputs of the multiplier accumulators and to provide a result to the stripe buffer.

4. An image processor as defined in claim 1 wherein the second Gaussian filter comprises parallel-in serial-out buffers configured to receive pixel data from the stripe buffer, multiplier accumulators configured to multiply outputs of the parallel-in serial-out buffers by Gaussian coefficients, and an adder tree configured to sum the outputs of the multiplier accumulators and to provide a Gaussian result.

5. An image processor as defined in claim 1 wherein the stripe buffer includes a multiplexer configured to receive outputs of rows of the stripe buffer and to provide an output to the second Gaussian filter.

6. An image processor as defined in claim 1 wherein the stripe buffer comprises a dual port Random Access Memory.

7. An image processor as defined in claim 1 wherein each synchronization buffer includes delays configured to compensate for different processing times of the scale sub-modules.

8. An image processor as defined in claim 1, wherein the interest point detectors include comparators configured to compare pixel data corresponding to adjacent pixels and delay elements configured to delay pixel data for comparison.

9. An image processor as defined in claim 8, wherein the second stage further includes interest point position circuits configured to provide interest point position data.

10. A method, comprising:

receiving, by a first stage of an image processor, pixel data from an image source, the first stage including a plurality of octave elements, each octave element including an octave module and a synchronization buffer to provide pixels to the octave module, wherein the octave module is configured to generate several Difference-of-Gaussian images; and

processing, by a second stage of the image processor including interest point detectors, the Difference-of-Gaussian images from respective octave elements, the processing including generating interest point values and positions, wherein

each octave module comprises scale sub-modules that receive pixel data from the synchronization buffer and Difference-of-Gaussian subtractors which receive blurred image data from adjacent scale sub-modules, and

each scale sub-module includes a first Gaussian filter, a second Gaussian filter and a stripe buffer coupled between the first and second Gaussian filters.

11. The method of claim 10, comprising:

compensating for different processing times of the scale sub-modules.

12. The method of claim 10, comprising:

comparing pixel data corresponding to adjacent pixels and delaying pixel data for comparison.

13. A system, comprising:

a plurality of octave elements configured to receive image pixels from an image source, the first stage including a plurality of octave elements, each octave element including an octave module and a synchronization buffer to provide pixels to the octave module, wherein the octave module is configured to generate several Difference-of-Gaussian images; and

a plurality of interest point detectors configured to receive the Difference-of-Gaussian images from respective octave modules and to generate interest point values and positions, wherein;

each octave module comprises scale sub-modules configured to receive pixel data from the synchronization buffer and Difference-of-Gaussian subtractors configured to receive blurred image data from adjacent scale sub-modules, and

each scale sub-module includes a first Gaussian filter, a second Gaussian filter and a stripe buffer coupled between the first and second Gaussian filters.

14. The system of claim 13, comprising:

an image sensor, wherein the synchronization buffer of the first octave element receives input pixel data from the image sensor and the synchronization buffer of each octave element following the first octave element receives pixel data from a previous octave module.

15. The system of claim 13, comprising:

an image processor including the plurality of octave elements and the plurality of interest point detectors; and

an image sensor.

16. The system of claim 13 wherein the first Gaussian filter comprises a plurality of multiplier accumulators configured to multiply pixel data by Gaussian coefficients, and an adder tree configured to add outputs of the multiplier accumulators and to provide a result to the stripe buffer.

17. The system of claim 13 wherein the second Gaussian filter comprises parallel-in serial-out buffers configured to receive pixel data from the stripe buffer, multiplier accumulators configured to multiply outputs of the parallel-in serial-out buffers by Gaussian coefficients, and an adder tree configured to sum the outputs of the multiplier accumulators and to provide a Gaussian result.

18. The system of claim 13 wherein the stripe buffer includes a multiplexer configured to receive outputs of rows of the stripe buffer and to provide an output to the second Gaussian filter.

19. The system of claim 13 wherein the stripe buffer comprises a dual port Random Access Memory.

20. The system of claim 13 wherein each synchronization buffer is configured to compensate for different processing times of the scale sub-modules.