TWO-STAGE CORRELATION METHOD FOR CORRESPONDENCE SEARCH

• US 20110279652A1
• Filed: 05/06/2011
• Published: 11/17/2011
• Est. Priority Date: 05/14/2010
• Status: Active Grant

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: EINECKE, Nils, EGGERT, Julian
• Granted Patent: US 9,208,576 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

The invention presents a method for comparing the similarity between image patches comprising the steps of receiving form at least two sources at least two image patches, wherein each source supplies an image patch, comparing the received image patches by extracting a number of corresponding subpart pairs from each image patch, calculating a normalized local similarity score between all corresponding subpart pairs, calculating a total matching score by integrating the local similarity scores of all corresponding subpart pairs, and using the total matching score as an indicator for an image patch similarity, determining corresponding similar image patches based on the total matching score.

Claims

1. A computer-implemented method for comparing the similarity between two image patches taken from a stereo camera system, preferably in order to compute a depth estimation value for an object in the input field of a stereo camera, the method comprising the steps of:

receiving from at least two vision sensors, such as e.g. a stereo camera, at least two image patches, wherein each sensor supplies an image patch, and

comparing the received image patches by;

extracting a number of corresponding subpart pairs from each image patch,

calculating a normalized local similarity score between all corresponding subpart pairs,

calculating a total matching score by integrating the local similarity scores of all corresponding subpart pairs, and

using the total matching score as an indicator for an image patch similarity,

determining corresponding similar image patches based on the total matching score.

2. A computer-implemented method for comparing the similarity between image patches, the method comprising the steps of:

receiving from at least two image sources, such as e.g. a stereo camera, at least two image patches, wherein each source supplies at least one image patch, and

comparing the received image patches by;

extracting a number of corresponding subpart pairs from each image patch,

calculating a normalized local similarity score between all corresponding subpart pairs,

calculating a total matching score by integrating the local similarity scores of all corresponding subpart pairs, and

using the total matching score as an indicator for an image patch similarity.

3. The method according to claim 2, wherein the subparts of each image patch are overlapping.

4. The method according to claim 2,

wherein the normalized similarity score is calculated using normalized cross-correlation (NCC) and/or weighted NCC (using a weighted window), comprising the steps of

calculating mean and standard deviation for each subpart, and

calculating a normalized cross-correlation coefficient.

5. The method according to claim 2, wherein an image patch is compared to a list of other patches in order to find the most similar image patch in the list.

6. The method according to claim 2, wherein the at least two image sources are at least two cameras and the image patches from one camera image are compared to the patches from the other camera image in order to find corresponding image patches.

7. The method according to claim 2, wherein a disparity-based depth value is estimated using the calculated patch similarity.

8. The method according to claim 2, wherein the image patches are extracted from a camera or cameras mounted on a robot or a car.

9. The method according to claim 2, wherein

a normalized local similarity score is calculated by a NCC matching providing local matching measures, and wherein

calculating the total matching score uses the local measures and wherein the total matching score calculation suppresses noise that arises from the local calculation.

10. The method according to claim 2, wherein the total matching score is calculated from averaging, weighted averaging, winner-takes-all selection and/or a median selection of the local matching scores.

11. The method according to claim 2, wherein the image patches are arbitrary data vectors.

12. The method according to claim 2, wherein the image patches consist of discrete elements with numerical values.

13. The method according to claim 12, wherein the numerical values of the discrete elements represent gray-scaled pixel values.

14. The method according to claim 2, wherein the image patches consist of multiple layers and/or multiple dimensions.

15. The method according to claim 2, wherein the image patches are RGB image patches.

16. The method according to claim 2, wherein the at least two image sources are images from one camera at distinct points in time.

17. The method according to claim 16, wherein the image patches from one time are compared to the image patches of another point in time in order to find corresponding image patches.

18. The method according to claim 2, wherein the calculated patch similarity is used for image patch-based motion estimation.

19. The method according to claim 2, wherein one image source is a template image patch depicting an object, wherein the template image patch is loaded from a database and/or memory, and wherein the template image patch is compared to extracted image patches from the second and/or another image source in order to find an object in the image.

20. The method according to claim 2, wherein the method is embedded into a system with a temporal framework of repetitive prediction and confirmation as used in a tracking method, in order to track an object over time, thereby restricting the similarity computation to a selected set of patches gained using a prediction function that formalizes this selection.

21. The method according to claim 2, wherein the averaging and filtering operations are optimized, e.g. using separable filters and/or box filter methods.

22. A robot or land, air, sea or space vehicle equipped with a system, especially a depth estimation, motion estimation, object detection and/or object tracking system, performing the method according to claim 2, the system comprising at least one camera, in particular for depth estimation a stereo camera, and a computing unit.

23. A vehicle with a driver assistance system performing the method according to claim 2 in order to compute a depth estimation value for at least one object in the input field of a stereo camera of the vehicle.

24. A computer program product performing, when executed on a computer the method according to claim 2.