Stereo vision apparatus and method

• US 9,317,923 B2
• Filed: 04/05/2013
• Issued: 04/19/2016
• Est. Priority Date: 04/06/2012
• Status: Active Grant

• Current Assignee: Brigham Young University (The Church of Jesus Christ of Latter-Day Saints)
• Original Assignee: Brigham Young University (The Church of Jesus Christ of Latter-Day Saints)
• Inventors: Tippetts, Beau Jeffrey, Lee, Dah-Jye
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): SHIN, SOO JUNG
• Application Number:
  US13/857,868
• Publication Number:
  US 20130266211A1
• Time in Patent Office: 1,110 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A method for stereo vision may include filtering a row or column in a stereo image to obtain intensity profiles, identifying peaks in the intensity profiles, pairing peaks within a maximum disparity distance, determining a shape interval for peak pairs, selecting a peak pair with a maximum shape interval, determining a disparity offset for the peak pairs, extending shape intervals to include all pixels in the intensity profiles, computing depths or distances from disparity offsets, and smoothing the stereo image disparity map along a perpendicular dimension. Another method for stereo vision includes filtering stereo images to intensity profiles, identifying peaks in the intensity profiles, pairing peaks within a maximum disparity distance, determining shape intervals for peak pairs, and selecting peak pairs with the maximum shape interval. Apparatus corresponding to the above methods are also disclosed herein.

Claims

1. A method for efficiently extracting depth information from a stereo image, the method comprising a plurality of operations including:

providing a stereo image comprising a left and a right image of a scene;

filtering a row or column in the left and right images to provide a left and a right intensity profile for the row or column, the row or column corresponding to a first image dimension;

identifying peaks in the left and right intensity profiles to provide identified left peaks and identified right peaks;

pairing left and right peaks that are within a maximum disparity distance to provide peak pairs;

determining a shape interval for each peak pair by computing a difference between adjacent pixels along the first image dimension in a profile for the row or column to provide a profile gradient for the row or column, and advancing an end offset from each peak and retreating a start offset from each peak until a gradient difference between the left and right images at the start offset and the end offset exceeds a selected gradient difference threshold;

selecting a peak pair with a maximum shape interval for each left peak or each right peak to provide selected peak pairs; and

determining a disparity offset for the selected peak pairs.

2. The method of claim 1, further comprising extending the shape intervals for the selected peak pairs until all pixels within the row or column reside within a shape interval of a selected peak pair.

3. The method of claim 2, wherein the operations of filtering, identifying, pairing, determining a shape interval and selecting are conducted for each row or column of the stereo image.

4. The method of claim 3, further comprising computing disparity offsets for each row or column in the left and right image to provide a disparity map for the stereo image.

5. The method of claim 4, further comprising computing a depth or distance from a disparity offset.

6. The method of claim 4, further comprising smoothing the disparity offsets along a second image dimension that is perpendicular to the first image dimension to provide a smoothed disparity map.

7. The method of claim 6, wherein smoothing the disparity values comprises determining a most frequent disparity value within a disparity window.

8. The method of claim 6, wherein smoothing the disparity values comprises determining a median disparity value within a disparity window.

9. The method of claim 8, further comprising increasing the selected gradient difference threshold to extend the shape intervals.

10. An apparatus for efficiently extracting depth information from a stereo image, the apparatus comprising:

a memory configured to store a stereo image comprising a left and a right image of a scene;

an image smoothing filter configured to filter a row or column in the left and right images to provide a left and a right intensity profile for the row or column, the row or column corresponding to a first image dimension;

a peak finding module configured to identify peaks in the left and right intensity profiles to provide identified left peaks and identified right peaks;

a shape determination module configured to pair identified left and right peaks that are within a maximum disparity distance to provide peak pairs;

the shape determination module configured to determine a shape interval for each peak pair;

the shape determination module configured to select a peak pair with a maximum shape interval for each left peak or each right peak to provide selected peak pairs by computing a difference between adjacent pixels along the first image dimension in a profile for the row or column to provide a profile gradient for the row or column, and advancing an end offset from each peak and retreating a start offset from each peak until a gradient difference between the left and right images at the start offset and the end offset exceeds a selected gradient difference threshold; and

a disparity determination module configured to determine a disparity offset for each selected peak pair.

11. The apparatus of claim 10, wherein the shape determination module is further configured to extend the shape intervals for the selected peak pairs until all pixels within the row or column reside within a shape interval of a selected peak pair.

12. The apparatus of claim 11, wherein the image smoothing filter, peak finding module, shape determination module, and disparity determination module are configured to operate on each row or column of the stereo image.

13. The apparatus of claim 12, further comprising a disparity smoothing filter configured to smooth the disparity offsets along a second image dimension that is perpendicular to the first image dimension to provide a smoothed disparity map.

14. The apparatus of claim 13, wherein the disparity smoothing filter determines a most frequent disparity value within a disparity window.

15. The apparatus of claim 13, wherein the disparity smoothing filter determines a median disparity value within a disparity window.

16. A method for determining shapes within a stereo image, the method comprising a plurality of operations including:

providing a stereo image comprising a left and a right image of a scene containing a plurality of shapes;

filtering a row or column in the left and right images to provide a left and a right intensity profile for the row or column, the row or column corresponding to a first image dimension;

identifying peaks in the left and right intensity profiles to provide identified left peaks and identified right peaks;

pairing left and right peaks that are within a maximum disparity distance to provide peak pairs, each peak pair corresponding to a shape in the plurality of shapes in the stereo image;

determining a shape interval for each peak pair by advancing an end offset from each peak and retreating a start offset from each peak until a gradient difference between the left and right images exceeds a selected gradient difference threshold; and

selecting a peak pair with a maximum shape interval for each left peak or each right peak to provide detected shape intervals for the plurality of shapes in the stereo image.

17. An apparatus for determining shapes within a stereo image, the apparatus comprising:

a memory configured to store a stereo image comprising a left and a right image of a scene containing a plurality of shapes;

an image smoothing filter configured to filter a row or column in the left and right images to provide a left and a right intensity profile for the row or column, the row or column corresponding to a first image dimension;

a peak finding module configured to identify peaks in the left and right intensity profiles to provide identified left peaks and identified right peaks;

a shape determination module configured to pair identified left and right peaks that are within a maximum disparity distance to provide peak pairs, each peak pair corresponding to a shape in the plurality of shapes in the stereo image; and

the shape determination module configured to determine a shape interval for each peak pair by advancing an end offset from each peak and retreating a start offset from each peak until a gradient difference between the left and right images exceeds a selected gradient difference threshold; and

the shape determination module configured to select a peak pair with a maximum shape interval for each left peak or each right peak to provide detected shape intervals for the plurality of shapes in the stereo image.