Method and apparatus for ground detection and removal in vision systems

• US 7,957,562 B2
• Filed: 05/08/2006
• Issued: 06/07/2011
• Est. Priority Date: 06/13/2003
• Status: Active Grant

• Current Assignee: Ford Motor Company, SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Chang, Peng, Hirvonen, David, Camus, Theodore Armand
• Primary Examiner(s): Mariam, Daniel G
• Assistant Examiner(s): Bitar, Nancy
• Application Number:
  US11/429,865
• Publication Number:
  US 20060210117A1
• Time in Patent Office: 1,856 Days
• Field of Search: 382/103, 382/106, 382/107, 382/105, 382/113, 382/291, 382/285, 382/154, 356/3, 356/3.02, 356/3.08, 356/3.11, 356/3.16, 356/4.06, 356/611, 340/854.1, 340/435, 340/686.6
• US Class Current: 382/113
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vision system that forms a map of a scene proximate a platform, e.g., a vehicle, that determines the actual ground plane form the map, and that corrects the map for differences between the actual ground plane and an assumed ground plane. The vision system may remove the actual ground plane from the map to prevent false positives. The vision system can further identify and classify objects and, if appropriate, take evasive action.

Claims

1. A method of performing vision processing comprising:

producing a depth map of a scene proximate a platform, wherein the depth map is based on an assumed ground plane;

identifying an actual ground plane using the depth map; and

compensating the depth map for differences between the assumed ground plane and the actual ground plane thereby resulting in a compensated depth map including a collection of 3-D points; and

comparing the depth map to a plurality of templates to identify a match between the depth map and at least one template, wherein the comparing step includes determining a difference between each pixel in the depth map and each similarly positioned pixel in a template, and determining that a pixel is a match if the difference at each pixel is less than a predefined amount.

2. The method of claim 1 further including the step of adjusting a parameter of the platform in response to a match.

3. The method of claim 1 wherein a parameter of the vehicle comprises at least one parameter selected from the group of:

an air bag deployment parameter, a seatbelt parameter, a vehicle height parameter, and a vehicle velocity and/or acceleration control parameter.

4. The method of claim 1 further comprising:

accessing at least one template from a database comprising a plurality of templates.

5. The method of claim 4 wherein the plurality of templates represent objects at varying positions and poses relative to the platform.

6. The method of claim 4 wherein at least one template in the plurality of templates is a model of a vehicle.

7. The method of claim 4 wherein at least one template in the plurality of templates is a model of a pedestrian.

16. The method of claim 1 further comprising:

removing the actual ground plane from the depth map.

8. A collision avoidance system comprising:

a collision detection system comprising;

a stereo camera pair for producing imagery of a scene;

a stereo image preprocessor for preprocessing said imagery;

a map generator for producing from said preprocessed imagery a map referenced to an assumed ground plane;

a target processor for determining an actual ground plane from said map and for compensating the depth map for differences between the assumed ground plane and the actual ground plane, thereby resulting in a compensated depth map including a collection of 3-D points, said target processor for comparing the depth map to a plurality of templates to identify a match between the depth map and at least one template, wherein said comparing includes determining a difference between each pixel in the depth map and each similarly positioned pixel in a template, and determining that a pixel is a match if the difference at each pixel is less than a predefined amount.

9. A system according to claim 8, wherein the collision detection system further includes a collision detector for determining if a collision is imminent;

wherein said collision detector detects a potential threat in said map;

wherein said collision detector estimates size, position, and velocity of said detected potential threat;

wherein said collision detector performs a trajectory analysis of said detected potential threat using said estimated position and said estimated velocity;

wherein said collision detector predicts a collision based on said trajectory analysis; and

wherein said collision detector determines if a collision is imminent based on said collision prediction and on said estimated size.

10. The system according to claim 9 further including a secondary sensor that provides said collision detector with information regarding the scene.

17. The collision avoidance system of claim 8 wherein the target processor removes the actual ground plane from the map.

11. A method of performing vision processing comprising:

producing a depth map of original stereo images of a scene proximate a platform, wherein the depth map is based on an assumed ground plane;

identifying an actual ground plane using the depth map; and

correcting the original stereo images for differences between the assumed ground plane and the actual ground plane thereby resulting in a corrected depth map including a collection of 3-D points; and

comparing the depth map to a plurality of templates to identifying a match between the depth map and at least one template, wherein the comparing step includes determining a difference between each pixel in the depth map and each similarly positioned pixel in a template, and determining that a pixel is a match if the difference at each pixel is less than a predefined amount.

12. The method of claim 11 wherein the technique of correction is a vertical translation of the original stereo images.

13. The method of claim 12 wherein the corrected stereo images are processed to refine the estimated height and width of a target.

14. A collision avoidance system comprising:

a platform;

a collision detection system on said platform, said collision detection system comprising;

a stereo camera pair for producing imagery of a scene;

a stereo image preprocessor for preprocessing said imagery;

a map generator for producing from said preprocessed imagery a map referenced to an assumed ground plane; and

a target processor for determining the actual ground plane from said map and for correcting the original stereo images based on the actual ground plane thereby resulting in a corrected depth map including a collection of 3-D points, said target processor for comparing the depth map to a plurality of templates to identify a match between the depth map and at least one template, wherein said comparing includes determining a difference between each pixel in the depth map and each similarly positioned pixel in a template, and determining that a pixel is a match if the difference at each pixel is less than a predefined amount.

15. A non-transitory computer readable medium having stored thereon a plurality of instructions on one or more storage devices, the plurality of instructions including instructions, which, when executed by at least one processor, cause the processor to perform the steps of a method for performing vision processing, said method comprising the steps of:

producing a depth map from input imagery;

determining an actual ground plane from said depth map; and

correcting the original stereo images based on the actual ground plane thereby resulting in a corrected depth map including a collection of 3-D points; and

comparing the depth map to a plurality of templates to identifying a match between the depth map and at least one template, wherein the comparing step includes determining a difference between each pixel in the depth map and each similarly positioned pixel in a template, and determining that a pixel is a match if the difference at each pixel is less than a predefined amount.