Obstacle detection system

• US 6,678,394 B1
• Filed: 11/30/1999
• Issued: 01/13/2004
• Est. Priority Date: 11/30/1999
• Status: Expired due to Term

• Current Assignee: Cognex Technology & Investment Corporation (Cognex Corporation)
• Original Assignee: Cognex Technology & Investment Corporation (Cognex Corporation)
• Inventors: Nichani, Sanjay
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Lu, Tom Y.
• Application Number:
  US09/450,570
• Time in Patent Office: 1,505 Days
• Field of Search: 382/154, 382/285, 382/103, 382/104, 348/584, 348/169, 348/148, 731/46, 345/419
• US Class Current: 382/103
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

A three-dimensional (3-D) machine-vision obstacle detection solution involving a method and apparatus for performing high-integrity, high efficiency machine vision. The machine vision obstacle detection solution converts two-dimensional video pixel data into 3-D point data that is used for calculation of the closest distance from the vehicle to points on the 3-D objects, for any object within view of at least one imaging device configured to provide obstacle detection. The obstacle detection apparatus includes an image acquisition device arranged to view a monitored scene stereoscopically and pass the resulting multiple video output signals to a computer for further processing. The multiple video output signals are connected to the input of a video processor adapted to accept the video signals. Video images from each camera are then synchronously sampled, captured, and stored in a memory associated with a general purpose processor. The digitized image in the form of pixel information can then be retrieved, manipulated and otherwise processed in accordance with capabilities of the vision system. The machine vision obstacle detection method and apparatus involves two phases of operation: training and run-time. Training is a series of steps in which 3-D image data and other 3-D data are combined into a 3-D model of a vehicle being navigated. During run-time, the entities observed and optionally segmented objects from a camera on the vehicle are compared against the model to detect obstacles and their relative position and trajectory.

Claims

1. A method of implementing a machine vision system to detect an obstacle in a viewed scene, said method comprising the steps of:

developing a 3-D reference model of a vehicle, said reference model including a set of 3-D reference points;

acquiring a runtime version of said viewed scene, said runtime version including a set of 3-D runtime points;

comparing said set of 3-D reference points to said set of 3-D runtime points; and

classifying said set of 3-D runtime points as obstacle, target or background as a function of a result of said comparing step.

2. The method of claim 1 in which said step of developing a 3-D reference model further comprises the steps of:

collecting stereoscopic images of a reference scene containing said vehicle during a training phase;

processing said stereoscopic images for stereoscopic information about said vehicle within the reference scene to develop said set of 3-D reference points.

3. The method of claim 1 in which said step of acquiring said runtime version of said viewed scene further comprises the steps of:

collecting a plurality of images of said viewed scene in a runtime phase;

processing said plurality of images for stereoscopic information about any entity within the viewed scene to determine said set of 3-D runtime points.

4. The method of claim 1 wherein the step of comparing includes generating an output corresponding to a 3-D position of any said obstacle relative to said 3-D reference model.

5. The method of claim 4 in which said step of generating said output corresponding to said 3-D position of any said obstacle further comprises the steps of:

calculating a shortest distance between each 3-D point of said obstacle and 3-D points of said 3-D reference model; and

determining whether said shortest distance is less than a predetermined threshold distance.

6. The method of claim 1 wherein the step of acquiring a run-time version involves processing said set of 3-D runtime points using a clustering algorithm to generate a set of 3-D objects.

7. A method of implementing a machine vision system to compare a model of a 3-D reference vehicle in a reference scene to a runtime scene, said method comprising:

storing information related to said model of said 3-D reference vehicle, said model including information related to said 3-D reference vehicle;

acquiring information related to said runtime scene;

processing said information related to said runtime scene to form stereoscopic information including a set of 3-D points related to said runtime scene;

comparing said information related to said 3-D reference vehicle with said set of 3-D points related to said runtime scene; and

defining any 3-D entity in said runtime scene as one of the 3-D reference vehicle, an obstacle, or background.

8. The method of claim 7 in which said information related to said 3-D reference vehicle is one of an analytical description and a set of 3-D points related to said 3-D reference vehicle.

9. The method of claim 7 in which said step of storing information related to said model of said 3-D reference vehicle further comprises the steps of:

collecting stereoscopic images of said 3-D reference vehicle during a training phase; and

processing said stereoscopic images for stereoscopic information to develop a set of 3-D points corresponding to the 3-D reference vehicle.

10. The method of claim 7 in which said step of acquiring information related to said runtime scene further comprises the step of:

collecting a plurality of successive images of said runtime scene in a runtime phase.

11. The method of claim 7 in which the step of comparing further comprises the step of:

calculating a 3-D distance from said 3-D reference vehicle to each said obstacle.

12. The method of claim 7 further including the step of generating an output corresponding to a 3-D position of any said obstacle relative to said 3-D reference vehicle.

13. The method of claim 12 in which said step of generating said output corresponding to said 3-D position of any said obstacle further comprises the steps of:

calculating a shortest distance between each 3-D point of said obstacle and 3-D points of said 3-D reference model; and

determining whether said shortest distance is less than a predetermined threshold distance.

14. The method of claim 7 in which said step of storing information related to said model of said 3-D reference vehicle further comprises the steps of:

focusing a stereoscopic camera on said reference scene;

collecting a substantially synchronous plurality of frames of video of said reference scene;

digitizing said plurality of frames to create a set of digitized frames forming said information related to said model.

15. The method of claim 7 in which said step of acquiring information related to said runtime scene further comprises the steps of:

focusing a stereoscopic camera on said runtime scene;

collecting a substantially synchronous plurality of frames of video of said runtime scene;

digitizing said plurality of frames to create a set of digitized frames forming said information related to said run-time scene.

16. The method of claim 15 further comprising the steps of:

storing said set of digitized frames in a memory; and

repeating said collecting, digitizing and storing steps for each of a plurality of runtime scenes.

17. The method of claim 7 wherein the step of processing said information related to said runtime scene involves processing said set of 3-D points using a clustering algorithm to generate a set of 3-D objects related to said runtime scene.

18. A machine vision apparatus to detect an obstacle in a viewed scene, comprising:

an image acquisition device;

a processor including,

means for developing a 3-D reference model of a vehicle including a set of 3-D reference points,

means for acquiring a runtime version of said viewed scene including a set of 3-D runtime points;

means for comparing said set of 3-D reference points to said set of 3-D runtime points; and

means for classifying said set of 3-D run-time points as obstacle, target or background as a function of output from said means for comparing.

19. The apparatus of claim 18 further comprising means for processing said set of 3-D runtime points using a clustering algorithm to generate a set of 3-D objects.