Landmark recognition for autonomous mobile robots

• US 5,144,685 A
• Filed: 03/31/1989
• Issued: 09/01/1992
• Est. Priority Date: 03/31/1989
• Status: Expired due to Term

• Current Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Original Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Inventors: Nasar, Hatem N., Bhanu, Bir
• Primary Examiner(s): Mancuso, Joseph
• Application Number:
  US07/331,710
• Time in Patent Office: 1,250 Days
• Field of Search: 382/10, 382/14, 382/15, 382/48, 382/9, 358/105, 358/103, 901/1, 901/8, 901/46, 901/47, 369/516, 395/80, 395/81, 395/88
• US Class Current: 382/153
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A dynamic model matching for landmark recognition wherein the model generation and matching process changes as a function of range to the landmark and of perspective as sensed from a mobile platform.

Claims

1. A landmark recognition system or autonomous vehicle navigation, comprising:

a camera, mounted on a vehicle, for outputting images of a field of view;

a first processor, connected to said camera, for processing the images into an image model having segmented regions, extracted objects and features;

a second processor, connected to said first processor, for outputting expected site models for each of the extracted objects;

a third processor, connected to said first and second processors, comprising means for matching expected site models and features from said second processor to extracted objects and features from said first processor, means for accruing evidence of the matching, means for determining site uncertainty of the models, and means for checking consistency between approximated range and sizes of the objects resulting in updated ranges of the objects;

a first data base, connected to said second processor, having model information, map information and camera information; and

a second data base, connected to said second and third processors, having current map location, speed and direction of the vehicle, and an updated estimated range and azimuth of the vehicle.

2. Apparatus of claim 1 wherein said second processor comprises:

an expected map site and objects reasoner connected to said first and second storage means; and

dynamic expectation models generator connected to said first and third processors, to said first and second storage means, and to said expected map site and objects reasoner.

3. Apparatus of claim 2 wherein said first processor comprises:

a low-level processing unit connected to said camera;

a mid-level processing unit connected to said low-level processing unit;

a blackboard, connected to said mid-level processing unit and to said third processor, for structuring information from said mid-level processor and said third processor, and for storing image information; and

an image model unit, connected to said blackboard, and to said second and third processors, for providing a mechanism for observing the results of the system.

4. A landmark recognition process for autonomous vehicle navigation, comprising:

sensing images from a perspective of a vehicle;

retrieving models of landmarks;

generating expectations about landmarks being in the images;

matching images of landmarks with models of landmarks;

accumulating evidence that results in a matching confidence of a relevant landmark model to a particular landmark image at a site of the vehicle;

determining site uncertainty of landmarks located in respective region segments of a map; and

storing site uncertainty information into a knowledge base of the map.

5. A landmark recognition system for autonomous vehicle navigation, comprising:

sensing means for acquiring images;

first processing means, connected to said sensing means, for processing the images;

second processing means comprising means for computing models, means for matching models, means for mapping landmarks, means for verifying matching, means for accumulating evidence and means for determining site uncertainty; and

interface means, connected to said first and second means, for interfacing signals between said first and second processing means.

6. A landmark recognition system for autonomous vehicle navigation, comprising:

sensing means for acquiring images having landmarks, mounted on a vehicle;

image processing means, connected to said sensing means, for providing features of the landmarks in the acquired images;

retrieving means for retrieving models of landmarks;

p1 generating means, connected to said image processing means and to said retrieving means, for generating expectations about landmarks in the images acquired by said sensing means;

matching means, connected to said generating means and to siad sensing means, for matching images of landmarks according to the features with models of landmarks form a landmark data base;

accumulating means, connected to said matching means and to said image processing means, for accumulating evidence that indicates a matching confidence of a relevant model of a landmark to an image of a landmark;

determining means, connected to image processing means and to said accumulating means, for determining site uncertainty of landmarks located in respective region segments of a geographical area; and

storing means, connected to said determining means, for storing site uncertainty information into a data base for the area.

7. A landmark recognition system for autonomous vehicle navigation, comprising:

sensing means, attached to a vehicle, for acquiring images;

extraction means, connected to said sensing means, for extracting objects in the images and extracting features of the objects resulting in an image model;

generating means for generating dynamic 2-D models of objects;

matching means, connected to said extraction means and to said generating means, for dynamic model matching of the 2-D models with the image model, thereby outputting a map location uncertainty;

model data base connected to said generating means;

map data base connected to said generating means;

sensing means model connected to said generating means; and

vehicle data base connected to said generating means.

8. A landmark recognition system for autonomous vehicle navigation, comprising:

camera, mounted on a vehicle, for making images from a field of view of siad camera;

extractor, connected to said camera for extracting of objects from the images and in turn for extracting features of the objects, resulting in an image model;

generator for generating dynamic 2-D models of objects;

correlator, connected to said extractor and to said generator, for dynamically matching the image model with the dynamic 2-D models, and outputting map location uncertainty of said camera;

a first data base, having models of objects, connected to said generator;

a second data base, having map information, connected to said generator;

a third data base, having parameters of siad camera, connected to said generator; and

a fourth data base, having location, speed and direction of the vehicle, connected to said generator.