Motion detection and tracking from a mobile platform

• US 5,036,474 A
• Filed: 03/31/1989
• Issued: 07/30/1991
• Est. Priority Date: 03/31/1989
• Status: Expired due to Fees

• Current Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Original Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Inventors: Burger, Wilhelm, Bhanu, Bir
• Primary Examiner(s): Lall, Parshotam S.
• Assistant Examiner(s): ZANELLI, MICHAEL J
• Application Number:
  US07/331,706
• Time in Patent Office: 851 Days
• Field of Search: 364/424.02, 364/423, 364/980, 364/514, 364/516, 358/103, 358/105, 358/93, 235/411
• US Class Current: 348/117
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

From two-dimensional displacement vectors of features of objects in successive two-dimensional images, computing a fuzzy focus of expansion and determining qualitative relationships among the features of the scene and motion parameters of the objects in the images and thereby constructing a qualitative three-dimensional representation of the scene captured in two-dimensional images.

Claims

1. A qualitative detection and tracking system for a mobile platform, comprising:

sensing means, mounted on the mobile platform, for acquiring images having entities, wherein each of the entities is either mobile or stationary relative to a world frame of reference;

selecting means, connected to said sensing means, for selecting features from the images;

matching means, connected to said selecting means, for matching the features of successive frames of the images, thereby resulting in two-dimensional displacement vectors between distinct features of the successive frames;

computing means, connected to said matching means, for computing possible focus of expansion locations within the images, from the two-dimensional displacement vectors;

combining means, connected to said computing means, for combining possible focus of expansion locations, thereby resulting in a fuzzy focus of expansion that indicates the approximate direction of heading and amount of rotation in space of the sensing means;

derotation means, connected to said computing means, for removing effects of rotation of said sensing means from the images so as to output derotated two-dimensional displacement vectors;

generating means, connected to said combining means and to said derotation means, for generating hypotheses of qualitative properties of entities in the images, that indicate which entities are in motion relative to the world frame of reference, which entities are stationary relative to the world frame of reference, and which entities are closer to the mobile platform;

qualitative scene model, connected to said generating means, for providing a qualitative three-dimensional representation of the images, wherein the three-dimensional representation is derived from the hypotheses; and

verification means, connected to said qualitative scene model, to said combining means and to said derotation means, for testing the hypotheses for consistency with changes occurring in the images and for deleting hypotheses that are inconsistent with changes occurring in the images, and, in turn, for indicating, according to the hypotheses that are consistent, stationary entities and mobile entities in the images, and for indicating the stationary entities to said computing means.

2. A detection and tracking system for a mobile platform, comprising:

qualitative scene model for providing a three-dimensional representation from a scene from a field of view from a platform in a world, from two-dimensional images, from hypotheses of qualitative properties of entities in the images, that indicate which entities are in motion relative to a world frame of reference, which entities are stationary relative to the world frame of reference, and which entities are closer to the mobile platform, and from premises about the world;

generation means, connected to said qualitative scene model, for generating the hypotheses;

verification means, connected to said qualitative scene model, for verifying internal consistency of the hypotheses and verifying compatibility of the hypotheses with the two-dimensional images;

derotation means, connected to said generation means and to said verification means, for removing effects of rotation from the two-dimensional images;

fuzzy focus of expansion means, connected to said generation means, to said verification means and to said derotation means, for computing a fuzzy focus of expansion wherein said fuzzy focus of expansion locations for each of the two-dimensional images; and

computing means, connected to said verification means, to said derotation means and to said fuzzy focus of expansion means, for computing the possible focus of expansion locations.

3. A qualitative detection and tracking system for a mobile platform, comprising:

a three-dimensional qualitative scene model having interpretations about distance and motion of entities in a scene relative to a world frame of reference, wherein each interpretation is a set of hypotheses about the entities, which portrays a three-dimensional view of the entities for three-dimensional scene model generation;

sensing means, mounted on the mobile platform, for sensing two-dimensional images from the scene;

computing means, connected to said sensing means, for selecting a set of points in each of two consecutive images and for computing displacement vectors for the two sets of points;

focus of expansion means, connected to said computing means, for computing a focus of expansion location from the displacement vectors and for using the focus of expansion location in a subsequent image after said each of two consecutive images;

derotation means, connected to said computing means, for mapping each set of points to locations in the images that the points would be at if said sensing means had not rotated about any axis;

fuzzy focus of expansion means, connected to said derotation means and to said focus of expansion means, for computing a region of possible focus of expansion locations, wherein the region is based on a calculated error of the focus of expansion location for the subsequent image;

generator means, connected to said qualitative scene model and to said fuzzy focus of expansion means, for generating, from the two-dimensional images, the sets of hypotheses that form the interpretations incorporated by said qualitative scene model, wherein the hypotheses are of qualitative properties of entities in the images, that indicate which entities are in motion relative to the world frame of reference, which entities are stationary relative to the world frame of reference, and which entities are closer to the mobile platform; and

verification means, connected to said qualitative scene model, to said fuzzy focus of expansion means and to said focus of expansion means, for verifying the hypotheses and for indicating the interpretations of the qualitative scene model that have all hypotheses consistent with other hypotheses and information in the images, and for indicating stationary entities to said focus of expansion means.

4. A qualitative three-dimensional detection and tracking method for a mobile platform, comprising:

sensing two-dimensional images, having a world frame of reference, from a mobile platform;

selecting entities in the images;

determining two-dimensional displacement vectors from successive images;

computing a focus of expansion location from the two-dimensional displacement vectors of successive images;

computing a fuzzy focus of expansion region that is a qualitative indication of a location of the focus of expansion, wherein said region is an area of possible focus of expansion locations for each image;

determining an approximate direction of heading and amount of rotation from the fuzzy focus of expansion of said sensing from the mobile platform;

removing the effects of rotation of said sensing from the images;

generating hypotheses of qualitative properties of entities in the images, that indicate which entities are in motion relative to the world frame of reference, which entities are stationary relative to the world frame of reference, and which entities are closer to the mobile platform;

constructing interpretations of the images from the hypotheses, wherein each interpretation is a set of hypotheses, which portrays a three-dimensional view of the entities for three-dimensional scene model construction;

verifying the hypotheses of each interpretation that have internal consistency with one another and are compatible with the fuzzy focus of expansion;

rejecting each interpretation that has at least one hypothesis of the same interpretation or that is incompatible with the fuzzy focus of expansion of the image of the interpretation; and

constructing a qualitative three-dimensional scene model from the interpretations having verified hypotheses.

5. A qualitative three-dimensional detection and tracking system for a mobile system, comprising;

a camera, mounted on the mobile platform, wherein said camera acquires sequential two-dimensional images in a world frame of reference;

a first computer, connected to said camera, wherein said computer processes the images, track tokens from successive images, matches the tokens in corresponding images, and determines two-dimensional displacement vectors from the tokens of the successive images;

a bidirectional network protocol device connected to said first computer; and

a second computer, connected to said protocol device, wherein said protocol device interfaces said first and second computers, and wherein said second computer derotates the displacement vectors to remove rotation effects of said camera from the vectors, computes a fuzzy focus of expansion that is a region of possible focus of expansion locations for each image, generating hypotheses that indicate which tokens are stationary and which tokens are mobile relative to the world frame of reference and which tokens are closer to said camera, constructing interpretations of the images from the hypotheses, wherein each interpretation is a set of hypotheses, that portrays a three-dimensional view of the tokens for three-dimensional scene model construction, rejecting interpretations having at least one hypothesis that is inconsistent with any other hypothesis or is incompatible with the fuzzy focus of expansion, and constructing a qualitative three-dimensional scene model from the remaining interpretations.