Method and apparatus for vision based motion determination

• US 8,238,612 B2
• Filed: 08/28/2008
• Issued: 08/07/2012
• Est. Priority Date: 05/06/2008
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Susca, Sara, Krishnaswamy, Kailash
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US12/200,827
• Publication Number:
  US 20090279741A1
• Time in Patent Office: 1,440 Days
• Field of Search: 382/107, 382/236, 382/285, 348/97, 348/154, 348/155, 348169-172, 348/208.1, 348/352, 348/407.1, 702/152, 702/153, 715/852
• US Class Current: 382/107
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

A method for determining motion is provided. The method determines a rotation of an object from a first time to a second time by analyzing a first 2D image obtained at the first time and a second 2D image obtained at the second time. Then, the method determines a translation of the object from the first time to the second time based on the determined rotation, 3D information relating to the first image, and 3D information relating to the second image.

Claims

1. A method for determining motion comprising:

determining a rotation of an object from a first time to a second time by analyzing a first two-dimensional (2D) image obtained at the first time and a second 2D image obtained at the second time, the rotation determined with an eight-point algorithm; and

determining a translation of the object from the first time to the second time based on the determined rotation, three-dimensional (3D) information relating to the first 2D image, and 3D information relating to the second 2D image, wherein determining the translation comprises;

inputting the rotation determined by the eight-point algorithm into an absolute orientation algorithm; and

calculating the translation from the determined rotation, the 3D information relating to the first 2D image, and the 3D information relating to the second 2D image.

2. The method of claim 1, further comprising:

obtaining the first 2D image with a first optical camera; and

obtaining the second 2D image with the first optical camera.

3. The method of claim 2, further comprising:

obtaining a third 2D image at the first time with a second optical camera;

obtaining a fourth 2D image at the second time with the second optical camera;

building the 3D information relating to the first 2D image based on the first 2D image and the third 2D image; and

building the 3D information relating to the second 2D image based on the second 2D image and the fourth 2D image.

4. The method of claim 3, further comprising:

determining a second rotation of at least one feature by analyzing the third 2D image and the fourth 2D image;

determining a final rotation based on the rotation from the first 2D image and the second 2D image and the second rotation from the third 2D image and the fourth 2D image; and

wherein determining a translation between the first time and the second time further comprises determining a translation based on the final rotation, 3D information relating to the first 2D image and the third 2D image, and 3D information relating to the second 2D image and the fourth 2D image.

5. The method of claim 1, further comprising:

obtaining the first 2D image with a light detection and ranging (LIDAR) camera; and

obtaining the second 2D image with the LIDAR camera;

wherein intensity images from the LIDAR camera are used as the first 2D image and the second 2D image.

6. The method of claim 5, further comprising:

building the 3D information relating to the first 2D image based on a first intensity image taken at the first time and a first range image taken at the first time;

building the 3D information relating to the second 2D image based on a second intensity image taken at the second time and a second range image taken at the second time.

7. An apparatus for determining motion comprising:

a processor for executing software;

at least one imaging device communicatively coupled to the processor;

a storage medium communicatively coupled to the processor from which the processor reads at least a portion of the software for execution thereby, wherein the software is configured to cause the processor to;

determine a rotation of an object from a first time to a second time by analyzing a first two-dimensional (2D) image obtained at the first time by the at least one imaging device and a second 2D image obtained at the second time by the at least one imaging device or another imaging device, the rotation determined with an eight-point algorithm, and

determine a translation of the object from the first time to the second time based on the determined rotation, three-dimensional (3D) information relating to the first 2D image, and 3D information relating to the second 2D image, wherein the translation is determined by;

inputting the rotation determined by the eight-point algorithm into an absolute orientation algorithm; and

calculating the translation from the determined rotation, the 3D information relating to the first 2D image, and the 3D information relating to the second 2D image.

8. The apparatus of claim 7, wherein the at least one imaging device comprises a first optical camera and a second optical camera, wherein the first optical camera obtains the first 2D image and the second 2D image and the second optical camera obtains a third 2D image at the first time and a fourth 2D image at the second time;

and

wherein the software is configured to cause the processor to;

build the 3D information relating to the first 2D image based on the first 2D image and the third 2D image; and

build the 3D information relating to the second 2D image based on the second 2D image and the fourth 2D image.

9. The apparatus of claim 7, wherein the at least on imaging device is a light detection and ranging (LIDAR) camera, and wherein the first 2D image and the second 2D image are intensity images from the LIDAR camera.

10. The apparatus of claim 9, wherein the software is configured to cause the processor to:

build the 3D information relating to the first 2D image based on a first intensity image taken at the first time and a first range image taken at the first time;

build the 3D information relating to the second 2D image based on a second intensity image taken at the second time and a second range image taken at the second time.

11. A program product comprising a processor-readable medium on which program instructions are embodied, wherein the program instructions are operable to:

extract a first set of features from a first two-dimensional (2D) image taken at a first time;

extract a second set of features from a second 2D image take at a second time;

match features from the first set of features with features from the second set of features;

determine a rotation between the first time and the second time based on the matched features, the rotation determined with an eight-point algorithm;

build a first set of three-dimensional (3D) feature information for the first 2D image from information from the first 2D image and other information obtained at the first time;

build a second set of 3D feature information for the second 2D image from information from the second 2D image and other information obtained at the second time; and

determine a translation between the first time and the second time based on the determined rotation, the first set of 3D feature information, and the second set of 3D feature information, wherein the translation is determined by;

inputting the rotation determined by the eight-point algorithm into an absolute orientation algorithm; and

calculating the translation from the determined rotation, the 3D information relating to the first 2D image, and the 3D information relating to the second 2D image.

12. The program product of claim 11, wherein the first 2D image and the second 2D image are obtained with a first optical camera, and a third 2D image is obtained at the first time with a second optical camera and a fourth 2D image is obtained at the second time with the second optical camera;

and

wherein the software is configured to cause the processor to;

build the 3D information relating to the first 2D image based on the first 2D image and the third 2D image; and

build the 3D information relating to the second 2D image based on the second 2D image and the fourth 2D image.

13. The program product of claim 11, wherein the first 2D image and the second 2D image are a first intensity image and a second intensity image from a light detection and ranging (LIDAR) camera;

wherein the software is configured to cause the processor to;

build the 3D information relating to the first 2D image based on the first intensity image taken at the first time and a first range image taken at the first time; and

build the 3D information relating to the second 2D image based on the second intensity image taken at the second time and a second range image taken at the second time.