METHOD AND APPARATUS FOR VISION BASED MOTION DETERMINATION

• US 20090279741A1
• Filed: 08/28/2008
• Published: 11/12/2009
• Est. Priority Date: 05/06/2008
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell (Honeywell International Inc.)
• Inventors: Krishnaswamy, Kailash, Susca, Sara
• Granted Patent: US 8,238,612 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

A method for determining motion is provided. The method determines a rotation of an object from a first time to a second time by analyzing a first 2D image obtained at the first time and a second 2D image obtained at the second time. Then, the method determines a translation of the object from the first time to the second time based on the determined rotation, 3D information relating to the first image, and 3D information relating to the second image.

Claims

1. A method for determining motion comprising:

determining a rotation of an object from a first time to a second time by analyzing a first 2D image obtained at the first time and a second 2D image obtained at the second time; and

determining a translation of the object from the first time to the second time based on the determined rotation, 3D information relating to the first image, and 3D information relating to the second image.

2. The method of claim 1, wherein determining a rotation further comprises determining a rotation with an eight-point algorithm.

3. The method of claim 1, wherein determining a translation further comprises determining a translation with an absolute orientation algorithm.

4. The method of claim 3, wherein determining a rotation further comprises determining a rotation with an eight-point algorithm;

and

wherein determining a translation further comprises determining a translation by inputting the rotation determined by the eight-point algorithm into the absolute orientation algorithm and calculating a translation from the determined rotation, 3D information relating to first image, and 3D information relating to the second image.

5. The method of claim 1, further comprising:

obtaining the first 2D image with an optical camera; and

obtaining the second 2D image with the optical camera.

6. The method of claim 5, further comprising:

obtaining a third 2D image at the first time with a second optical camera;

obtaining a fourth 2D image at the second time with the second optical camera;

building the 3D information relating to the first 2D image based on the first 2D image and the third 2D image; and

building the 3D information relating to the second 2D image based on the second 2D image and the fourth 2D image.

7. The method of claim 6, further comprising:

determining a second rotation of the at least one feature by analyzing the third image and the fourth image;

determining a final rotation based on the rotation from the first image and the second image and the second rotation from the third image and the fourth image; and

wherein determining a translation between the first time and the second time further comprises determining a translation based on the final rotation, 3D information relating to the first image and the third image, and 3D information relating to the second image and the fourth image.

8. The method of claim 1, further comprising:

obtaining the first 2D image with a light intensity and ranging (LIDAR) camera; and

obtaining a second 2D image with the LIDAR camera;

wherein the intensity images from the LIDAR camera are used as the first 2D image and the second 2D image.

9. The method of claim 8, further comprising:

building the 3D information relating to the first 2D image based on a first intensity image taken at the first time and a first range image taken at the first time;

building the 3D information relating to the second 2D image based on a second intensity image taken at the second time and a second range image taken at the second time.

10. An apparatus for determining motion comprising:

a processor for executing software;

at least one imaging device communicatively coupled to the processor;

a storage medium communicatively coupled to the processor from which the processor reads at least a portion of the software for execution thereby, wherein the software is configured to cause the processor to;

determine a rotation of an object from a first time to a second time by analyzing a first 2D image obtained at the first time by the at least one imaging device and a second 2D image obtained at the second time by the at least one imaging device; and

determine a translation of the object from the first time to the second time based on the determined rotation, 3D information relating to the first image, and 3D information relating to the second image.

11. The apparatus of claim 10, wherein the software is configured to cause the processor to determine the rotation with an eight-point algorithm.

12. The apparatus of claim 10, wherein the software is configured to cause the processor to determine the translation with an absolute orientation algorithm.

13. The apparatus of claim 12, wherein the software is configured to cause the processor to determine the rotation with an eight-point algorithm;

and

determine the translation by inputting the rotation determined by the eight-point algorithm into the absolute orientation algorithm and calculating a translation from the determined rotation, 3D information relating to first image, and 3D information relating to the second image.

14. The apparatus of claim 10, wherein the at least one imaging device comprises a first optical camera and a second optical camera, wherein the first optical camera obtains the first 2D image and the second 2D image and the second optical camera obtains a third 2D image at the first time and a fourth 2D image at the second time;

and

wherein the software is configured to cause the processor to

build the 3D information relating to the first 2D image based on the first 2D image and the third 2D image; and

build the 3D information relating to the second 2D image based on the second 2D image and the fourth 2D image.

15. The apparatus of claim 10, wherein the at least on imaging device is a light intensity and ranging (LIDAR) camera, and wherein the first 2D image and the second 2D image are intensity images from the LIDAR camera.

16. The apparatus of claim 15, wherein the software is configured to cause the processor to:

build the 3D information relating to the first 2D image based on a first intensity image taken at the first time and a first range image taken at the first time;

build the 3D information relating to the second 2D image based on a second intensity image taken at the second time and a second range image taken at the second time.

17. A program product comprising a processor-readable medium on which program instructions are embodied, wherein the program instructions are operable to:

extract a first set of features from the first 2D image taken at a first time;

extract a second set of features from the second 2D image take at a second time;

match features from the first set of features with features from the second set of features;

determine a rotation between the first time and the second time based on the matched features;

build a first set of 3D feature information for the first 2D image from information from the first 2D image and other information obtained at the first time;

build a second set 3D feature information for the second 2D image from information from the second 2D image and other information obtained at the second time; and

determine a translation between the first time and the second time based on the determined rotation, the first set of 3D feature information, and the second set of 3D feature information.

18. The program product of claim 17, wherein the program instructions are operable to:

determine the rotation with an eight-point algorithm;

determine the rotation with an absolute orientation algorithm; and

determine the translation by inputting the rotation determined by the eight-point algorithm into the absolute orientation algorithm and calculating a translation from the determined rotation, 3D information relating to first image, and 3D information relating to the second image.

19. The program product of claim 17, wherein the first 2D image and the second 2D image are obtained with a first optical camera and a third 2D image is obtained at the first time with a second optical camera and a fourth 2D image is obtained at the second time with a second optical camera;

and

wherein the software is configured to cause the processor to

build the 3D information relating to the first 2D image based on the first 2D image and the third 2D image; and

build the 3D information relating to the second 2D image based on the second 2D image and the fourth 2D image.

20. The apparatus of claim 17, wherein the first 2D image and the second 2D image are a first intensity image and a second intensity image of a light intensity and ranging (LIDAR) camera;

wherein the software is configured to cause the processor to;

build the 3D information relating to the first 2D image based on the first intensity image taken at the first time and a first range image taken at the first time;

build the 3D information relating to the second 2D image based on the second intensity image taken at the second time and a second range image taken at the second time.