Egomotion estimation system and method

• US 8,903,127 B2
• Filed: 09/16/2011
• Issued: 12/02/2014
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: Harman International (China) Holdings Co. Ltd. (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International (China) Holdings Co. Ltd. (Samsung Electronics Co. Ltd.)
• Inventors: Yankun, Zhang, Chuyang, Hong, Weyrich, Norman
• Primary Examiner(s): YENTRAPATI, AVINASH
• Application Number:
  US13/234,482
• Publication Number:
  US 20130070962A1
• Time in Patent Office: 1,173 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

A computer-implemented method for determining an egomotion parameter using an egomotion estimation system is provided. First and second image frames are obtained. A first portion of the first image frame and a second portion of the second image frame are selected to respectively obtain a first sub-image and a second sub-image. A transformation is performed on each of the first sub-image and the second sub-image to respectively obtain a first perspective image and a second perspective image. The second perspective image is iteratively adjusted to obtain multiple adjusted perspective images. Multiple difference values are determined that respectively correspond to the respective difference between the first perspective image and the adjusted perspective images. A translation vector for an ego motion parameter is determined. The translation vector corresponds to one of the multiple difference values.

Claims

1. A computer-implemented method for determining an egomotion parameter using an egomotion estimation system comprising:

obtaining a first image frame and a second image frame;

selecting a first portion of the first image frame to obtain a first sub-image that is formed from the first image frame, and selecting a second portion of the second image frame to obtain a second sub-image that is formed from the second image frame;

performing a transformation on the first sub-image and the second sub-image to respectively obtain a first perspective image and a second perspective image;

iteratively adjusting the second perspective image to obtain a plurality of adjusted perspective images;

determining a plurality of difference values respectively corresponding to a respective difference between the first perspective image and the plurality of adjusted perspective images; and

determining a translation vector for an egomotion parameter, wherein the translation vector corresponds to one of the difference values in the plurality of difference values.

2. The computer-implemented method of claim 1 further comprising determining an angle of rotation for the egomotion parameter using the translation vector.

3. The computer-implemented method of claim 1 where the first perspective image and the plurality of adjusted perspective images respectively comprise a plurality of pixels, each pixel having a corresponding pixel value, and further comprising:

summing an absolute difference between corresponding pixel values in the first perspective image and the plurality of adjusted perspective images to respectively determine the plurality of difference values.

4. The computer-implemented method of claim 1 where the translation vector includes a horizontal component and a vertical component and further comprising:

performing a first iteration to obtain a first plurality of difference values and performing a second iteration to obtain a second plurality of difference values;

determining a horizontal translation value that corresponds to one of the difference values in the first plurality of difference values and determining a vertical translation value that corresponds to one of the difference values in the second plurality of difference values; and

identifying the horizontal translation value as the horizontal component of the translation vector and identifying the vertical translation value as the vertical component of the translation vector.

5. The computer-implemented method of claim 4 where the first iteration and the second iteration are performed in parallel.

6. The computer-implemented method of claim 4 where:

the first iteration is performed over a first interval that is based on a maximum horizontal component of a maximum translation vector; and

the second iteration is performed over a second interval that is based on a maximum vertical component of the maximum translation vector.

7. The computer-implemented method of claim 6 further comprising determining the maximum horizontal component and the maximum vertical component of the maximum translation vector based on a predetermined maximum speed, a predetermined image frame rate, and a time interval between the first image frame and the second image frame.

8. The computer-implemented method of claim 7 further comprising determining the time interval between the first image frame and the second image frame based on the image frame rate and a predetermined image frame interval.

9. The computer-implemented method of claim 1 where the second image frame is obtained subsequent to the first image frame.

10. The computer-implemented method of claim 1 where the translation vector corresponds to a smallest difference value in the plurality of difference values.

11. The computer-implemented method of claim 1 where:

the first image frame and the second image frame are obtained by an image capture device mounted to a vehicle; and

the first image frame and the second image frame depict a path of movement of the vehicle.

12. A system for determining an egomotion parameter comprising:

an image processing module that receives a first image frame and a second image frame;

an image segmentation module that divides each of the first image frame and the second image frame into a plurality of portions, selects a first portion of the first image frame to obtain a first sub-image, and selects a second portion of the second image frame to obtain a second sub-image;

an image transformation module that performs a respective transformation on the first sub-image and the second sub-image to respectively obtain a first perspective image and a second perspective image;

a motion compensation module that adjusts the second perspective image to obtain an adjusted perspective image;

an image difference module that determines a difference value corresponding to a difference between the first perspective image and the adjusted image; and

an egomotion parameter determination module that iteratively obtains a plurality of difference values respectively corresponding to a respective difference between the first perspective image and a plurality of adjusted perspective images and determines a translation vector for an egomotion parameter, wherein the translation vector corresponds to one of the difference values in the plurality of difference values.

13. The system of claim 12 where the egomotion parameter determination module determines an angle of rotation for the egomotion parameter using the translation vector.

14. The system of claim 12 where:

the first perspective image and the plurality of adjusted perspective images respectively comprise a plurality of pixels, each pixel having a corresponding pixel value; and

the image difference module sums an absolute difference between corresponding pixel values in the first perspective image and the adjusted perspective image to determine the difference value.

15. The system of claim 12 where:

the translation vector includes a horizontal component and a vertical component;

the egomotion parameter determination module performs a first iteration to obtain a first plurality of difference values and performs a second iteration to obtain a second plurality of difference values;

the egomotion parameter determination module determines a horizontal translation value that corresponds to one of the difference values in the first plurality of difference values and determining a vertical translation value that corresponds to one of the difference values in the second plurality of difference values; and

the egomotion parameter determination module identifies the horizontal translation value as the horizontal component of the translation vector and identifies the vertical translation value as the vertical component of the translation vector.

16. The system of claim 15 where the first iteration and the second iteration are performed in parallel.

17. The system of claim 15 where:

the egomotion parameter determination module iterates over a first interval for the first iteration where the first interval is based on a maximum horizontal component of a maximum translation vector; and

the egomotion parameter determination module iterates over a second interval for the second iteration where the second interval is based on a maximum vertical component of the maximum translation vector.

18. The system of claim 17 where the maximum horizontal component and the maximum vertical component of the maximum translation vector is based on a predetermined maximum speed, a predetermined image frame rate, and a time interval between the first image frame and the second image frame.

19. The system of claim 18 where the time interval between the first image frame and the second image frame is based on the image frame rate and a predetermined image frame interval.

20. The system of claim 12 where the second image frame is subsequently obtained relative to the first image frame, and where the first portion is selected to obtain the first sub-image based on an amount of imaged road surface included in the first portion relative to other portions of the first image frame.

21. The system of claim 12 where the translation vector corresponds to a smallest difference value in the plurality of difference values.

22. The system of claim 12 where:

the image processing module is in signal communication with an image capture device mounted to a vehicle;

the image processing module receives the first image frame and the second image frame from the image capture device; and

the first image frame and the second image frame depict a path of movement of the vehicle.