Methods and apparatus to perform visual odometry using a vehicle camera system

• US 10,545,506 B2
• Filed: 02/14/2018
• Issued: 01/28/2020
• Est. Priority Date: 02/14/2018
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Nariyambut Murali, Vidya, Jales Costa, Bruno Sielly, Sairamesh, Binduhasini, Nagraj Rao, Nikhil
• Primary Examiner(s): Coleman, Stephen P
• Application Number:
  US15/897,013
• Publication Number:
  US 20190250631A1
• Time in Patent Office: 713 Days
• Field of Search: 382103
• CPC Class Codes: B60W 2050/0052 Filtering, filters
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 40/105 Speed
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed to perform visual odometry using a monocular camera. An example apparatus includes a method fusioner to determine a visual odometry method using a surface characterization to calculate first odometry data, a displacement determiner to calculate second odometry data based on sensor information, an odometry calculator to calculate an odometry parameter of a target object with respect to a vehicle based on the first and the second odometry data, and a vehicle command generator to generate a vehicle command based on the odometry parameter.

Claims

1. An apparatus comprising:

a method fusioner to;

determine a visual odometry method using a surface characterization to calculate first odometry data;

calculate a first weight factor for the visual odometry method based on the surface characterization; and

calculate a second weight factor corresponding to an odometry method using sensor information;

a displacement determiner to calculate second odometry data based on the sensor information;

an odometry calculator to;

multiply the first weight factor and the first odometry data to produce a first odometry parameter;

multiply the second weight factor and the second odometry data to produce a second odometry parameter; and

calculate a third odometry parameter of a target object with respect to a vehicle based on a fusion of the first odometry parameter and the second odometry parameter; and

a vehicle command generator to generate a vehicle command based on the third odometry parameter.

2. The apparatus of claim 1, further including a surface evaluator to determine the surface characterization by determining a light reflectivity of a surface on which the vehicle is traveling or whether the surface is uneven.

3. The apparatus of claim 1, wherein the visual odometry method is at least one of a ground feature tracking method or a global motion vector method.

4. The apparatus of claim 1, wherein the third odometry parameter is at least one of a lateral displacement, a longitudinal displacement, or a pose of the target object.

5. The apparatus of claim 1, wherein the sensor information includes at least one of a vehicle speed or a steering wheel angle.

6. The apparatus of claim 1, wherein the vehicle command includes moving the vehicle from a first position to a second position without user input.

7. The apparatus of claim 1, wherein the odometry calculator is to:

obtain a first noise model corresponding to the visual odometry method;

obtain a second noise model corresponding to the odometry method; and

calculate the third odometry parameter by executing a tracking filter using the first and the second noise models.

8. A method comprising:

determining a visual odometry method based on characterizing a surface to calculate first odometry data;

calculating a first weight factor for the visual odometry method based on characterizing the surface;

multiplying the first weight factor and the first odometry data to produce a first odometry parameter;

calculating second odometry data based on sensor information;

calculating a second weight factor corresponding to an odometry method using the sensor information;

multiplying the second weight factor and the second odometry data to produce a second odometry parameter;

calculating a third odometry parameter of a target object with respect to a vehicle based on a fusion of the first odometry parameter and the second odometry parameter; and

generating a vehicle command based on the third odometry parameter.

9. The method of claim 8, wherein characterizing the surface includes determining a light reflectivity of the surface or whether the surface is uneven.

10. The method of claim 8, wherein the visual odometry method is at least one of a ground feature tracking method or a global motion vector method.

11. The method of claim 8, wherein the third odometry parameter is at least one of a lateral displacement, a longitudinal displacement, or a pose of the target object.

12. The method of claim 8, wherein the sensor information includes at least one of a vehicle speed or a steering wheel angle.

13. The method of claim 8, wherein the vehicle command includes moving the vehicle from a first position to a second position without user input.

14. The method of claim 8, further including:

obtaining a first noise model corresponding to the visual odometry method;

obtaining a second noise model corresponding to the odometry method; and

calculating the third odometry parameter by executing a tracking filter using the first and the second noise models.

15. A non-transitory computer readable storage medium comprising instructions which, when executed, cause a machine to at least:

determine a visual odometry method based on characterizing a surface to calculate first odometry data;

calculate a first weight factor for the visual odometry method based on characterizing the surface;

multiply the first weight factor and the first odometry data to produce a first odometry parameter;

calculate second odometry data based on sensor information;

calculate a second weight factor corresponding to an odometry method using the sensor information;

multiply the second weight factor and the second odometry data to produce a second odometry parameter;

calculate a third odometry parameter of a target object with respect to a vehicle based on a fusion of the first odometry parameter and the second odometry parameter; and

generate a vehicle command based on the third odometry parameter.

16. The non-transitory computer readable storage medium of claim 15, wherein the sensor information includes at least one of a vehicle speed or a steering wheel angle.

17. The non-transitory computer readable storage medium of claim 15, wherein the instructions, when executed, cause the machine to:

obtain a first noise model corresponding to the visual odometry method;

obtain a second noise model corresponding to the odometry method; and

calculate the third odometry parameter by executing a tracking filter using the first and the second noise models.

18. An apparatus comprising:

a method fusioner to determine a visual odometry method based on a characterization of a surface to calculate first odometry data;

a displacement determiner to calculate second odometry data based on sensor information;

an odometry calculator to calculate an odometry parameter of a target object with respect to a vehicle based on the first odometry data and the second odometry data by executing a tracking filter using a first noise model corresponding to the visual odometry method and a second noise model corresponding to an odometry method using the sensor information; and

a vehicle command generator to generate a vehicle command based on the odometry parameter.

19. The apparatus of claim 18, wherein the odometry parameter is a third odometry parameter, and wherein:

the method fusioner is to;

calculate a first weight factor for the visual odometry method based on the characterization of the surface; and

calculate a second weight factor corresponding to the odometry method using the sensor information; and

the odometry calculator is to;

multiply the first weight factor and the first odometry data to produce a first odometry parameter;

multiply the second weight factor and the second odometry data to produce a second odometry parameter; and

calculate the third odometry parameter based on a fusion of the first odometry parameter and the second odometry parameter.

20. The apparatus of claim 18, wherein the odometry parameter is at least one of a lateral displacement, a longitudinal displacement, or a pose of the target object.