System and method for measuring a displacement of a mobile platform

• US 10,527,416 B2
• Filed: 12/15/2017
• Issued: 01/07/2020
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Zhang, Honghui
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/844,341
• Publication Number:
  US 20180112979A1
• Time in Patent Office: 753 Days
• Field of Search: 348144
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 11/08 the pictures not being supported in the same relative position as when they were taken
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

A method for detecting a displacement of a mobile platform includes obtaining a first frame and a second frame using an imaging device associated with the mobile platform and determining the displacement of the mobile platform based upon the first frame and the second frame.

Claims

1. A method for detecting a displacement of a mobile platform, comprising:

obtaining a first frame and a second frame using an imaging device associated with the mobile platform;

acquiring a height of the mobile platform; and

determining the displacement of the mobile platform based upon the first frame, the second frame, and the height of the mobile platform.

2. The method of claim 1, further comprising:

acquiring an angle of the imaging device with respect to a ground level; and

determining the displacement of the mobile platform further based upon the angle of the imaging device.

3. The method of claim 2, further comprising:

measuring a rotation angle of the mobile platform relative to the first frame to generate rotation data; and

determining the displacement of the mobile platform further based upon the rotation data.

4. The method of claim 3, wherein obtaining the first frame and the second frame comprises acquiring the first frame and the second frame at different time points with an interval not less than one-sixtieth of a second and not greater than one-twentieth of a second.

5. The method of claim 3, wherein determining the displacement of the mobile platform comprises:

acquiring a stereoscopic point cloud based on at least one of the first frame, the height of the mobile platform, or the angle of the imaging device,

wherein the stereoscopic point cloud is an array of feature points {P₁, P₂, P₃, . . . , P_n}, n being a number of the feature points selected in the first frame.

6. The method of claim 5, wherein determining the displacement of the mobile platform further comprises:

acquiring a first projective array in an x-y plane {(x₁¹, y₁¹), (x₂¹, y₂¹), (x₃¹, y₃¹), . . . , (x_n¹, y_n¹)} based on the stereoscopic point cloud; and

matching a plurality of feature points of the first projective array onto the second frame to generate a second projective array {(x₁², y₁²), (x₂², y₂²), (x₃², y₃²), . . . , (x_m², y_m²)}, m being a number of feature points matched in the first frame.

7. The method of claim 6, wherein determining the displacement of the mobile platform further comprises:

calculating a translation array T based on the rotation data, the stereoscopic point cloud, and the second projective array.

8. The method of claim 7, wherein calculating the translation array T comprises applying a relation:

9. The method of claim 7, wherein calculating the translation array T further comprises:

verifying the translation array T by introducing T into an equation

10. The method of claim 6, wherein m is equal to or less than n.

11. An apparatus for detecting displacement of a mobile platform, comprising:

an imaging device associated with the mobile platform and configured to obtain a first frame and a second frame; and

a processor coupled to the imaging device and configured to;

acquire a height of the mobile platform; and

determine the displacement of the mobile platform based upon the first frame and the second frame, and the height of the mobile platform.

12. The apparatus of claim 11, wherein to determine the displacement of the mobile platform, the processor is configured to:

acquire an angle of the imaging device with respect to a ground level; and

determine the displacement of the mobile platform further based upon the angle of the imaging device.

13. The apparatus of claim 12, wherein to determine the displacement of the mobile platform, the processor is configured to:

acquire a rotation angle of the mobile platform relative to the first frame to generate rotation data; and

determine the displacement of the mobile platform further based upon the rotation data.

14. The apparatus of claim 13, wherein the first frame and the second frame are acquired at different time points, an interval between the time points being not less than one-sixtieth of a second and not greater than one-twentieth of a second.

15. The apparatus of claim 13, wherein to determine the displacement of the mobile platform, the processor is further configured to acquire a stereoscopic point cloud based on at least one of the first frame, the height of the mobile platform, or the angle of the imaging device,

wherein the stereoscopic point cloud is an array of feature points {P₁, P₂, P₃, . . . , P_n}, n being a number of the feature points selected in the first frame.

16. The apparatus of claim 15, wherein to determine the displacement of the mobile platform, the processor is further configured to:

acquire a first projective array in an x-y plane {(x₁¹, y₁¹), (x₂¹, y₂¹), (x₃¹, y₃¹), . . . , (x_n¹, y_n¹)} based on the stereoscopic point cloud; and

match a plurality of feature points of the first projective array onto the second frame to generate a second projective array {(x₁², y₁²), (x₂², y₂²), (x₃², y₃²), . . . , (x_m², y_m²)}, m being a number of feature points matched in the first frame.

17. The apparatus of claim 16, wherein to determine the displacement of the mobile platform, the processor is further configured to calculate a translation array T based on the rotation data, the stereoscopic point cloud, and the second projective array.

18. The apparatus of claim 17, wherein to calculate the translation array T, the processor is further configured to calculate the translation array T by applying a relation:

19. The apparatus of claim 17, wherein to calculate the translation array T, the processor is further configured to:

verify the translation array T by introducing T into an equation

20. The apparatus of claim 16, wherein m is equal to or less than n.