Vision Based Calibration System For Unmanned Aerial Vehicles

• US 20180002017A1
• Filed: 08/04/2016
• Published: 01/04/2018
• Est. Priority Date: 03/28/2016
• Status: Active Grant

• Current Assignee: Zipline International Inc.
• Original Assignee: Zipline International Inc.
• Inventors: Abeles, Peter, Wyrobek, Keenan
• Granted Patent: US 10,029,788 B2
• Time in Patent Office: Days
• CPC Class Codes: B64F 5/60 Testing or inspecting aircraft components or systems
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 10/25 Fixed-wing aircraft VTOL aircraft B64U10/20
• CPC Class Codes: B64U 10/30 Lighter-than-air aircraft, e.g. aerostatic aircraft
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 70/00 Launching, take-off or landing arrangements
• CPC Class Codes: B64U 80/00 Transport or storage specially adapted for UAVs
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An unmanned aircraft system includes a testing and calibration system that enables automated testing of movable parts of an unmanned aircraft. The testing and calibration system uses a camera-based technique to determine the position and angle of movable parts, in order to establish whether or not those parts are moving in a manner consistent with correct function.

Claims

1. An automated calibration system, comprising:

a camera configured to be positioned relative to a vehicle such that a base fiducial and a movable fiducial on the vehicle are visible within a field of view of the camera; and

and a testing and calibration computer configured to;

receive an image of the base fiducial and the movable fiducial from the camera, the image comprising image data;

determine a relative orientation between the base fiducial and the movable fiducial based on the image data;

determine a real position of a movable part of the vehicle based on the relative orientation between the base fiducial and the movable fiducial, and a position of a base surface of the vehicle;

compare the real position of the movable part to a reported position for the movable part; and

generate a notification if a difference between the real position and the reported position is greater than a threshold amount.

2. The system of claim 1, wherein determining the relative orientation between the base fiducial and the movable fiducial based on the image data further comprises:

determining a 2D orientation of the base fiducial and a 2D orientation of the movable fiducial based on the image data;

determining a 3D orientation of the base fiducial and a 3D orientation of the movable fiducial based on the 2D orientations of the movable and base fiducials and a known set of intrinsic parameters for the camera; and

determining the relative orientation between the base fiducial and the movable fiducial based on the 3D orientations of the fiducials.

3. The system of claim 2, wherein determining the 2D orientation of the base fiducial and the 2D orientation of the movable fiducial further comprises identifying, in the image data, fiducial features of the base fiducial and the movable fiducial.

4. The system of claim 3, wherein determining the 3D orientation of the base fiducial and the 3D orientation of the movable fiducial further comprises determining fiducial geometries of the base fiducial and the movable fiducial based on the fiducial features.

5. The system of claim 1, wherein the image data comprises greyscale pixels.

6. The system of claim 1, wherein the base fiducial and the movable fiducial comprise rectangular patterns in black and white.

7. The system of claim 1, wherein the movable fiducial is affixed to the movable part of the vehicle and the base fiducial is affixed to a surface of the vehicle adjacent to the movable part, the surface fixed relative to the movable part.

8. The system of claim 1, wherein the reported position is a value reported by an actuator control system of the vehicle.

9. The system of claim 1, wherein the vehicle is an unmanned aerial vehicle, and the movable part is a control surface of the unmanned aerial vehicle.

10. A method for automated calibration, comprising:

capturing, with a camera, an image of a base fiducial and a movable fiducial, the image comprising image data;

determining a relative orientation between the base fiducial and the movable fiducial based on the image data;

determining a real position of a movable part of a vehicle based on the relative orientation between the base fiducial and the movable fiducial, and a position of a base surface of the vehicle;

comparing the real position of the movable part to a reported position for the movable part; and

generating a notification if a difference between the real position and the reported position is greater than a threshold amount.

11. The method of claim 10, wherein determining the relative orientation between the base fiducial and the movable fiducial based on the image data, further comprises:

determining a 2D orientation of the base fiducial and a 2D orientation of the movable fiducial based on the image data;

determining a 3D orientation of the base fiducial and a 3D orientation of the movable fiducial based on the 2D orientations of the movable and base fiducials and a known set of intrinsic parameters for the camera; and

determining the relative orientation between the base fiducial and the movable fiducial based on the 3D orientations of the fiducials.

12. The method of claim 11, wherein determining the 2D orientation of the base fiducial and the 2D orientation of the movable fiducial further comprises identifying, in the image data, fiducial features of the base fiducial and the movable fiducial.

13. The method of claim 12, wherein determining the 3D orientation of the base fiducial and the 3D orientation of the movable fiducial further comprises determining fiducial geometries of the base fiducial and the movable fiducial based on the fiducial features.

14. The method of claim 10, wherein the image data comprises greyscale pixels.

15. The method of claim 10, wherein the base fiducial and the movable fiducial comprise rectangular patterns in black and white.

16. The method of claim 10, wherein the movable fiducial is affixed to the movable part of the vehicle and the base fiducial is affixed to a surface of the vehicle adjacent to the movable part, the surface fixed relative to the movable part.

17. The method of claim 10, wherein the reported position is a value reported by an actuator control system of the vehicle.

18. The method of claim 10, wherein the vehicle is an unmanned aerial vehicle, and the movable part is a control surface of the unmanned aerial vehicle.

19. A method for predicting failures in a movable part of a vehicle, comprising:

a data collection process, the data collection process comprising;

capturing, with a camera, an image of a base fiducial affixed to a surface of the vehicle adjacent to the movable part and a movable fiducial affixed to the movable part, the image comprising image data;

determining a relative orientation between the base fiducial and the movable fiducial based on the image data;

determining a real position of a movable part of a vehicle based on the relative orientation between the fiducials and a known real position of a base surface of the vehicle; and

generating an actuator variance based on a difference between the real position of the movable part and a reported position for the movable part;

a model development process comprising;

executing the data collection process a plurality of times over a period of time for the movable part of the vehicle and storing a plurality of actuator variances generated by the data collection process over the period of time;

observing a failure in the movable part; and

developing a predictive model based on the stored plurality of actuator variances; and

a diagnostic process comprising;

executing the data collection process a second plurality of times for a second movable part of a second vehicle over a second period of time and storing a second plurality of actuator variances generated by the data collection process over the second period of time;

applying the predictive model to the second plurality of actuator variances; and

generating a notification based on the output of the predictive model.

20. The method of claim 19, wherein determining the relative orientation between the base fiducial and the movable fiducial based on the image data, further comprises:

determining a 2D orientation of the base fiducial and a 2D orientation of the movable fiducial based on the image data;

determining a 3D orientation of the base fiducial and a 3D orientation of the movable fiducial based on the 2D orientations of the movable and base fiducials and a known set of intrinsic parameters for the camera; and

determining the relative orientation between the base fiducial and the movable fiducial based on the 3D orientations of the fiducials.

21. The method of claim 19, wherein the predictive model is generated using a machine learning algorithm.

22. The method of claim 19, wherein the notification is generated when the predictive model determines that there is a high likelihood of failure of the second movable part based on the stored second plurality of actuator variances.

23. The method of claim 19, wherein the first and second movable parts are control surfaces and the first and second vehicles are unmanned aerial vehicles.