Camera calibration using fixed calibration targets

• US 9,986,233 B1
• Filed: 03/16/2017
• Issued: 05/29/2018
• Est. Priority Date: 03/16/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Curlander, James Christopher, Kimchi, Gur, O'Brien, Barry James, Peacock, Jason Leonard
• Primary Examiner(s): Lee, Michael
• Application Number:
  US15/461,335
• Time in Patent Office: 439 Days
• Field of Search: 348187
• CPC Class Codes: B64F 5/60 Testing or inspecting aircraft components or systems
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/19 for use as targets or decoys
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: B64U 50/13 using external fans or propellers
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

Methods and systems for collecting camera calibration data using at least one fixed calibration target are described. Calibration parameters of a camera that is attached to a vehicle may be accessed. A set of calibration instructions for the camera may be determined based at least in part on the calibration parameters. The set of calibration instructions may include navigational instructions for the vehicle to follow to present the camera to the at least one fixed calibration target. Calibration data collected by the camera viewing the at least one fixed calibration target may be received. The camera may be calibrated based at least in part on the calibration data.

Claims

1. An unmanned aerial vehicle, comprising:

a frame;

a propulsion system attached to the frame;

a plurality of cameras attached to the frame and oriented radially about the frame; and

a computing device in communication with the propulsion system and the plurality of cameras, the computing device configured to at least;

operate in one of a plurality of modes comprising a calibration mode, the calibration mode being triggered based at least in part on the unmanned aerial vehicle approaching a set of fixed calibration targets;

receive a set of calibration instructions for gathering calibration data usable to calibrate the plurality of cameras, the set of calibration instructions comprising navigational instructions for the unmanned aerial vehicle to fly with respect to the set of fixed calibration targets such that individual cameras of the plurality of cameras are presented to individual fixed calibration targets at predefined distances and at predefined orientations;

instruct the propulsion system to propel the unmanned aerial vehicle in accordance with the navigational instructions when the calibration mode is activated;

receive the calibration data collected by the plurality of cameras viewing the set of fixed calibration targets while the propulsion system propels the unmanned aerial vehicle in accordance with the set of navigational instructions; and

use the calibration data to perform a photogrammetric calibration of the plurality of cameras.

2. The unmanned aerial vehicle of claim 1, wherein the plurality of cameras are part of a sense and avoid system of the unmanned aerial vehicle.

3. The unmanned aerial vehicle of claim 1, wherein each fixed calibration target comprises a calibration pattern that is recognizable by the plurality of cameras.

4. The unmanned aerial vehicle of claim 1, wherein the set of fixed calibration targets are disposed at various elevations with respect to each other and are of various sizes with respect to each other.

5. The unmanned aerial vehicle of claim 1, wherein:

the computing device is further configured to receive an error message that at least one camera of the plurality of cameras requires calibration; and

instructing the propulsion system to propel the unmanned aerial vehicle in accordance with the navigational instructions is performed in response to receiving the error message.

6. A computer-implemented method, comprising:

accessing calibration parameters of a camera that is attached to a vehicle, the camera usable by the vehicle for navigation of the vehicle, the vehicle being configured to operate in one of a plurality of modes comprising a calibration mode, and the calibration mode being triggered based at least in part on the vehicle approaching a fixed calibration target;

determining a set of calibration instructions based at least in part on the calibration parameters, the set of calibration instructions comprising navigational instructions for the vehicle to follow to present the camera to the fixed calibration target when the calibration mode is activated;

receiving, from the camera, calibration data collected by the camera viewing the fixed calibration target as a result of the vehicle navigating in accordance with the navigational instructions when the calibration mode is activated; and

calibrating the camera based at least in part on the calibration data.

7. The computer-implemented method of claim 6, wherein the camera is one of a plurality of cameras attached to the vehicle at various locations with respect to the vehicle, the plurality of cameras forming at least a part of an object detection system of the vehicle.

8. The computer-implemented method of claim 6, wherein the vehicle is an unmanned aerial vehicle or an automobile.

9. The computer-implemented method of claim 8, wherein:

the camera is one of a plurality of cameras attached to a frame of the unmanned aerial vehicle; and

the navigational instructions comprise instructions for the unmanned aerial vehicle to rotate about a rotational axis of the unmanned aerial vehicle so as to rotationally present the plurality of cameras to the fixed calibration target.

10. The computer-implemented method of claim 6, wherein the fixed calibration target is one of a plurality of fixed calibration targets, each fixed calibration target of the plurality of fixed calibration targets is fixed at a various position and at a various orientation with respect to each other.

11. The computer-implemented method of claim 6, further comprising:

receiving sensor data from one or more onboard sensors of the vehicle, the sensor data indicating at least one of a position of the vehicle or an orientation of the vehicle with respect to the fixed calibration target; and

updating the navigational instructions based at least in part on the sensor data.

12. The computer-implemented method of claim 6, wherein:

the navigational instructions define a path for the vehicle to follow to present the camera to the fixed calibration target; and

the path overlaps with a route that the vehicle commonly follows.

13. The computer-implemented method of claim 12, wherein:

the vehicle is an unmanned aerial vehicle configured to transport items between a home station and a plurality of customers; and

the route is a transportation route followed by the unmanned aerial vehicle when leaving the home station or when returning to the home station.

14. The computer-implemented method of claim 6, wherein

receiving the calibration data comprises receiving the calibration data at least until a calibration data threshold is fulfilled, the vehicle navigating in accordance with the navigational instructions at least until the calibration data threshold is fulfilled.

15. A system, comprising:

an unmanned aerial vehicle comprising;

a plurality of position tracking objects attached to an exterior surface of the unmanned aerial vehicle; and

a camera; and

a motion tracking system comprising;

a plurality of sensors configured to collect sensor data by sensing the position tracking objects;

a plurality of fixed calibration targets disposed within a motion capture area of the motion tracking system; and

a computing device in communication with the plurality of sensors, the computing device configured to at least;

receive sensor data from at least one sensor of the plurality of sensors;

determine, based at least in part on the sensor data, a first position of the unmanned aerial vehicle within the motion capture area and a first vehicle orientation of the unmanned aerial vehicle with respect to a fixed calibration target of the plurality of fixed calibration targets;

instruct the unmanned aerial vehicle to move to a second position within the motion capture area;

instruct the unmanned aerial vehicle to orient into a second vehicle orientation with respect to the fixed calibration target, the camera presented at the fixed calibration target in a first camera orientation when the unmanned aerial vehicle is in the second vehicle orientation;

receive first calibration data from the camera based at least in part on the camera being presented at the fixed calibration target in the first camera orientation;

attempt to calibrate the camera based at least in part on the first calibration data; and

when the camera cannot be calibrated based at least in part on the first calibration data, instruct the unmanned aerial vehicle to orient into a third vehicle orientation with respect to the fixed calibration target, the camera being presented at the fixed calibration target in a second camera orientation when the unmanned aerial vehicle is in the third vehicle orientation.

16. The system of claim 15, wherein the plurality of sensors include optical sensors and the plurality of position tracking objects is configured to reflect light.

17. The system of claim 15, wherein:

the computing device is further configured to receive other sensor data from at least one other sensor of the plurality of sensors; and

wherein instructing the unmanned aerial vehicle to orient into the second vehicle orientation is based at least in part on the other sensor data.

18. The system of claim 17, wherein the computing device is further configured to at least:

access the calibration data from the camera; and

calibrate the camera based at least in part on the calibration data.

19. The system of claim 15, wherein the plurality of calibration targets is mounted within the motion capture area at various positions and at various orientations with respect to each other.