Unmanned aerial vehicle heat sensor calibration

• US 9,969,486 B1
• Filed: 06/21/2016
• Issued: 05/15/2018
• Est. Priority Date: 06/21/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: O'Brien, Barry James, Wilcox, Scott Michael, Watson, Joshua John
• Primary Examiner(s): Badii, Behrang
• Assistant Examiner(s): Testardi, David A
• Application Number:
  US15/188,922
• Time in Patent Office: 693 Days
• CPC Class Codes: B64C 13/20 using radiated signals
• CPC Class Codes: B64C 25/001 Devices not provided for in the groups B64C25/02 - B64C25/68
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64F 1/125 Mooring or ground handling devices for helicopters
• CPC Class Codes: B64F 5/40 Maintaining or repairing aircraft
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/104 using satellite radio beacon positioning systems, e.g. GPS
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: B64U 50/19 using electrically powered motors
• CPC Class Codes: G01C 25/00 Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass testing, calibrating or compensating compasses G01C17/38
• CPC Class Codes: G01J 2005/0077 Imaging
• CPC Class Codes: G01J 5/53 Reference sources, e.g. standard lamps; Black bodies
• CPC Class Codes: G01K 13/00 Thermometers specially adapted for specific purposes
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0653 during a phase of take-off or landing
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30208 : Marker matrix
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 : Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

This disclosure describes systems, methods, and apparatus for automating the verification of aerial vehicle sensors as part of a pre-flight, flight departure, in-transit flight, and/or delivery destination calibration verification process. At different stages, aerial vehicle sensors may obtain sensor measurements about objects within an environment, the obtained measurements may be processed to determine information about the object, as presented in the measurements, and the processed information may be compared with the actual information about the object to determine a variation or difference between the information. If the variation is within a tolerance range, the sensor may be auto adjusted and operation of the aerial vehicle may continue. If the variation exceeds a correction range, flight of the aerial vehicle may be aborted and the aerial vehicle routed for a full sensor calibration.

Claims

1. An aerial vehicle sensor calibration method, comprising:

landing an aerial vehicle at a delivery destination;

ascending from the delivery destination to a defined altitude;

obtaining image information that includes a representation of at least a portion of the delivery destination;

processing the image information to determine at least one of a size of a heat signature at the delivery destination as represented in the image information, a shape of the heat signature at the delivery destination as represented in the image information, a pattern of the heat signature at the delivery destination as represented in the image information, a position of the heat signature at the delivery destination as represented in the image information, an orientation of the heat signature at the delivery destination as represented in the image information, or an intensity of the heat signature at the delivery destination as represented in the image information, wherein the heat signature was produced from at least a portion of the aerial vehicle while the aerial vehicle was landed at the delivery destination;

determining a variation between at least one of the size of the heat signature at the delivery destination as represented in the image information, the shape of the heat signature at the delivery destination as represented in the image information, the pattern of the heat signature at the delivery destination as represented in the image information, the position of the heat signature at the delivery destination as represented in the image information, the orientation of the heat signature at the delivery destination as represented in the image information, or the intensity of the heat signature at the delivery destination as represented in the image information and an actual size of the heat signature, an actual shape of the heat signature, an actual pattern of the heat signature, an actual position of the heat signature, an actual orientation of the heat signature, or an actual intensity of the heat signature;

determining that the variation is within a tolerance range; and

altering a calibration of at least one sensor of the aerial vehicle based at least in part on the variation; and

continuing a flight of the aerial vehicle away from the delivery destination.

2. The aerial vehicle sensor calibration method of claim 1, further comprising:

heating a landing gear of the aerial vehicle; and

wherein at least a portion of the heat signature was produced from a heat generated by the landing gear while the aerial vehicle was landed at the delivery destination.

3. The aerial vehicle sensor calibration method of claim 1, wherein at least a portion of the heat signature was produced from a heat generated by a motor of the aerial vehicle while the aerial vehicle was landed at the delivery destination.

4. The aerial vehicle sensor calibration method of claim 1, wherein the heat signature includes a plurality of heat signatures, each of the plurality of heat signatures generated from a different component of the aerial vehicle while the aerial vehicle was landed at the delivery destination.

5. An aerial vehicle apparatus, comprising:

a frame;

a sensor coupled to the frame;

an aerial vehicle control system in communication with the sensor and configured to at least;

cause the aerial vehicle to land at a location;

cause the aerial vehicle to ascend from the location to a defined altitude;

cause the sensor to obtain data representative of the location;

process the data to generate processed information that includes information indicative of at least one aspect of a heat signature present at the location as represented in the data;

compare the processed information with actual information corresponding to the heat signature expected to be generated by the aerial vehicle to determine a variation between the processed information and the actual information;

determine if the variation is within a tolerance range; and

in response to a determination that the variation is within the tolerance range, continue an aerial navigation of the aerial vehicle.

6. The aerial vehicle apparatus of claim 5, wherein the aerial vehicle control system is further configured to at least:

adjust a calibration of the sensor of the aerial vehicle based at least in part on the variation.

7. The aerial vehicle apparatus of claim 5, wherein the comparison of the processed information with the actual information includes comparing a shape of the heat signature as represented in the processed information with an actual shape of the heat signature to determine a variation of the shape of the heat signature as represented in the data and the actual shape of the heat signature.

8. The aerial vehicle apparatus of claim 5, wherein the aerial vehicle control system is further configured to at least:

heat a landing gear of the aerial vehicle.

9. The aerial vehicle apparatus of claim 8, wherein at least a portion of the heat signature is generated from a heat generated by the landing gear while the aerial vehicle is landed at the location.

10. The aerial vehicle apparatus of claim 8, wherein at least a portion of the heat signature is generated from a heat from at least one motor of the aerial vehicle.

11. The aerial vehicle apparatus of claim 5, wherein the aerial vehicle control system is further configured to at least:

prior to the aerial vehicle landing at the location,

cause the aerial vehicle to descend toward the location;

obtain with a second sensor coupled to the aerial vehicle, data representative of the location;

process the data representative of the location to determine at least one of a size of a marker as represented in the data, a shape of the marker as represented in the data, a color of the marker as represented in the data, or a pattern of the marker as represented in the data;

determine a variation between at least one of the size of a marker as represented in the data, the shape of the marker as represented in the data, the color of the marker as represented in the data, or the pattern of the marker as represented in the data and an actual size of the marker, an actual shape of the marker, an actual color of the marker, or an actual pattern of the marker; and

determine that the variation is below a tolerance threshold; and

wherein the aerial vehicle is caused to land at the location in response to a determination that the variation is below the tolerance threshold.

12. The aerial vehicle apparatus of claim 5, wherein the aerial vehicle control system is further configured to at least:

while the aerial vehicle is at the defined altitude, obtain with a second sensor coupled to the aerial vehicle, data representative of the location, wherein the data is further representative of a payload released at the location.

13. The aerial vehicle apparatus of claim 12, wherein the aerial vehicle control system is further configured to at least:

process the data representative of the location and the payload to determine at least one of a size of the payload as represented in the data, a shape of the payload as represented in the data, a color of the payload as represented in the data, a pattern of the payload as represented in the data, a size of a label included on the payload as represented in the data, a shape of the label included on the payload as represented in the data, a color of the label included on the payload as represented in the data, or a pattern of the label included on the payload as represented in the data;

determine a variation between at least one of the size of the payload as represented in the data, the shape of the payload as represented in the data, the color of the payload as represented in the data, the pattern of the payload as represented in the data, the size of the label included on the payload as represented in the data, the shape of the label included on the payload as represented in the data, the color of the label included on the payload as represented in the data, or the pattern of the label included on the payload as represented in the data and an actual size of the payload, an actual shape of the payload, an actual color of the payload, an actual pattern of the payload, an actual size of a label included on the payload, an actual shape of the label included on the payload, an actual color of the label included on the payload, or an actual pattern of the label included on the payload;

determine that the variation is below a correction threshold; and

wherein the aerial navigation is continued based at least in part on a determination that the variation is below the correction threshold.

14. The aerial vehicle apparatus of claim 5, further comprising:

a plurality of motors coupled to the frame, each motor generating heat while operating; and

wherein the heat signature is generated at the location based at least in part on heat radiating from each of the plurality of motors.

15. The aerial vehicle apparatus of claim 14, wherein the plurality of motors are at defined positions on the frame and produce a known actual heat signature pattern at the location.

16. A sensor calibration method, comprising:

obtaining, while an unmanned aerial vehicle (“

UAV”

) is aerially navigating away from a location, data representative of a heat signature produced by the UAV while the UAV was positioned at the location;

processing the data to generate processed information that includes at least one aspect of the heat signature as represented in the data;

determining a variation between the processed information and actual information that corresponds to at least one actual aspect of the heat signature;

determining that the variation between the processed information and the actual information does not exceed a threshold;

altering a calibration of at least one sensor of the UAV based at least in part on the variation; and

continuing the aerial navigation of the UAV.

17. The sensor calibration method of claim 16, further comprising:

determining that each of a plurality of motors of the UAV are operating based at least in part on the variation.

18. The sensor calibration method of claim 16, wherein the heat signature is generated as a result of at least one of a heat generated by a motor of the UAV or a heat generated by a landing gear of the UAV.

19. The sensor calibration method of claim 16, further comprising:

delivering a payload at the location; and

obtaining with a sensor of the UAV, data that includes a representation of the payload delivered at the location.