Unmanned aerial vehicle sensor calibration via sensor channel

• US 10,302,452 B1
• Filed: 10/20/2017
• Issued: 05/28/2019
• Est. Priority Date: 06/21/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wilcox, Scott Michael, Busek, Naimisaranya Das, Decker, Trevor Joseph Myslinski, Regner, Michael William
• Primary Examiner(s): Paige, Tyler D
• Application Number:
  US15/789,310
• Time in Patent Office: 585 Days
• Field of Search: 701 7
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 20/30 for safety, e.g. with frangible components rotor guards B64U30/299
• CPC Class Codes: B64U 20/70 Constructional aspects of the UAV body
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 30/10 Wings
• CPC Class Codes: B64U 60/50 with landing legs
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 25/00 Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass testing, calibrating or compensating compasses G01C17/38
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G06Q 10/083 Shipping

Abstract

This disclosure describes systems, methods, and apparatus for automating the verification of aerial vehicle sensors as part of a pre-flight, flight departure, in-transit flight, and/or delivery destination calibration verification process. At different stages, aerial vehicle sensors may obtain sensor measurements about objects within an environment, the obtained measurements may be processed to determine information about the object, as presented in the measurements, and the processed information may be compared with the actual information about the object to determine a variation or difference between the information. If the variation is within a tolerance range, the sensor may be auto adjusted and operation of the aerial vehicle may continue. If the variation exceeds a correction range, flight of the aerial vehicle may be aborted and the aerial vehicle routed for a full sensor calibration.

Claims

1. An unmanned aerial vehicle (UAV) flight verification apparatus comprising:

a channel through which a UAV aerially navigates;

a plurality of sensors positioned within the channel and configured to obtain information about the UAV as the UAV aerially navigates through the channel, the plurality of sensors including at least one of;

a microphone configured to receive sound generated by the UAV;

a camera configured to form a plurality of images, each of the plurality of images including a representation of the UAV;

or

a distance determining element configured to determine a distance between the distance determining element and the UAV;

an aerial vehicle management system in communication with each of the plurality of sensors and in communication with the UAV, the aerial vehicle management system configured to at least;

receive channel information, the channel information including at least one of;

the sound received from the microphone;

the plurality of images formed by the camera;

or

distance information representative of the distance determined by the distance determining element;

receive from the UAV, UAV information, the UAV information including at least one of;

revolution per minute (“

RPM”

) information corresponding to a motor of the UAV;

altitude information corresponding to an altitude of the UAV;

position information corresponding to a position of the UAV;

pose information corresponding to a pose of the UAV;

heading information corresponding to a heading of the UAV;

or

velocity information corresponding to a velocity of the UAV;

process the channel information and the UAV information to determine a plurality of variations between the channel information and the UAV information;

determine that each of the plurality of variations are within a respective tolerance range; and

for each of the plurality of variations, provide correction information to the UAV to alter a calibration of a respective sensor of the UAV, wherein the correction information is based at least in part on the variation.

2. The UAV flight verification apparatus of claim 1, wherein:

the plurality of variations include an RPM variation, the RPM variation determined based at least in part on the RPM information corresponding to the motor of the UAV as received from the UAV and an actual RPM determined based at least in part on a processing of the plurality of images; and

the RPM variation corresponds to a difference between the RPM information and the actual RPM.

3. The UAV flight verification apparatus of claim 1, wherein:

the plurality of variations include an altitude variation, the altitude variation determined based at least in part on the altitude information received from the UAV and an actual altitude of the UAV determined based at least in part on the distance information; and

the altitude variation corresponds to a difference between the altitude information received from the UAV and the actual altitude.

4. The UAV flight verification apparatus of claim 1, wherein:

the plurality of variations include a position variation, the position variation determined based at least in part on the position information received from the UAV and an actual position of the UAV determined based at least in part on a processing of the plurality of images or based at least in part on the distance information; and

the position variation corresponds to a difference between the position information received from the UAV and the actual position.

5. The UAV flight verification apparatus of claim 1, wherein:

the plurality of variations include a pose variation, the pose variation determined based at least in part on the pose information received from the UAV and an actual pose of the UAV determined based at least in part on a processing of the plurality of images; and

the pose variation corresponds to a difference between the pose information received from the UAV and the actual pose of the UAV.

6. The UAV flight verification apparatus of claim 1, wherein:

the plurality of variations include a heading variation, the heading variation determined based at least in part on the heading information received from the UAV and an actual heading of the UAV determined based at least in part on a processing of the plurality of images; and

the heading variation corresponds to a difference between the heading information received from the UAV and the actual heading.

7. The UAV flight verification apparatus of claim 1, wherein:

the plurality of variations include a velocity variation, the velocity variation determined based at least in part on the velocity information received from the UAV and an actual velocity of the UAV determined based at least in part on a processing of the plurality of images; and

the velocity variation corresponds to a difference between the velocity information received from the UAV and the actual velocity.

8. An unmanned aerial vehicle (“

UAV”

) sensor calibration method, comprising;

detecting a UAV aerially navigating through a channel;

collecting, with a plurality of sensors positioned within the channel, channel information that includes at least one aspect of the UAV within the channel;

receiving from the UAV, UAV information, the UAV information including information recorded by one or more of a plurality of sensors coupled to the UAV;

processing the channel information and the UAV information to determine a variation between the channel information and the UAV information;

determining that the variation is within a tolerance range; and

authorizing the UAV to continue a flight toward a destination.

9. The method of claim 8, further comprising:

in response to determining that the variation is within a tolerance range, providing correction information to the UAV to alter a calibration of at least one sensor, wherein the correction information is based at least in part on the variation.

10. The method of claim 8, further comprising:

determining that the variation does not exceed a correction threshold.

11. The method of claim 8, wherein the at least one aspect of the UAV included in the channel information includes at least one of an actual position of, a plurality of images of, or a sound generated by the UAV within the channel;

and

wherein the variation indicates a difference between the channel information and the UAV information.

12. The method of claim 8, further comprising:

receiving from the UAV, image information representative of an image formed by a camera coupled to the UAV as the UAV is aerially navigating through the channel, the image including a representation of at least a portion of the channel; and

processing the image information to determine an image variation between the at least a portion of the channel as represented in the image and an actual at least a portion of the channel.

13. The method of claim 12, wherein the actual at least a portion of the channel includes a marker having at least one of a known shape, a known size, or a known color.

14. The method of claim 12, further comprising:

determining that the image variation is within an image tolerance range; and

allowing the UAV to continue the flight toward the destination.

15. An aerial vehicle calibration system, comprising:

a channel through which an aerial vehicle aerially navigates;

the channel including a plurality of sensors configured to obtain information about the aerial vehicle as the aerial vehicle aerially navigates through the channel;

an aerial vehicle management system in communication with the plurality of sensors, the aerial vehicle management system configured to at least;

process the information received from the plurality of sensors to determine that one or more sensors coupled to the aerial vehicle needs calibration; and

in response to a determination that the one or more sensors of the aerial vehicle needs calibration, cause the one or more sensors of the aerial vehicle to be calibrated based at least in part on the processed information.

16. The aerial vehicle calibration system of claim 15, wherein the one or more sensors are manually calibrated by an agent based at least in part on the processed information.

17. The aerial vehicle calibration system of claim 15, wherein:

the aerial vehicle management system processes the information to determine a variation for each of the one or more sensors coupled to the aerial vehicle; and

the aerial vehicle management system is further configured to at least;

determine that at least one variation exceeds a correction threshold; and

cause the aerial vehicle to be routed to a calibration station for a full calibration.

18. The aerial vehicle calibration system of claim 15, wherein the plurality of sensors include at least one of a camera, a distance determining element, a laser, or a microphone.

19. The aerial vehicle calibration system of claim 15, wherein the aerial vehicle management system is in communication with the aerial vehicle and receives aerial vehicle information from the aerial vehicle as the aerial vehicle is aerially navigating through the channel.

20. The aerial vehicle calibration system of claim 19, wherein the aerial vehicle management system processes the information received from the plurality of sensors to determine a variation between the information and the aerial vehicle information.