Validation of camera calibration data using augmented reality

• US 10,447,995 B1
• Filed: 03/16/2017
• Issued: 10/15/2019
• Est. Priority Date: 03/16/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Curlander, James Christopher, Kimchi, Gur, O'Brien, Barry James, Peacock, Jason Leonard
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Smith, Stephen R
• Application Number:
  US15/461,287
• Time in Patent Office: 943 Days
• Field of Search: 348 47, 348148, 348175, 348188, 701 2, 701 3
• CPC Class Codes: B64U 10/16 with five or more distinct rotor axes, e.g. octocopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/69 the UAVs provided with means for airdropping goods, e.g. deploying a parachute during descent
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

Methods and systems for collecting camera calibration data using wearable devices are described. An augmented reality interface may be provided at a wearable device. Directions for a user to present a calibration target to a camera may be presented at the augmented reality interface. Calibration data collected by the camera viewing the calibration target may be received. Existing calibration data for the camera may be validated based at least in part on the collected calibration data.

Claims

1. A system, comprising:

an unmanned aerial vehicle comprising a plurality of cameras;

a head-mounted wearable device comprising;

a transparent display through which physical objects of a physical, real-world environment are visible and on which is presented virtual information in relation to the physical objects; and

a pattern that is recognizable by the plurality of cameras; and

a computing device in communication with the plurality of cameras and the head-mounted wearable device, the computing device configured to;

provide an augmented reality interface at the transparent display of the head-mounted wearable device;

provide, at the augmented reality interface, directions for a wearer of the head-mounted wearable device to move between a plurality of locations with respect to the plurality of cameras;

receive a first indication that a first camera of the plurality of cameras mounted on a first side of the unmanned aerial vehicle recognized the pattern at a first location with respect to the first camera;

receive a second indication that a second camera of the plurality of cameras mounted adjacent to the first camera recognized the pattern at a second location with respect to the second camera;

generate a confirmation message when it is determined that the first location and the second location correspond to a same location with respect to the unmanned aerial vehicle;

access existing calibration data for at least the first camera and the second camera; and

validate the existing calibration data with respect to the first camera and the second camera based at least in part on the confirmation message.

2. The system of claim 1, wherein the directions for the wearer of the head-mounted wearable device comprise directions for the wearer to inspect one or more systems of the unmanned aerial vehicle.

3. The system of claim 1, wherein the pattern comprises a set of lighting elements, each lighting element independently controllable with respect to each other.

4. The system of claim 3, wherein:

the set of lighting elements form a first pattern when the pattern is recognized at the first location; and

the set of lighting elements form a second pattern when the pattern is recognized at the second location.

5. The system of claim 1, wherein the existing calibration data is collected at an earlier time by the wearer moving a moveable calibration target to different positions with respect to the plurality of cameras.

6. A computer-implemented method, comprising:

providing an augmented reality interface at a transparent display of a head-mounted wearable device that is worn by a user;

providing, for presentation at the augmented reality interface, directions for the user to present an updatable calibration target to a camera, the updatable calibration target comprising an updatable pattern that automatically updates based at least in part on one or more of a distance between the head-mounted wearable device and the camera, a resolution value of the camera, a lens type of the camera, a camera type of the camera, or an orientation of the camera;

receiving, from the camera, calibration data collected by the camera viewing the updatable calibration target when the user presents the updatable calibration target at the camera;

accessing existing calibration data for the camera; and

validating the existing calibration data with respect to the camera based at least in part on the calibration data.

7. The computer-implemented method of claim 6, wherein the calibration data comprises a smaller quantity of data than the existing calibration data.

8. The computer-implemented method of claim 6, wherein the camera is one of a plurality of cameras attached to an unmanned aerial vehicle.

9. The computer-implemented method of claim 8, wherein:

the method further comprises;

receiving, from a second camera, second calibration data collected by the second camera viewing the updatable calibration target when the user presents the updatable calibration target at the second camera; and

accessing second existing calibration data for the second camera; and

validating the existing calibration data comprises validating the second existing calibration data with respect to the second camera based at least in part on the second calibration data.

10. The computer-implemented method of claim 6, wherein the directions comprise one or more three-dimensional indicia.

11. The computer-implemented method of claim 6, further comprising providing, for presentation at the augmented reality interface, a three-dimensional indicia that identifies the camera.

12. The computer-implemented method of claim 6, wherein the updatable pattern is attached to the head-mounted wearable device, attached to a first article worn by the user other than the head-mounted wearable device, or attached to a second article held by the user.

13. The computer-implemented method of claim 6, wherein the updatable pattern comprises a set of lighting elements, each lighting element independently controllable with respect to each other.

14. The computer-implemented method of claim 6, wherein the directions for the user comprise directions for inspection of one or more systems of an unmanned aerial vehicle to which the camera is attached.

15. A wearable device, comprising:

a transparent display at which is presented an augmented reality interface;

a calibration target disposed at a front portion of the wearable device, the calibration target comprising an updatable pattern; and

a computing device configured to communicate with a remote computing device, the computing device configured to;

present first information at the augmented reality interface, the first information comprising first indicia identifying a first camera of an unmanned aerial vehicle;

cause the calibration target to present a first pattern to the first camera when the wearable device is disposed at a first position with respect to the first camera;

present second information at the augmented reality interface, the second information comprising second indicia identifying a second camera of the unmanned aerial vehicle; and

cause the calibration target to present a second pattern to the second camera when the wearable device is disposed at a second position with respect to the second camera,

wherein detection of the first pattern by the first camera and detection of the second pattern by the second camera enables the remote computing device to validate an existing calibration of the first camera and the second camera.

16. The wearable device of claim 15, wherein the first position is located a first distance from the first camera and the second position is located at a second, different distance from the second camera.

17. The wearable device of claim 15, wherein the updatable pattern comprises a set of lighting elements, each lighting element independently controllable with respect to each other.

18. The wearable device of claim 15, the first camera and the second camera are held at fixed locations and the wearable device is moved by a user with respect to the first camera and the second camera.

19. The wearable device of claim 15, wherein the transparent display is configured such that the first camera and the second camera are visible through the transparent display and the first indicia is presented as virtual information on the transparent display in relation to the visible first camera and the visible second camera.