Vehicle guidance camera with zoom lens

• US 10,642,271 B1
• Filed: 08/26/2016
• Issued: 05/05/2020
• Est. Priority Date: 08/26/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Graber, Sarah, Watson, Joshua John
• Primary Examiner(s): Kalapodas, Dramos
• Application Number:
  US15/249,225
• Time in Patent Office: 1,348 Days
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 20/80 Arrangement of on-board electronics, e.g. avionics systems or wiring
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2201/102 adapted for flying in formations
• CPC Class Codes: G01B 11/026 by measuring distance between sensor and object G01B11/0608 takes precedence
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 15/00 Optical objectives with means for varying the magnification anamorphotic objectives G02B13/08
• CPC Class Codes: G02B 15/16 with interdependent non-linearly related movements between one lens or lens group, and another lens or lens group G02B15/22 takes precedence
• CPC Class Codes: G02B 3/0006 Arrays G02B3/02, G02B5/188 take precedence
• CPC Class Codes: G02B 7/005 Motorised alignment
• CPC Class Codes: G02B 7/021 for more than one lens
• CPC Class Codes: G02B 7/09 adapted for automatic focusing or varying magnification
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A guidance camera deployed on a vehicle may be equipped with a zoom lens configured to change a zoom level of the guidance camera. The zoom level may be changed in response to a scenario performed by a vehicle equipped with the guidance camera, vehicle controls issued by a vehicle controller, and/or based on identification of objects in imagery captured by the guidance camera. The zoom lens may be implemented as a lens array that includes different lenses, as a wheel lens array that includes different lenses, or by a light direction device that guides light from a specific lens of different lenses into the guidance camera. Stereo cameras may be configured with the zoom lenses, and may be repositioned to ensure suitable overlap in a field of view to enable calculation of a distance of objects captured in the imagery of the stereo camera.

Claims

1. A method comprising:

implementing a vehicle control, determined by an autonomous vehicle controller, to control movement of a vehicle;

determining a first zoom level for a guidance camera system coupled to the vehicle based at least in part on the vehicle control, the guidance camera system including a stereo camera pair and one or more zoom lens devices configured to provide different zoom levels to each camera of the stereo camera pair;

capturing imagery at the first zoom level determined for the guidance camera system;

identifying, based at least in part on the imagery captured at the first zoom level, an objective marker in the imagery;

calculating a distance between the objective marker and the vehicle based at least in part on the first zoom level of the one or more zoom lens devices and a disparity of cameras for the stereo camera pair;

determining a second zoom level based at least in part on the distance of the objective marker being less than or equal to a distance threshold;

causing the one or more zoom lens devices to decrease the stereo camera pair from the first zoom level to the second zoom level;

tracking the objective marker in the imagery captured by the guidance camera system; and

determining, by the autonomous vehicle controller, a subsequent vehicle control to control the movement of the vehicle based at least in part on the second zoom level.

2. The method as recited in claim 1, further comprising:

analyzing the imagery at the first zoom level captured by the guidance camera system to identify an object in the imagery; and

wherein determining the second zoom level is based at least in part on a second distance between the object in the imagery and the vehicle.

3. The method as recited in claim 1, further comprising adjusting the first zoom level for each camera of the stereo camera pair at a same time or near a same time prior to capturing the imagery.

4. The method as recited in claim 3, further comprising repositioning at least one camera of the stereo camera pair in response to determining the first zoom level, the repositioning to create an overlap in one or more fields of view associated with stereo camera pair to enable the calculating of the distance.

5. The method as recited in claim 1, wherein the one or more zoom lens devices include a lens array, further comprising implementing the first zoom level by causing the lens array to move relative to the stereo camera pair to align a lens of the lens array with each camera of the stereo camera pair.

6. The method as recited in claim 1, wherein the one or more zoom lens devices include a light direction device, further comprising implementing the first zoom level by causing the light direction device to be oriented to direct light from a lens of a plurality of lenses into each camera of the stereo camera pair.

7. The method as recited in claim 1, further comprising:

determining a scenario to perform using the vehicle, the scenario including at least one of a transport scenario or a deposit of a package scenario; and

adjusting the first zoom level for the guidance camera system coupled to the vehicle based at least in part on the scenario.

14. The method as recited in claim 2, wherein:

the imagery is a first imagery;

the method further comprises capturing a second imagery at the second zoom level; and

determining the subsequent vehicle control is based at least in part on the second imagery captured at the second zoom level.

8. A vehicle comprising:

a propulsion system to cause movement of the vehicle;

a guidance camera system to capture imagery, the guidance camera system including a stereo camera pair and a zoom lens array to direct light into each camera of the stereo camera pair, the zoom lens array to include a zoom range at least partly encompassed by a first zoom level, a second zoom level, and a third zoom level, wherein the first zoom level and the third zoom level include fields of view that are larger than the second zoom level;

one or more processors; and

memory to store computer-executable instructions executable by the one or more processors to perform operations comprising;

generating, by a vehicle controller, vehicle controls for the vehicle;

transmitting the vehicle controls from the vehicle controller to the propulsion system to cause directed movement of the vehicle;

detecting an object in the imagery captured by the guidance camera system using the first zoom level;

causing the zoom lens array to change the first zoom level to the second zoom level that is greater than the first zoom level;

identifying that the object is an objective marker based at least in part on the change from the first zoom level to the second zoom level;

causing, based at least in part on the objective marker, the zoom lens array to change the second zoom level to the third zoom level;

updating the vehicle controls for the vehicle based at least in part on identifying the objective marker; and

tracking the objective marker in the imagery captured by the guidance camera using the third zoom level.

9. The vehicle as recited in claim 8, wherein the vehicle controls generated by the vehicle controller cause at least one of a first change in heading, a second change in velocity, or a third change in altitude.

10. The vehicle as recited in claim 8, wherein the third zoom level is further modified within the zoom range, based at least in part on a distance between the objective marker and the vehicle.

11. The vehicle as recited in claim 8, wherein the vehicle controls include a vehicle maneuver scenario including at least one of a takeoff scenario, a landing scenario, or a transport scenario.

12. The vehicle as recited in claim 8, wherein the guidance camera system is configured to move a lens of the zoom lens array in alignment with each camera of the stereo camera pair.

13. The vehicle as recited in claim 8, further comprising a light direction device to selectively cause light from a lens of a plurality of lenses of the zoom lens array to be directed into the guidance camera system.

15. The vehicle as recited in claim 8, wherein:

the imagery is a first imagery; and

the guidance camera system is further configured to capture second imagery at the second zoom level and third imagery at the third zoom level; and

the vehicle controller performs a scenario based at least on the second imagery and the third imagery.

16. A guidance camera system comprising:

a stereo camera pair;

a zoom lens array to direct light to the stereo camera pair;

one or more processors; and

memory to store computer-executable instructions executable by the one or more processors to perform operations comprising;

detecting one or more objects in imagery captured by the guidance camera system at a first zoom level;

identifying an objective marker in the imagery captured by the guidance camera, wherein the objective marker is included in the one or more objects;

determining, based at least in part on the objective marker, a distance between the guidance camera and the objective marker;

determining, based at least in part on the distance, a second zoom level that is less than the first zoom level;

modifying, based at least in part on the distance being less than or equal to a distance threshold, the zoom lens array to change from the first zoom level to the second zoom level and track the one or more objects detected in the captured imagery; and

transmitting information associated with the one or more objects to a vehicle controller.

17. The guidance camera system as recited in claim 16, wherein the zoom lens array has a linear configuration, a grid configuration, or a wheel configuration.

18. The guidance camera system as recited in claim 16, wherein the operations further comprise modifying a position of at least one of a first camera or a second camera of the stereo camera pair to provide sufficient overlap of a field of view of each of the first camera and the second camera to enable calculation of a distance of objects.

19. The guidance camera system as recited in claim 16, wherein the one or more objects include at least an unidentified object and the operations further comprise:

determining that the one or more objects include an unidentified object;

modifying the zoom lens array to capture second imagery at a third zoom level;

identifying, based at least in part on the second imagery, the unidentified object captured by the guidance camera system; and

modifying based at least in part on identification of the unidentified object, the zoom lens array to change from the third zoom level to the second zoom level.

20. The guidance camera system as recited in claim 16, wherein the operations further comprise:

determining, based at least in part on the objective marker, an approach velocity associated with the objective marker; and

modifying, based at least in part on the approach velocity, the second zoom level proportional to the approach velocity, wherein the second zoom level is reduced as the distance between the zoom lens array and the objective marker is reduced.