Image capture in a vibrational environment

• US 10,178,315 B1
• Filed: 08/02/2016
• Issued: 01/08/2019
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Buchmueller, Daniel
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/226,572
• Time in Patent Office: 889 Days
• Field of Search: 348144
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 20/65 Composite materials
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/104 using satellite radio beacon positioning systems, e.g. GPS
• CPC Class Codes: B64U 30/10 Wings
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: B64U 30/299 Rotor guards ducted or shrouded rotors B64U30/26; guards used as ground propulsion B64U60/60
• CPC Class Codes: B64U 50/19 using electrically powered motors
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/684 performed by controlling the image sensor readout, e.g. by controlling the integration time
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

This disclosure describes optimizing a clarity of images captured by a camera exposed to vibrations, such as a camera mounted on an aerial vehicle. The vibrations can be caused by rotors, motors, forces (e.g., lift, drag, etc.) acting on the UAV, environmental factors (e.g., wind, turbulence, etc.), or any other force that may cause asymmetry. An inertial measurement unit can measure the vibrations and determine a vibrational pattern imposed upon the camera. The inertial measurement unit can identify one or more dead points in the vibrational pattern, and times associated therewith. The inertial measurement unit can send the one or more dead points and/or the times associated therewith to the camera, and cause the camera to capture and/or store images at times corresponding to the one or more dead points to enable capture of images with little or no blur.

Claims

1. A system comprising:

one or more processors; and

one or more memories coupled to the one or more processors, the one or more memories storing instructions executable by the one or more processors to perform acts comprising;

measuring, by an inertial measurement unit of a vehicle, a vibrational pattern imposed on a camera of the vehicle;

determining, by a computing device of the vehicle, one or more dead points associated with the vibrational pattern, wherein the one or more dead points comprise points of minimal acceleration in the vibrational pattern;

identifying, by the computing device, a time window corresponding to a dead point of the one or more dead points;

capturing, by the camera, an image based at least in part on the time window;

storing the image in a cache of the camera; and

sending the image to a datastore.

2. The system of claim 1, wherein the capturing the image comprises causing a shutter of the camera to be open during the time window.

3. The system of claim 1, the acts further comprising:

opening and closing a shutter of the camera to enable the camera to capture a plurality of images, wherein the image is an image of the plurality of images;

comparing times associated with each image of the plurality of images to the time window; and

determining that a time associated with the image corresponds to the time window,

wherein the sending the image to the datastore is based at least in part on the time associated with the image corresponding to the time window.

4. The system of claim 1, wherein the image is an underexposed image of an object, the acts further comprising:

capturing a plurality of underexposed images of the object;

identifying one or more time windows corresponding to the one or more dead points;

determining that a first time associated with a first underexposed image of the plurality of underexposed images corresponds to a first time window of the one or more time windows;

determining that a second time associated with a second underexposed image of the plurality of underexposed images corresponds to a second time window of the one or more time windows; and

combining information from the first underexposed image and the second underexposed image to adjust an exposure of the image.

5. The system of claim 1, the acts further comprising:

identifying, by the computing device, a change in a path of the camera;

predicting, by the computing device, an anticipated vibrational pattern based at least in part on the change in the path;

determining, by the computing device, one or more anticipated dead points of the anticipated vibrational pattern; and

capturing a second image based at least in part on the one or more anticipated dead points.

6. An unmanned aerial vehicle (UAV) comprising:

an airframe;

a camera coupled to the airframe;

an inertial measurement unit (IMU);

one or more processors; and

one or more memories storing instructions executable by the one or more processors to perform acts comprising;

measuring, by the IMU, a vibrational pattern imposed on the camera;

determining, by the IMU, one or more dead points associated with the vibrational pattern, wherein the one or more dead points comprise points of minimal acceleration in the vibrational pattern measured by the IMU;

identifying, by the IMU, one or more time windows corresponding to the one or more dead points; and

selecting an image based at least in part on a time of capture of the image corresponding to at least one of the one or more time windows.

7. The UAV of claim 6, wherein the selecting the image comprises:

causing a shutter of the camera to be open during a time window of the one or more time windows; and

storing the image in a datastore.

8. The UAV of claim 6, the acts further comprising:

sending, from the IMU to the camera, a signal comprising at least one of the one or more dead points or the one or more time windows; and

capturing the image based at least in part on the signal.

9. The UAV of claim 6, wherein the selecting the image comprises:

capturing a plurality of images by the camera;

identifying a first time associated with the image of the plurality of images captured by the camera; and

comparing the first time associated with the image to the one or more time windows.

10. The UAV of claim 6, wherein the image is an underexposed image of an object, the acts further comprising:

capturing a plurality of underexposed images of the object;

determining that a first time associated with a first underexposed image of the plurality of underexposed images corresponds to a first time window of the one or more time windows;

determining that a second time associated with a second underexposed image of the plurality of underexposed images corresponds to a second time window of the one or more time windows; and

combining information from the first underexposed image and the second underexposed image to adjust an exposure of the image.

11. The UAV of claim 6, the acts further comprising:

identifying, by a computing device of the UAV, a change in a flight path;

predicting, by the computing device, an anticipated vibrational pattern based at least in part on the change in the flight path;

determining, by the computing device, one or more anticipated dead points of the anticipated vibrational pattern; and

capturing one or more images based at least in part on the one or more anticipated dead points.

12. The system of claim 11, wherein the predicting the anticipated vibrational pattern further comprises:

identifying, by the computing device, a maneuver associated with the change in the flight path;

determining, by the computing device, a flight control input used to effect the maneuver; and

identifying, by the computing device, one or more vibrations caused by the flight control input.

13. The UAV of claim 6, wherein the IMU is mounted in the camera.

14. The UAV of claim 6, wherein the IMU is mounted in the airframe proximate to the camera.

15. A method comprising:

identifying, by an inertial measurement unit on a vehicle, a vibrational pattern;

determining, by a computing device on the vehicle, one or more dead points of the vibrational pattern, wherein the one or more dead points comprise points of minimal acceleration in the vibrational pattern;

determining, by the computing device, one or more time windows associated with the one or more dead points; and

storing an image in a cache of a camera based at least in part on the one or more dead points.

16. The method of claim 15, further comprising:

opening and closing a shutter of the camera at a time corresponding to a time window of the one or more time windows, wherein a cycle of the shutter enables the camera to capture the image, and

sending the image to a datastore.

17. The method of claim 15, further comprising:

capturing a plurality of images by the camera;

identifying a time associated with each image of the plurality of images captured by the camera; and

comparing the time associated with each image to the one or more time windows,

wherein the storing the image in the cache of the camera is based at least in part on the time associated with each image corresponding to a time window of the one or more time windows.

18. The method of claim 15, further comprising:

receiving a first signal from an image sensor;

responsive to receiving the first signal, opening an electronic shutter of the camera for a first period and capturing light on a plurality of pixels;

determining a sequence of the one or more time windows; and

collecting a group of the plurality of pixels corresponding to the sequence of the one or more time windows,

wherein the group of the plurality of pixels corresponds to the image stored in the cache of the camera.

19. The method of claim 18, further comprising:

determining that the image is an underexposed image; and

receiving a second signal from the image sensor based at least in part on a determination that the image is underexposed, wherein the second signal comprises an instruction to open the shutter for a second period, the second period being longer than the first period.

20. The method of claim 15, further comprising:

identifying, by the computing device, a change in a path of the camera;

predicting, by the computing device, an anticipated vibrational pattern based at least in part on the change in the path, wherein the predicting the anticipated vibrational pattern comprises;

identify a maneuver associated with the change in the path;

determining an input used to effect the maneuver; and

identifying one or more vibrations caused by the input;

determining, by the computing device, one or more anticipated dead points of the anticipated vibrational pattern;

determining, by the computing device, an anticipated sequence of one or more anticipated time windows corresponding to the one or more anticipated dead points; and

capturing, by the camera, images based at least in part on the anticipated sequence.