Control of image triggering for aerial image capturing in nadir alignment for an unmanned aircraft

• US 10,273,000 B2
• Filed: 04/03/2014
• Issued: 04/30/2019
• Est. Priority Date: 04/05/2013
• Status: Active Grant

• Current Assignee: Leica Geosystems AG (Hexagon AB)
• Original Assignee: Leica Geosystems AG (Hexagon AB)
• Inventors: Wagner, Rudiger J., Naderhirn, Michael
• Primary Examiner(s): Dang, Hung Q
• Application Number:
  US14/781,540
• Publication Number:
  US 20160133019A1
• Time in Patent Office: 1,853 Days
• Field of Search: 386227-228
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G03B 15/006 Apparatus mounted on flying objects
• CPC Class Codes: G03B 17/38 Releasing-devices separate from shutter integral with shutter G03B9/08
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G08G 5/003 Flight plan management
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Method for aerial image capturing by means of an unmanned and controllable aircraft comprising a camera, more particularly a drone, during a flight maneuver of said aircraft, comprising continual determining of a camera position and alignment of an optical camera axis and acquiring of a series of aerial images. For each aerial image of said aerial image series, the capturing of the respective aerial image is triggered by flying through a respective image trigger region with said aircraft, wherein the location of said respective image trigger region is determined at least in each case by one trigger position assigned to said respective image trigger region and triggered subject to the alignment of the camera axis when flying through said respective image trigger region, with respect to fulfilling a defined, maximum angle deviation relative to a predetermined spatial alignment.

Claims

1. A control unit for use in an unmanned aircraft during an aerial image capture flight, the control unit comprising:

a sensor unit configured for progressively determining a camera position and a real time alignment of an optical camera axis of a camera; and

a memory device configured for providing flight plan data for the unmanned aircraft,

wherein;

the control unit is configured to;

obtain camera position information from the sensor unit;

obtain alignment information from the sensor unit; and

obtain flight plan data for the unmanned aircraft having the control unit from the memory device; and

generate control signals for controlling the camera of the unmanned aircraft for aerial image capture such that the capture of each aerial image of an aerial image series is triggered in a manner controlled based on the control signals;

if a camera position derivable from the position information is present inside a respective image triggering region derivable from the flight plan data, wherein the location of the respective image triggering region is determined in each case at least by a triggering position, which is associated with the respective image triggering region, according to the flight plan data, and

in dependence on the real time alignment of the optical camera axis, which is derivable from the alignment information, in regard to a fulfillment of a defined maximum angle deviation (α

) with respect to a predetermined spatial alignment, wherein no image is captured if the real time alignment is not within the defined maximum angle deviation (α

).

2. The control unit as claimed in claim 1, wherein:

the image triggering region is defined by a predefined overlap range for two aerial images to be captured successively during the flight movement, wherein the overlap range corresponds to an image area between 60% and 80% of the aerial image, wherein the two aerial images to be captured successively are obtained in the same viewing direction, wherein the two aerial images to be captured successively are obtained in the same viewing direction.

3. The control unit as claimed in claim 2, wherein:

the image triggering region is determined such that a minimum image overlap range, which is predetermined with respect to a flight route predefined by the flight plan, is defined for the two aerial images to be captured successively, wherein the minimum image overlap range is maintained during the capture of the two aerial images to be captured successively.

4. The control unit as claimed in claim 2, wherein:

the minimum image overlap range is 60% of the image area of the aerial image.

5. The control unit as claimed in claim 1, wherein:

the image triggering region is defined by a predefined overlap range for two aerial images to be captured immediately successive during the flight movement, wherein the overlap range corresponds to an image area between 50% and 90% of the aerial image.

6. The control unit as claimed in claim 1, wherein:

for the respective image triggering region, in dependence on the associated triggering position, a starting position and an end position are defined in dependence on the predefined overlap range.

7. The control unit as claimed in claim 1, wherein:

for the respective image triggering region, in dependence on the associated triggering position, a starting position and an end position are defined with respect to the flight route predefined by the flight plan.

8. The control unit as claimed in claim 1, wherein:

the predetermined spatial alignment is a nadir alignment or a horizontal alignment of the camera axis.

9. The control unit as claimed in claim 1, wherein:

the flight plan defines a plurality of triggering positions, wherein each triggering position defines a location of an image triggering region associated in each case for capturing a respective aerial image, wherein the triggering positions defined by the flight plan are approached in flight in a controlled manner using the aircraft.

10. The control unit as claimed in claim 1, wherein:

the flight plan defines a plurality of triggering positions, wherein each triggering position defines a location of an image triggering region associated in each case for capturing a respective aerial image, wherein the triggering positions defined by the flight plan are approached in flight in a controlled manner using the aircraft according to a predefined flight pattern.

11. The control unit as claimed in claim 1, wherein:

the flight plan defines a plurality of triggering positions, wherein each triggering position defines a location of an image triggering region associated in each case for capturing a respective aerial image, wherein the triggering positions defined by the flight plan are approached in flight in a controlled manner in a meander.

12. The control unit as claimed in claim 1, wherein:

in that an update of the next triggering position of the aerial image series with regard to its position in the flight plan is performed in dependence on;

a capture time for the aerial image capture; and

/or

a present camera position during the aerial image capture.

13. The control unit as claimed in claim 1, wherein:

to achieve the defined maximum angle deviation (a) with respect to the predetermined spatial alignment, a flight speed of the aircraft is adapted, wherein an available capture time window for the aerial image capture is enlarged by reducing the flight speed.

14. The control unit as claimed in claim 1, wherein:

in dependence on the fulfillment of the defined maximum angle deviation (a) with respect to the predetermined spatial alignment as the aircraft flies through the image triggering region, a repeated approach flight of the triggering position associated with the image triggering region and capture of the aerial image at this triggering position are performed, and/or

the maximum angle deviation (a) with respect to the predetermined spatial alignment is 5°

.

15. The control unit as claimed in claim 1, wherein:

in dependence on the fulfillment of the defined maximum angle deviation (a) with respect to the predetermined spatial alignment as the aircraft flies through the image triggering region, a repeated approach flight of the triggering position associated with the image triggering region and capture of the aerial image at this triggering position are performed, and/or

the maximum angle deviation (a) with respect to the predetermined spatial alignment is 2°

.

16. A method for aerial image capture using an unmanned and controllable aircraft with a camera during a flight movement of the aircraft, the method comprising:

providing the control unit of claim 1;

progressively determining the camera position and real time alignment information of an optical camera axis under control of the control unit; and

capturing an aerial image series under control of the control unit, wherein;

for each aerial image of the aerial image series, the capture of the respective aerial image is triggered;

as the aircraft flies through a respective image triggering region, wherein the location of the respective image triggering region is determined at least by in each case a triggering position, which is associated with the respective image triggering region, in a flight plan, and

in dependence on the real time alignment of the camera axis during the flight through the respective image triggering region in regard to a fulfillment of a defined maximum angle deviation with respect to a predetermined spatial alignment.

17. A system comprising:

a camera;

a sensor unit configured for progressively determining a camera position and a real time alignment of an optical camera axis of the camera; and

a memory device configured for providing flight plan data for an unmanned aircraft,

a control unit for use in the unmanned aircraft during an aerial image capture flight, wherein;

the control unit is configured to;

obtain camera position information from the sensor unit;

obtain alignment information from the sensor unit; and

obtain the flight plan data for the unmanned aircraft having the control unit from the memory device; and

generate control signals for controlling the camera of the unmanned aircraft for aerial image capture such that the capture of each aerial image of an aerial image series is triggered in a manner controlled based on the control signals;

if a camera position derivable from the position information is present inside a respective image triggering region derivable from the flight plan data, wherein the location of the respective image triggering region is determined in each case at least by a triggering position, which is associated with the respective image triggering region, according to the flight plan data, and

in dependence on the real time alignment of the optical camera axis, which is derivable from the alignment information, in regard to a fulfillment of a defined maximum angle deviation (a) with respect to a predetermined spatial alignment;

wherein;

the position information and the real time alignment information is determined and provided by the sensor unit,

the flight plan data is provided by the memory device, and

the camera is activated by the control signals, which is generated by the control unit, for the aerial image capture, wherein no image is captured if the real time alignment is not within the defined maximum angle deviation (α

).

18. An unmanned and controllable aircraft, having a system as claimed in claim 17, wherein:

the sensor unit comprises a GNSS receiving unit and inertial measuring unit.

19. An unmanned and controllable aircraft, having a system as claimed in claim 17, wherein:

by an aerial image capture functionality such that, during the execution thereof, controlled by the control unit during a flight movement of the aircraft, a capture of at least one aerial image is triggered;

as the aircraft flies through an image triggering region, wherein the location of the image triggering region is determined at least by a triggering position, which is associated with the image triggering region, in a flight plan, and

in dependence on the alignment of the camera axis during the flight through the image triggering region in regard to a fulfillment of a defined maximum angle deviation (a) with respect to a predetermined spatial alignment.

20. A computer program product having program code which is stored on a computer-readable non-transitory storage medium carrier, when executed by a processor causes the processor to perform the following steps for an unmanned aircraft with a camera having a optical camera axis:

progressively obtaining;

position information,

real time alignment information of the optical camera axis, and

flight plan data,

and

controlling an aerial image capture by the camera in dependence on the position information, the real time alignment information, and the flight plan data, wherein the aerial image capture occurs in dependence on an alignment and position of the camera and wherein no image is captured if the real time alignment is not within a defined maximum angle deviation (α

).