Aerial image collection

• US 9,811,082 B2
• Filed: 06/28/2016
• Issued: 11/07/2017
• Est. Priority Date: 03/05/2013
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Tillmann, Marco
• Primary Examiner(s): SMITH, JELANI A
• Application Number:
  US15/195,123
• Publication Number:
  US 20160306354A1
• Time in Patent Office: 497 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 10/17 Helicopters flying platforms B64U10/13
• CPC Class Codes: B64U 10/60 Tethered aircraft
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 11/08 the pictures not being supported in the same relative position as when they were taken
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

In one embodiment, an aerial collection system includes an image collection field vehicle that travels at street level and an image collection aerial vehicle that travels in the air above the street. The aerial vehicle collects image data including at least a portion of the field vehicle. The field vehicle includes a marker, which is identified from the collected image data. The marker is analyzed to determine an operating characteristic of the aerial vehicle. In one example, the operating characteristic in the marker includes information for a flight instruction for the aerial vehicle. In another example, the operating characteristic in the marker includes information for the three dimensional relationship between the vehicles. The three dimensional relationship is used to combine images collected from the air and images collected from the street level.

Claims

1. A method for building a model through cooperation of movement of an aerial vehicle and a terrestrial vehicle, the method comprising:

receiving location data for a mobile device associated with the aerial vehicle or the terrestrial vehicle;

receiving a first image collected by the aerial vehicle;

receiving a second image collected by the terrestrial vehicle;

storing the first image and the second image in a geographic database;

determining a spatial relationship between the aerial vehicle and the terrestrial vehicle;

deriving the model from a combination of the first image and the second image and the spatial relationship between the aerial vehicle and the terrestrial vehicle from the cooperation of movement of the aerial vehicle and the terrestrial vehicle; and

indexing, by a processor, the model from the combination of the first image and the second image in the geographic database based on the location data and the spatial relationship between the aerial vehicle and the terrestrial vehicle.

2. The method of claim 1, wherein the first image and the second image are combined based on an orientation of the aerial vehicle.

3. The method of claim 1, further comprising:

receiving first range data collected at the aerial vehicle and second range data collected at the terrestrial vehicle, wherein the combined image is based on the first range data and the second range data.

4. The method of claim 1, wherein a first height portion of the combined image is derived from the first image and a second height portion of the combined image is derived from the second image.

5. The method of claim 4, wherein between the first height portion and the second height portion the combined image is derived from both the first image and the second image.

6. The method of claim 1, further comprising:

interpolating the first image from the aerial vehicle and the second image from the terrestrial vehicle to generate a single image from a single perspective.

7. The method of claim 1, wherein the first image collected by the aerial vehicle depicts the terrestrial vehicle and the aerial vehicle tracks the terrestrial vehicle.

8. The method of claim 1, wherein the first image includes a first portion of a building faç

ade and the second image includes a second portion of a building faç

ade.

9. An apparatus for building a model through cooperation of movement of an aerial vehicle and a terrestrial vehicle, the apparatus comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to;

receive location data for a mobile device associated with the aerial vehicle or the terrestrial vehicle;

receive a first image collected by the aerial vehicle;

receive a second image collected by the terrestrial vehicle, wherein the aerial vehicle tracks and follows movement of the terrestrial vehicle;

determine a spatial relationship between the aerial vehicle and the terrestrial vehicle;

derive the model from a combination of the first image and the second image and the spatial relationship between the aerial vehicle and the terrestrial vehicle from the cooperation of movement of the aerial vehicle and the terrestrial vehicle; and

index the model from the combination of the first image and the second image in a geographic database based on the location data.

10. The apparatus of claim 9, wherein the first image and the second image are combined based on an orientation of the aerial vehicle.

11. The apparatus of claim 9, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:

receive first range data collected by the aerial vehicle and second range data collected by the terrestrial vehicle, wherein the combined image is based on the first range data and the second range data.

12. The apparatus of claim 9, wherein a first height portion of the combined image is derived from the first image and a second height portion of the combined image is derived from the second image.

13. The apparatus of claim 12, wherein between the first height portion and the second height portion, the combined image is derived from both the first image and the second image.

14. The apparatus of claim 9, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:

interpolate the first image from the aerial vehicle and the second image from the terrestrial vehicle to generate a single image from a single perspective.

15. The apparatus of claim 9, wherein the first image collected by the aerial vehicle depicts the terrestrial vehicle.

16. The apparatus of claim 9, wherein the first image includes a first portion of a building faç

ade and the second image includes a second portion of a building faç

ade.

17. A non-transitory computer readable medium including instructions that when executed are operable to:

receive location data for a mobile device associated with an aerial vehicle or a terrestrial vehicle;

receive a first image collected by the aerial vehicle;

receive a second image collected by the terrestrial vehicle, wherein the first image includes a first portion of a building faç

ade and the second image includes a second portion of the building faç

ade;

deriving a model of the building faç

ade from a combination of the first image and the second image from cooperation of movement of the aerial vehicle and the terrestrial vehicle;

store the model of the building faç

ade from the combination of the first image and the second image in a geographic database; and

index the model of the building faç

ade in the first image and the second image in the geographic database based on the location data.

18. The non-transitory computer readable medium of claim 17, including instructions that when executed are operable to:

identify a first time code of the first image;

identify a second time code of the second image; and

determine a time differential for a distance between the first image and the second image, wherein the first image and the second image are indexed in the geographic database based on the time differential.