Aerial image collection

• US 9,429,425 B2
• Filed: 03/05/2013
• Issued: 08/30/2016
• Est. Priority Date: 03/05/2013
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Tillmann, Marco
• Primary Examiner(s): SMITH, JELANI A
• Application Number:
  US13/785,796
• Publication Number:
  US 20140257595A1
• Time in Patent Office: 1,274 Days
• Field of Search: 244/175, 244/30, 340/435, 340/436, 340/438, 345/156, 345/419, 345/423, 345/629, 345/633, 345/672, 382/154, 382/195, 701/2, 701/3, 701/8, 701/50, 703/1, 703/2
• US Class Current: 1/1
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 10/17 Helicopters flying platforms B64U10/13
• CPC Class Codes: B64U 10/60 Tethered aircraft
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 11/08 the pictures not being supported in the same relative position as when they were taken
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

In one embodiment, an aerial collection system includes an image collection field vehicle that travels at street level and an image collection aerial vehicle that travels in the air above the street. The aerial vehicle collects image data including at least a portion of the field vehicle. The field vehicle includes a marker, which is identified from the collected image data. The marker is analyzed to determine an operating characteristic of the aerial vehicle. In one example, the operating characteristic in the marker includes information for a flight instruction for the aerial vehicle. In another example, the operating characteristic in the marker includes information for the three dimensional relationship between the vehicles. The three dimensional relationship is used to combine images collected from the air and images collected from the street level.

Claims

1. A method comprising:

receiving first panoramic image data of a terrestrial vehicle, wherein the image data is collected by an aerial vehicle;

identifying, by a processor, a marker on the terrestrial vehicle from the first panoramic image data;

analyzing, by the processor, the marker from the image data of the terrestrial vehicle to determine an operating characteristic of the aerial vehicle; and

controlling the aerial vehicle according to the operating characteristic, wherein the operating characteristic is based on a path of the terrestrial vehicle from a geographic database, wherein the operating characteristic includes an orientation of the aerial vehicle;

receiving a second panoramic image collected by the terrestrial vehicle; and

combining the first panoramic image and the second panoramic image based on the orientation of the aerial vehicle, wherein combining the first panoramic image and the second panoramic image comprises interpolating the first panoramic image and the second image according to a variable rate.

2. The method of claim 1, wherein the operating characteristic includes a flight instruction for the aerial vehicle.

3. The method of claim 2, wherein the flight instruction includes a height or a speed for the aerial vehicle.

4. The method of claim 1, wherein the variable rate varies according to elevation such that only the first panoramic image is used at a first height in a combined image and only the second panoramic image is used at a second height in the combined image, wherein the first height is greater than the second height.

5. The method of claim 1, wherein the marker is an augmented reality code or a quick response code.

6. The method of claim 1, wherein the marker is displayed on a display screen mounted on the terrestrial vehicle.

7. The method of claim 1, wherein the path of the terrestrial vehicle includes a tunnel, overpass, or other obstruction to the aerial vehicle.

8. The method of claim 1, wherein the operating characteristic is based on an environmental factor that impacts the aerial vehicle.

9. An apparatus comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to;

receive image data of a terrestrial vehicle having a marker, wherein the image data is collected by an aerial vehicle;

receive a street side image collected by the terrestrial vehicle; and

combine the street side image with the image data collected by the aerial vehicle, wherein the street side image and the image data collected by the aerial vehicle are combined according to geometry derived from the marker, wherein the marker is analyzed to determine an operating characteristic of the aerial vehicle to control the aerial vehicle; and

control the aerial vehicle according to the operating characteristic.

10. The apparatus of claim 9, wherein the operating characteristic includes a flight instruction for the aerial vehicle.

11. The apparatus of claim 10, wherein the flight instruction includes a height for the aerial vehicle.

12. The apparatus of claim 10, wherein the operating characteristic includes a camera angle for a camera mounted on the aerial vehicle.

13. The apparatus of claim 9, wherein the marker is an augmented reality code or a quick response code.

14. A non-transitory computer readable medium including instructions that when executed are operable to:

receive first image data collected by a terrestrial vehicle;

receive second image data of the terrestrial vehicle collected by an aerial vehicle;

calculate a spatial relationship between the terrestrial vehicle and the aerial vehicle based on the second image data, wherein a flight command for the aerial vehicle is calculated from the second image data; and

combine the first image data and the second image data according to the spatial relationship between the terrestrial vehicle and the aerial vehicle calculated from a marker in the second image data collected by the aerial vehicle, control the aerial vehicle according to the marker.

15. The non-transitory computer readable medium of claim 14, wherein the second image data includes data depicting the terrestrial vehicle and a marker.