APPARATUS AND METHOD FOR GENERATING AN OVERVIEW IMAGE OF A PLURALITY OF IMAGES USING AN ACCURACY INFORMATION

• US 20120050524A1
• Filed: 08/23/2011
• Published: 03/01/2012
• Est. Priority Date: 08/25/2010
• Status: Active Grant

• Current Assignee: Lakeside Labs GmbH
• Original Assignee: Lakeside Labs GmbH
• Inventors: RINNER, Bernhard, QUARITSCH, Markus, WISCHOUNIG-STRUCL, Daniel, YAHYANEJAD, Saeed
• Granted Patent: US 8,797,400 B2
• Time in Patent Office: Days
• US Class Current: 348/117
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

An apparatus for generating an overview image of a plurality of images includes an image preprocessor which preprocesses a new image by assigning the new image to a position in the overview image based on position information contained by meta-data of the new image. A storage unit stores a plurality of images of the overview image and provides the overview image for display. Further, the image processor receives accuracy information of the position information. The image processor determines an overlap region of the preprocessed new image and a stored image within the overview image based on the assigned positions of the preprocessed new image and of the stored image. Further, a controllable processing engine processes the preprocessed new image by re-adjusting the assigned position of the preprocessed new image based on comparing features of the overlap region of the preprocessed new image and the stored image. The controllable processing engine is controlled by accuracy information of the position information.

Claims

1. An apparatus for generating an overview image of a plurality of images, wherein each image of the plurality of images comprises associated meta-data, the apparatus comprising:

an image preprocessor configured to preprocess a new image by assigning the new image to a position in the overview image based on a position information comprised by the meta-data of the new image;

a storage unit configured to store a plurality of preprocessed or processed images of the overview image, wherein each preprocessed or processed image of the plurality of preprocessed or processed images is assigned to a position in the overview image, wherein the storage unit is configured to provide the overview image containing the plurality of preprocessed or processed images at their assigned positions for displaying; and

an image processor comprising an accuracy information input for receiving an accuracy information of the position information and a controllable processing engine, wherein the image processor is configured to determine an overlap region of the preprocessed new image and a stored preprocessed or stored processed image within the overview image based on the assigned position of the preprocessed new image and the assigned position of the stored preprocessed or stored processed image,

wherein the controllable processing engine is configured to process the preprocessed new image by re-adjusting the assigned position of the preprocessed new image based on a comparison of features of the overlap region of the preprocessed image and features of the overlap region of the stored preprocessed or stored processed image, wherein the controllable processing engine is controlled by an accuracy information of the position information received by the accuracy information input, so that a maximal re-adjustment of the assigned position of the preprocessed new image is limited based on the received accuracy information of the position information,

wherein the storage unit is configured to add the processed new image with the re-adjusted assigned position to the plurality of preprocessed or processed images.

2. The apparatus according to claim 1, wherein the image preprocessor is configured to preprocess the new image by correcting an orientation and a perspective distortion of the new image based on a roll information, a pitch information and a yaw information, wherein the roll information, the pitch information and the yaw information is comprised by the meta-data of the new image.

3. The apparatus according to claim 1, wherein the controllable processing engine is configured to process the preprocessed new image by re-adjusting the assigned position, an orientation or a perspective distortion of the preprocessed new image based on the comparison of the features of the overlap region, wherein the controllable processing engine is controlled by an accuracy information of a roll information, a pitch information or a yaw information received by the accuracy information input, so that the maximal re-adjustment of the assigned position of the processed new image, a maximal re-adjustment of the orientation or the perspective distortion of the preprocessed new image is limited based on the accuracy information of the roll information, the pitch information or the yaw information.

4. The apparatus according to claim 1, wherein the storage unit is configured to add the preprocessed new image with the assigned position to the plurality of preprocessed or processed images, and wherein the storage unit is configured to provide the overview image containing the preprocessed new image at the assigned position for displaying before the preprocessed new image is processed by the image processor.

5. The apparatus according to claim 4, wherein the storage unit is configured to add the processed new image by replacing the stored preprocessed new image by the processed new image, and wherein the storage unit is configured to provide the overview image containing the processed new image at the re-adjusted assigned position for displaying.

6. The apparatus according to claim 1, wherein the image preprocessor is configured to receive successively a plurality of new images and configured to preprocess each received new image, wherein the storage unit is configured to store each preprocessed new image and configured to provide an updated overview image after storing a predefined number of preprocessed new images.

7. The apparatus according to claim 1, comprising an unmanned aerial vehicle control unit configured to generate a control signal for piloting an unmanned aerial vehicle to a region corresponding to an area of the overview image being not covered by an image of the plurality of preprocessed or processed images.

8. The apparatus according to claim 1, wherein the image processor is configured to determine feature points of the preprocessed new image and configured to compare the determined feature points of the preprocessed new image with feature points of a stored processed image to identify common feature points and to achieve 3-dimensional positions of the common feature points, wherein the image processor is configured to determine common feature points located within a predefined maximum distance of relevance to a reference plane based on the 3-dimensional positions of the common feature points to identify relevant common feature points, wherein the image processor is configured to re-adjust the position of the preprocessed new image in the overview image based on a comparison of an image information of each relevant common feature point of the preprocessed new image with an image information of each corresponding relevant common feature point of the stored processed image without considering common feature points located beyond the predefined maximum distance of relevance to the reference plane.

9. The apparatus according to claim 8, wherein the image processor is configured to determine the reference plane based on the 3-dimensional positions of the common feature points.

10. The apparatus according to claim 8, wherein the storage unit is configured to add the new processed image together with the determined feature points of the new processed image to the plurality of preprocessed or processed images.

11. An unmanned aerial vehicle comprising:

a camera configured to take an image during a flight of the unmanned aerial vehicle;

a sensor system configured to determine a position of the unmanned aerial vehicle at a time the image is taken to acquire a position information associated to the taken image, wherein the sensor system is configured to determine a yaw, a nick or a roll of the unmanned aerial vehicle at a time, the image is taken, to acquire a yaw information, a nick information or a roll information associated to the taken image;

an accuracy determiner configured to determine an accuracy of the determination of the position of the unmanned aerial vehicle to acquire an accuracy information associated to the determined position information, wherein the accuracy determiner is configured to determine an accuracy of the determination of the yaw, the nick or the roll of the unmanned aerial vehicle to acquire an accuracy information associated to the determined yaw information, the nick information or the roll information;

a transmitter configured to transmit the taken image together with associated meta-data containing the position information of the taken image and the accuracy information of the position information, wherein the meta-data further contains the yaw information, the nick information, the roll information, the accuracy information of the yaw information, the accuracy information of the nick information or the accuracy information of the roll information.

12. A method for providing an image together with associated meta-data taken by an unmanned aerial vehicle, comprising:

taking an image during a flight of the unmanned aerial vehicle;

determining a position of the unmanned aerial vehicle at a time the image is taken to acquire a position information associated to the taken image;

determining a yaw, a nick or a roll of the unmanned aerial vehicle at a time, the image is taken, to acquire a yaw information, a nick information or a roll information associated to the taken image;

determining an accuracy of the determination of the position of the unmanned aerial vehicle to acquire an accuracy information associated to the determined position information;

determining an accuracy of the determination of the yaw, the nick or the roll of the unmanned aerial vehicle to acquire an accuracy information associated to the determined yaw information, the nick information or the roll information;

transmitting the taken image together with associated meta-data containing the position information of the taken image and the accuracy information of the position information, wherein the meta-data further contains the yaw information, the nick information, the roll information, the accuracy information of the yaw information, the accuracy information of the nick information or the accuracy information of the roll information.

13. A method for generating an overview image of a plurality of images, wherein each image of the plurality of images comprises associated meta-data, the method comprising:

preprocessing a new image by assigning the new image to a position in the overview image based on a position information comprised by the meta-data of the new image;

storing a plurality of preprocessed or processed images of the overview image, wherein each preprocessed or processed image of the plurality of preprocessed or processed images comprises an assigned position in the overview image;

determining an overlap region of the preprocessed new image and a stored preprocessed or stored processed image within the overview image based on the assigned position of the preprocessed new image and the assigned position of the stored preprocessed or stored processed image;

processing the preprocessed image by re-adjusting the assigned position of the preprocessed new image based on a comparison of features of the overlap region of the preprocessed new image and the stored preprocessed or stored processed image, wherein a maximal re-adjustment of the assigned position of the preprocessed new image is limited based on an accuracy information of the position information;

adding the processed new image with the re-adjusted assigned position to the plurality of preprocessed or processed images; and

providing the overview image containing the plurality of preprocessed or processed images at their assigned positions for displaying.

14. The method according to claim 13, further comprising:

adding the preprocessed new image to the plurality of preprocessed or processed images of the overview image;

displaying the overview image containing a preprocessed new image at the assigned position; and

displaying the overview image containing the processed new image at the re-adjusted assigned position.

15. A non-transitory computer readable medium including a computer program with a program code for performing a method, when the computer program runs on a computer or micro controller, for providing an image together with associated meta-data taken by an unmanned aerial vehicle, said method comprising:

taking an image during a flight of the unmanned aerial vehicle;

determining a position of the unmanned aerial vehicle at a time the image is taken to acquire a position information associated to the taken image;

determining a yaw, a nick or a roll of the unmanned aerial vehicle at a time, the image is taken, to acquire a yaw information, a nick information or a roll information associated to the taken image;

determining an accuracy of the determination of the position of the unmanned aerial vehicle to acquire an accuracy information associated to the determined position information;

determining an accuracy of the determination of the yaw, the nick or the roll of the unmanned aerial vehicle to acquire an accuracy information associated to the determined yaw information, the nick information or the roll information;

transmitting the taken image together with associated meta-data containing the position information of the taken image and the accuracy information of the position information, wherein the meta-data further contains the yaw information, the nick information, the roll information, the accuracy information of the yaw information, the accuracy information of the nick information or the accuracy information of the roll information.

16. A non-transitory computer readable medium including a computer program with a program code for performing a method, when the computer program runs on a computer or micro controller, for generating an overview image of a plurality of images, wherein each image of the plurality of images comprises associated meta-data, said method comprising:

preprocessing a new image by assigning the new image to a position in the overview image based on a position information comprised by the meta-data of the new image;

storing a plurality of preprocessed or processed images of the overview image, wherein each preprocessed or processed image of the plurality of preprocessed or processed images comprises an assigned position in the overview image;

determining an overlap region of the preprocessed new image and a stored preprocessed or stored processed image within the overview image based on the assigned position of the preprocessed new image and the assigned position of the stored preprocessed or stored processed image;

processing the preprocessed image by re-adjusting the assigned position of the preprocessed new image based on a comparison of features of the overlap region of the preprocessed new image and the stored preprocessed or stored processed image, wherein a maximal re-adjustment of the assigned position of the preprocessed new image is limited based on an accuracy information of the position information;

adding the processed new image with the re-adjusted assigned position to the plurality of preprocessed or processed images; and

providing the overview image containing the plurality of preprocessed or processed images at their assigned positions for displaying.