APPARATUS AND METHOD FOR GENERATING AN OVERVIEW IMAGE OF A PLURALITY OF IMAGES USING A REFERENCE PLANE

• US 20120050525A1
• Filed: 08/23/2011
• Published: 03/01/2012
• Est. Priority Date: 08/25/2010
• Status: Active Grant

• Current Assignee: Lakeside Labs GmbH
• Original Assignee: Lakeside Labs GmbH
• Inventors: RINNER, Bernhard, QUARITSCH, Markus, WISCHOUNIG-STRUCL, Daniel, YAHYANEJAD, Saeed
• Granted Patent: US 8,902,308 B2
• Time in Patent Office: Days
• US Class Current: 348/117
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching

Abstract

An apparatus for generating an overview image of a plurality of images comprises a storage unit and an image processor. The storage unit stores a plurality of processed images of the overview image and is able to provide the overview image containing the plurality of processed images at their assigned positions for displaying. The image processor determines feature points of a new image and compares the determined feature points of the new image with feature points of a stored processed image to identify common feature points and to obtain 3-dimensional positions of the common feature points. Further, the image processor determines common feature points located within a predefined maximum distance of relevance to a reference plane based on the 3-dimensional positions of the common feature points to identify relevant common feature points. Further, the image processor processes the new image by assigning the new image to a position in the overview image based on a comparison of an image information of each relevant common feature point of the new image with an image information of each corresponding relevant common feature point of the stored processed image without considering common feature points located beyond the predefined maximum distance of relevance to the reference plane.

Claims

1. Apparatus for generating an overview image of a plurality of images, the apparatus comprising:

a storage unit configured to store a plurality of processed images of the overview image, wherein each processed image of the plurality of processed images is assigned to a position of the overview image, wherein the storage unit is configured to provide the overview image comprising the plurality of processed images at their assigned positions for displaying; and

an image processor configured to determine feature points of a new image and configured to compare the determined feature points of the new image with feature points of a stored processed image to identify common feature points and to acquire 3-dimensional positions of the common feature points, wherein the image processor is configured to determine common feature points located within a predefined maximum distance of relevance to a reference plane based on the 3-dimensional positions of the common feature points to identify relevant common feature points, wherein the image processor is configured to process the new image by assigning the new image to a position in the overview image based on a comparison of an image information of each relevant common feature point of the new image with an image information of each corresponding relevant common feature point of the stored processed image without considering common feature points located beyond the predefined maximum distance of relevance to the reference plane, wherein the storage unit is configured to add the processed new image with the assigned position to the plurality of processed images of the overview image.

2. Apparatus according to claim 1, wherein the image processor is configured to determine the reference plane based on the 3-dimensional positions of the common feature points.

3. Apparatus according to claim 2, wherein the image processor is configured to determine the reference plane by fitting a horizontal plane to the 3-dimensional positions of the common feature points, so that a maximal number of common feature points are located within a predefined maximum fitting distance to the reference plane or so that the reference plane is as far away as possible from a camera position, the new image was taken from, with at least one common feature point located within the predefined maximum fitting distance to the reference plane.

4. Apparatus according to claim 1, wherein the storage unit is configured to add the processed new image together with the determined feature points of the new image to the plurality of processed images.

5. Apparatus according to claim 1, wherein the image processor is configured to determine an overlap region of the new image and the stored processed image, and wherein the image processor is configured to compare determined feature points of the new image located within the overlap region with feature points of the stored image located within the overlap region, while determined feature points outside the overlap region are neglected for the identification of the common feature points.

6. Apparatus according to claim 1, wherein the image information of a relevant common feature point in the new image is a position of the relevant common feature point in the new image and the image information of a relevant common feature point in the stored image is a position of the relevant common feature point in the stored image, wherein the image processor is configured to assign the new image to a position in the overview image, so that the distance between a position of a relevant common feature point in the new image and a position of the corresponding relevant common feature point in the stored image is minimized or so that a sum of distances between the position of the relevant common feature points in the new image and the position of the respective corresponding relevant common feature points in the stored image is minimized.

7. Apparatus according to claim 1, wherein the image information of the relevant common feature point in the new image is an area of the new image around the relevant common feature point of a predefined size and the image information of a relevant common feature point in the stored image is an area of the stored image around the relevant common feature point of the predefined size, wherein the image processor is configured to assign the new image to a position in the overview image, so that the correlation value of an area around a relevant common feature point of the new image and an area around a corresponding relevant common feature point of the stored image is maximized or so that a sum of correlation values of areas around the relevant common feature points of the new image and areas around the corresponding relevant common feature points of the stored image is maximized.

8. Apparatus according to claim 6, wherein the image processor is configured to assign the new image to a position in the overview image based on the minimization of the distance or the sum of distances and configured to readjust the assignment of the new image to a position in the overview image based on the maximization of the correlation value or the sum of correlation values.

9. Apparatus according to claim 1, comprising an image preprocessor configured to preprocess the new image by assigning the new image to a position in the overview image based on a position information comprised by meta-data of the new image, wherein each image of the plurality of images of the overview image comprises associated meta-data, wherein the storage unit is configured to store a plurality of preprocessed or processed images of the overview image, wherein the image processor comprises an accuracy information input for receiving an accuracy information of the position information and a controllable processing engine, wherein the image processor is configured to determine an overlap region of the preprocessed new image and the stored processed image within the overview image based on the assigned position of the preprocessed new image and the assigned position of the stored processed image, wherein the controllable processing engine is configured to process the preprocessed new image by readjusting the assigned position of the preprocessed new image based on the comparison of the image information of each relevant common feature point of the preprocessed new image with the image information of each corresponding relevant common feature point of the stored processed image, wherein the controllable processing engine is controlled by an accuracy information of the position information received by the accuracy information input, so that a maximal readjustment of the assigned position of the preprocessed new image is limited based on the received accuracy information of the position information.

10. Apparatus according to claim 9, wherein the image preprocessor is configured to preprocess the new image by correcting an orientation and a perspective distortion of the new image based on a roll information, a pitch information and a yaw information, wherein the roll information, the pitch information and the yaw information is comprised by the meta-data of the new image.

11. Apparatus according to claim 9, wherein the storage unit is configured to add the preprocessed new image with the assigned position to the plurality of preprocessed or processed images, and wherein the storage unit is configured to provide the overview image comprising the preprocessed new image at the assigned position for displaying before the preprocessed new image is processed by the image processor.

12. Apparatus according to claim 9, wherein the image preprocessor is configured to receive successively a plurality of new images and configured to preprocess each received new image, wherein the storage unit is configured to store each preprocessed new image and configured to provide an updated overview image after storing a predefined number of preprocessed new images.

13. Apparatus according to claim 9, comprising an unmanned aerial vehicle control unit configured to generate a control signal for piloting an unmanned aerial vehicle to a region corresponding to an area of the overview image being not covered by an image of the plurality of preprocessed or processed images.

14. Method for generating an overview image of a plurality of images, comprising:

storing a plurality of processed images of the overview image, wherein each processed image of the plurality of processed images is assigned to a position in the overview image;

determining feature points of a new image;

comparing the determined feature points of the new image with feature points of a stored processed image to identify common feature points and to acquire 3-dimensional positions of the common feature points;

determining common feature points located within a predefined maximum distance of relevance to a reference plane based on the 3-dimensional positions of the common feature points to identify relevant common feature points;

processing a new image by assigning the new image to a position in the overview image based on a comparison of an image information of each relevant common feature point of the new image with an image information of each corresponding relevant common feature point of the stored processed image without considering common feature points located beyond the predefined maximum distance of relevance to the reference plane;

adding the new image with the assigned position to the plurality of processed images of the overview image;

providing the overview image comprising the plurality of processed images at their assigned positions for displaying.

15. A non-transitory computer readable medium including a computer program with a program code for performing a method, when the computer program runs on a computer or a micro controller, for generating an overview image of a plurality of images, comprising:

storing a plurality of processed images of the overview image, wherein each processed image of the plurality of processed images is assigned to a position in the overview image;

determining feature points of a new image;

comparing the determined feature points of the new image with feature points of a stored processed image to identify common feature points and to acquire 3-dimensional positions of the common feature points;

determining common feature points located within a predefined maximum distance of relevance to a reference plane based on the 3-dimensional positions of the common feature points to identify relevant common feature points;

processing a new image by assigning the new image to a position in the overview image based on a comparison of an image information of each relevant common feature point of the new image with an image information of each corresponding relevant common feature point of the stored processed image without considering common feature points located beyond the predefined maximum distance of relevance to the reference plane;

adding the new image with the assigned position to the plurality of processed images of the overview image;

providing the overview image comprising the plurality of processed images at their assigned positions for displaying.