Arrangement and method for providing a three dimensional map representation of an area

• US 9,094,673 B2
• Filed: 07/04/2007
• Issued: 07/28/2015
• Est. Priority Date: 07/04/2007
• Status: Active Grant

• Current Assignee: Saab AB
• Original Assignee: Saab AB
• Inventors: Isaksson, Folke, Lundberg, Kristian
• Primary Examiner(s): Shaawat, Mussa A
• Assistant Examiner(s): Stroud, James E
• Application Number:
  US12/667,560
• Publication Number:
  US 20100250125A1
• Time in Patent Office: 2,946 Days
• Field of Search: 701/400, 701/420, 701/421, 701/422, 701/423, 701/1, 701/408, 701/409, 701/410, 701/412, 701/417, 701/430, 701/436, 701/445, 701/448, 345/419, 345/428, 345/618, 345/619, 345/623, 345/624, 345/629, 345/632, 345/640, 356/16
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30188 Vegetation; Agriculture
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

An arrangement and a method for providing a three dimensional map representation of an area. The arrangement includes an image generating unit and a processing unit. The image generating unit is arranged to generate time recorded images. The image generating unit is arranged to provide a plurality of at least partly overlapping images each covering at least a part of the area. The processing unit is arranged to stereo process an arbitrary number of at least partly overlapping image sets generated by the image generating unit so as to provide the three dimensional map representation. A navigation unit is arranged to output time recorded navigation states related to the image generating unit. The processing unit is arranged to, for each time recorded image to be stereo processed, associate the navigation states relating to a corresponding time record.

Claims

1. An arrangement comprising an image generating unit, a navigation unit and processing unit for generating a three dimensional map representation of an area, the arrangement comprising:

an image generating unit arranged to generate time recorded images, said image generating unit being arranged to provide a plurality of at least partly overlapping images each covering at least a part of said area, a navigation unit arranged to output time recorded navigation states of the image generating unit as the images are generated, wherein the time recorded navigation states comprises a three-dimensional position vector and a three-dimensional attitude vector and optionally a three-dimensional velocity vector, and a processing unit arranged to, based on said navigation states, stereo process an arbitrary number of the plurality of at least partly overlapping image sets generated by said image generating unit so as to provide said three dimensional map representation, wherein the processing unit is arranged to, for each time recorded image to be stereo processed, associate each pixel to a corresponding one of the navigation states relating to a corresponding time record and to perform the stereo processing based on the associated navigation states so that all pixels in the map representation are specified in three geographical dimensions associated with a geographical coordinate on the ground, wherein the processing unit is arranged to form a plurality of three dimensional map representation parts each corresponding to a unique image set and to generate the three dimensional map representation of the area based upon the three dimensional map representation parts.

2. The arrangement according to claim 1, wherein the image generating unit is mountable on a movable carrier.

3. The arrangement according to claim 1, wherein the navigation unit comprises an inertial navigation system.

4. The arrangement according to claim 1, wherein the navigation unit comprises provides position data in a predetermined coordinate system and relates each pixel to a point in the predetermined coordinate system.

5. The arrangement according to claim 1, wherein the processing unit is arranged to determine a relation between the images in each image set based on the navigation states associated to the respective images so as to provide matching between the images.

6. The arrangement according to claim 1, wherein the processing unit is arranged to transform each map representation part based on height data so as to perform ortho projection.

7. The arrangement according to claim 1, wherein the arrangement is arranged to form the three dimensional map representation in real time.

13. The arrangement according to claim 2, wherein the movable carrier comprises a land vehicle, satellite, aircraft, watercraft, or a person.

14. The arrangement according to claim 4, wherein the means for providing position data in a predetermined coordinate system comprises a GPS receiver or a map.

18. The arrangement according to claim 1, wherein the navigation unit calculates motions of the image generating unit, continuously calculates the position and attitude of the image generating unit, and is mounted in known relation to image generating unit.

19. The arrangement according to claim 1, wherein the navigation states comprise a three dimensional position vector and a three dimensional attitude vector in the predefined coordinate system.

20. The arrangement according to claim 1, wherein the navigation states comprise a three-dimensional velocity vector in a predefined coordinate system.

22. The method according to claim 19, further comprising:

performing volume determinations utilizing the difference map representation.

23. The arrangement according to claim 4, wherein the navigation unit relates each pixel to a point in the predetermined coordinate system.

8. A platform comprising an image generating unit, a navigation unit and processing unit for generating a three dimensional map representation of an area, comprising:

an arrangement for providing a three dimensional map representation of an area, the arrangement comprising;

an image generating unit arranged to generate time recorded images, said image generating unit being arranged to provide a plurality of at least partly overlapping images each covering at least a part of said area, a navigation unit arranged to output time recorded navigation states of the image generating unit as the images are generated, wherein the time recorded navigation states comprises a three-dimensional position vector and a three-dimensional attitude vector and optionally a three-dimensional velocity vector, and a processing unit arranged to, based on said navigation states, stereo process an arbitrary number of the plurality of at least partly overlapping image sets generated by said image generating unit so as to provide said three dimensional map representation, wherein the processing unit is arranged to, for each time recorded image to be stereo processed, associate each pixel to a corresponding one of the navigation states relating to a corresponding time record and to perform the stereo processing based on the associated navigation states so that all pixels in the map representation are specified in three geographical dimensions associated with a geographical coordinate on the ground, wherein the processing unit is arranged to form a plurality of three dimensional map representation parts each corresponding to a unique image set and to generate the three dimensional map representation of the area based upon the three dimensional map representation parts.

9. The platform according to claim 8, wherein the platform is chosen from a group comprising a land vehicle, satellite, and aircraft watercraft, or wherein the arrangement is configured to be arranged on a person.

15. The platform according to claim 9, wherein the platform comprises a lorry, airplane, ship, or submarine.

10. A method for generating a three dimensional map representation of an area with an image generating unit, a navigation unit and processing unit, the method comprising:

generating time recorded images with an image generating unit, generating with said image generating unit a plurality of at least partly overlapping images each covering at least a part of said area, generating with said image generating unit an arbitrary number of at least partly overlapping image sets from said provided at least partly overlapping images, outputting with a navigation unit time recorded navigation states comprising a three-dimensional position vector and a three-dimensional attitude vector and optionally a three-dimensional velocity vector of the image generating unit as the images are generated, and stereo processing with a processing unit said image sets based on the navigational states so as to provide said three dimensional map representation, wherein for each time recorded image to be stereo processed the navigation states relating to a corresponding time record are associated with the image, said associating comprising correlating each pixel of each time recorded image to a corresponding navigation state, and wherein the stereo processing is performed based on the associated navigation states so that all pixels in the map representation are specified in three geographical dimensions associated with a geographical coordinate on the ground, wherein the stereo processing comprises forming a plurality of three dimensional map representation parts each corresponding to a unique image set and generating the three dimensional map representation of the area based upon the three dimensional map representation parts.

16. The method according to claim 10, further comprising:

forming a difference map representation based on a relation between heights in two three-dimensional map representations of corresponding areas.

17. The method according to claim 16, further comprising:

utilizing the difference map representation for performing volume determinations.

21. The method according to claim 10, further comprising:

forming a difference map representation based on a relation between heights in two three-dimensional map representations of corresponding areas.

11. A computer program product configured to generate a three dimensional map representation of an area with an image generating unit, a navigation unit and processing unit, comprising:

a non-transitory computer readable medium; and

a programme code recorded on the computer readable medium for performing a method for generating a three dimensional map representation of an area when said computer programme is run on a computer, the method comprising generating time recorded images with an image generating unit, generating with said image generating unit a plurality of at least partly overlapping images each covering at least a part of said area, generating with said image generating unit an arbitrary number of at least partly overlapping image sets from said provided at least partly overlapping images, outputting with a navigation unit time recorded navigation states comprising a three-dimensional position vector and a three-dimensional attitude vector and optionally a three-dimensional velocity vector of the image generating unit as the images are generated, and stereo processing with a processing unit said image sets based on the navigational states so as to provide said three dimensional map representation, wherein for each time recorded image to be stereo processed the navigation states relating to a corresponding time record are associated with the image, said associating comprising correlating each pixel of each time recorded image to a corresponding navigation state, and wherein the stereo processing is performed based on the associated navigation states so that all pixels in the map representation are specified in three geographical dimensions associated with a geographical coordinate on the ground, wherein the stereo processing comprises forming a plurality of three dimensional map representation parts each corresponding to a unique image set and generating the three dimensional map representation of the area based upon the three dimensional map representation parts.

12. A non-transitory computer readable medium having embodied thereon a computer program for processing by a computer to generate a three dimensional map representation of an area with an image generating unit, a navigation unit and processing unit, the computer program comprising:

a first code segment for generating time recorded images with an image generating unit; and

a second code segment for providing a plurality of at least partly overlapping images with the image generating unit, a fourth code segment for outputting with a navigation unit time recorded navigation states of the image generating unit as the images are recorded, the outputting time recorded navigation states comprising a three-dimensional position vector and a three-dimensional attitude vector and optionally a three-dimensional velocity vector of the image generating unit as the images are generated, and a fifth code segment for associating for each generated time recorded image the navigation states relating to a corresponding time record by correlating each pixel of each time recorded image to a corresponding navigation state with a navigation unit, and a third code segment for stereo processing with a processing unit, based on said associated navigation states, an arbitrary number of overlapping image sets generated by said image generating unit so as to provide a three dimensional map representation wherein all pixels are specified in three geographical dimensions associated with a geographical coordinate on the ground, wherein the stereo processing comprises forming a plurality of three dimensional map representation parts each corresponding to a unique image set and generating the three dimensional map representation of the area based upon the three dimensional map representation parts.