Stereo image processing apparatus and method of processing stereo image

• US 6,970,593 B2
• Filed: 11/08/2001
• Issued: 11/29/2005
• Est. Priority Date: 11/22/2000
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Furukawa, Hisao
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): STREGE, JOHN B
• Application Number:
  US09/986,358
• Publication Number:
  US 20020061132A1
• Time in Patent Office: 1,482 Days
• Field of Search: 382/109, 382/113, 382/154, 382/168, 382/170, 382/172, 382/254, 382/284, 382/294, 345/419, 345/427, 348/42, 348/47, 701/208
• US Class Current: 382/154
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

To provide a stereo image processing device enabling sufficiently accurate three-dimensional data to be obtained from satellite stereo images and aerial stereo images automatically with no involvement of operators and for complicated objects such as buildings in urban areas. A satellite image storing unit stores satellite image data. A stereo processing unit subjects satellite image data obtained from the satellite image storing unit to stereo matching processing to generate three-dimensional data. A DEM data automatic correcting unit uses information of external shape of buildings and the like obtained from map data in a map data storing unit to correct automatically erroneous data such as noises and losses in the three-dimensional data obtained with the stereo processing unit. The map data storing unit provides map data such as external shapes of buildings and the like to the DEM data automatically correcting unit.

Claims

1. A stereo image processing apparatus subjecting first image data obtained from a first flying object (airframe) during a first time period to stereo matching processing to generate first three-dimensional data and subjecting second image data obtained from a second flying object during a second time period different from said first time period to stereo matching processing to generate second three-dimensional data, comprising:

data correcting means for correcting erroneous data corrected including at least noises and losses in inputted three-dimensional data by using information of external shapes of buildings obtained from map data, said data correcting means inputting said first three-dimensional data to provide first modified three-dimensional data and inputting said second three-dimensional data to provide second modified three-dimensional data;

map data storing means for storing said map data;

data storing means for storing said first modified three-dimensional data;

data comparing means for comparing said first modified three-dimensional data stored in said data storing means with said second modified three-dimensional data; and

map data modifying means for modifying said map data stored in said map data storing means based on differential information obtained from said data comparing means to provide modified two-dimensional map data.

2. The stereo image processing apparatus according to claim 1, wherein said data correcting means comprises:

registration means for superimposing said inputted three-dimensional data on said map data;

area data setting means for setting, for each building defining an area in the map data superimposed by said registration means, the inputted three-dimensional data included in the area as candidate three-dimensional data of each building;

in-area histogram analysis means for obtaining, in each area, statistic information from the candidate three-dimensional data included in the area; and

data modifying means for modifying, based on the statistic information in each area obtained by said in-area histogram analysis means, the inputted three-dimensional data included in the area.

3. The stereo image processing apparatus according to claim 1, further comprising:

image storing means for storing said first image data.

4. The stereo image processing apparatus according to claim 1, wherein said first flying object is an artificial satellite, and said first image data is satellite image data obtained from said artificial satellite.

5. The stereo image processing apparatus according to claim 1, wherein said first flying object is an aircraft, and said first image data is aircraft image data obtained from said aircraft.

13. The stereo image processing apparatus according to claim 2, further comprising:

image storing means for storing said first image data.

15. The stereo image processing apparatus of claim 1,

wherein said second flying object is the same as said first flying object.

6. A method of processing first stereo images in which first image data obtained from a first flying object during a first time period is subjected to stereo matching processing to generate first three-dimensional data and processing second stereo images in which second image data obtained from a second flying object during a second time period different from said first time period is subjected to stereo matching processing to generate second three-dimensional data, comprising:

correcting first erroneous data including at least noises and losses in said first three-dimensional data by using information of external shapes of buildings obtained from map data to provide first modified three-dimensional data;

correcting second erroneous data including at least noises and losses in said second three-dimensional data by using said information of external shapes of buildings obtained from said map data to provide second modified three-dimensional data;

comparing said first modified three-dimensional data with said second modified three-dimensional data to provide differential information; and

modifying said map data based on said differential information to provide modified two-dimensional map data.

7. The method according to claim 6, wherein said step of correcting first erroneous data comprises the steps of:

superimposing said first three-dimensional data on said map data;

setting, for each building defining an area in the superimposed map data, first three-dimensional data included in the area as candidate three-dimensional data of each building;

obtaining, for each area, statistic information from the candidate three-dimensional data included in the area; and

modifying, based on the statistic information in said each area, the first three-dimensional data included in the area to provide first modified three-dimensional data.

8. The method according to claim 6, wherein said first image data is obtained from image storing means for storing the first image data.

9. The method according to claim 6, wherein said map data is obtained from map storing means for storing the map data.

10. The method according to claim 6, wherein said first flying object is an artificial satellite, and said first image data is satellite image data obtained from said artificial satellite.

11. The method according to claim 6, wherein said first flying object is an aircraft, and said first image data is aircraft image data obtained from said aircraft.

16. The method of claim 6,

wherein said second flying object is the same as said first flying object.

12. A recording medium in which a program for processing stereo images is recorded for making a computer carry out a process, the process comprising:

subjecting first image data obtained from a first flying object during a first time period to stereo matching processing to provide first three-dimensional data;

subjecting second image data obtained from a second flying object during a second time period different from said first time period to stereo matching processing to provide second three-dimensional data;

correcting first erroneous data including at least noises and losses in the first three-dimensional data by using at least information of external shapes of buildings obtained from map data to provide first modified three-dimensional data;

correcting second erroneous data including at least noises and losses in said second three-dimensional data by using said information of external shapes of buildings obtained from said map data to provide second modified three-dimensional data;

comparing said first modified three-dimensional data with said second modified three-dimensional data to provide differential information; and

modifying said map data based on said differential information to provide modified two-dimensional map data.

17. The recording medium of claim 12,

wherein said second flying object is the same as said first flying object.

14. A method for correcting two-dimensional map data, comprising:

subjecting first image data obtained from a first flying object during a first time period to stereo matching processing to provide first three-dimensional data;

subjecting second image data obtained from a second flying object during a second time period different from the first time period to stereo matching processing to provide second three-dimensional data;

correcting the first three-dimensional data based on the two-dimensional map data to provide first modified three-dimensional data;

correcting the second three-dimensional data based on the two-dimensional map data to provide second modified three-dimensional data;

comparing the first modified three-dimensional data with the second modified three-dimensional data to provide differential information; and

correcting the two-dimensional map data based on the differential information to provide modified two-dimensional map data.

18. The method of claim 14,

wherein said second flying object is the same as said first flying object.