Method of recognizing an object in an image using multi-sensor integration through conditionally optimal geoscene generation and registration

• US 7,899,272 B1
• Filed: 10/19/2005
• Issued: 03/01/2011
• Est. Priority Date: 03/07/2005
• Status: Active Grant

• Current Assignee: Shin-Yi Hsu
• Original Assignee: Shin-Yi Hsu
• Inventors: Hsu, Shin-yi
• Primary Examiner(s): Bhatnagar; Anand
• Assistant Examiner(s): CHU, RANDOLPH I
• Application Number:
  US11/253,523
• Time in Patent Office: 1,959 Days
• Field of Search: 382/284, 382/113, 382/201
• US Class Current: 382/294
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A method of recognizing an object in an image is provided using multi-sensor integration through conditionally optimal geo-scene generation and registration. At least two images, one of which is a conditionally optimum, ortho-rectified base image, are input and used to generate a geoscene using ground control points in a latitude-longitude geospatial domain. Georegistration of the geoscene produces a registered geoimage which may be output. A virtual geospatial information system database may be compiled from the georegistered images. A Virtual Transverse Mercator (VTM) projection is defined which allows processing of images falling on both sides of the equator or across traditional UTM boundaries. The georegistration process utilizes the union and the intersection of image pixels, and geooverlaying with interacting layers including geogrids and text layers, to define main body and background pixels to facilitate object recognition.

Claims

1. A method of generating a virtual geospatial information system (VGIS) database, the steps comprising:

a) inputting at least two images, one of said images being a base image for georegistration;

b) generating at least one geoscene using at least two ground control points in a lat-long geospatial domain;

c) georegistering said at least one geoscene in said lat-long geospatial domain to produce a registered geoimage; and

d) outputting a geoimage comprising at least one of a non-georegistered image, a registered geoimage, a geomosaic, a geogridded geoimage, a geooverlayed geoscene, a geomasked geoscene, a geoindex geoscene, a geocontour, a geocontour interval, a geo-3D-fly-through geoscene, and a geo-4D-fly-through geoscene;

whereby a pixel in said at least one geoscene is represented by;

i) (x,y) coordinates in the image domain;

ii) (z) coordinate in the spectral and elevation/height domains;

iii) UTM representation in the geospatial domain;

iv) latitude/longitude in the geospatial domain; and

v) Virtual Transverse Mercator.

2. The method of generating a VGIS database as recited in claim 1, wherein said base image comprises a conditionally optimal base image (COBI).

3. The method of generating a VGIS database as recited in claim 2, wherein said COBI comprises an ortho-rectified image.

4. The method of generating a VGIS database as recited in claim 1, wherein said generating step (b) comprises using at least one of a UTM and a VTM projection.

5. The method of generating a VGIS database as recited in claim 1, wherein said generating step (b) comprises performing an Affine transform for coordinate transformation.

6. The method of generating a VGIS database as recited in claim 1, wherein said georegistering step (c) comprises locating at least one of:

the intersection, and the union of pixels in said at least two input images, said intersected and said unioned pixels defining a main body of said registered geoimage.

7. The method of generating a VGIS database as recited in claim 6, wherein pixels other than pixels of said main body are background pixels.

8. The method of generating a VGIS database as recited in claim 1, wherein said georegistering step (c) comprises at least one of the processes:

geogridding, geooverlaying, and geomasking.

9. The method of generating a VGIS database as recited in claim 1, wherein said georegistering step (c) is performed in at least two stages.

10. The method of generating a VGIS database as recited in claim 1, wherein a geogrid associated with said geogridded geoimage comprises text.

11. The method of generating a VGIS database as recited in claim 1, wherein said registered geoimage comprises geoscene characteristic information.

12. The method of generating a VGIS database as recited in claim 11, wherein said geoscene characteristic information comprises at least one of:

a scene dimension, pixel resolution in at least one of an x-scale and a y-scale, the central meridian, the geoscene used, degrees of offset from the north, degrees of offset from the east, a UTM meridian, and a VTM meridian.

13. The method of generating a VGIS database as recited in claim 4, wherein said VTM projection comprises a local central meridian determined from a distribution of geolocations of pixels in at least one of said at least two input images.

14. The method of generating a VGIS database as recited in claim 13, wherein at least one of said generating step (b) and said georegistering step (c) utilize a VTM system whereby said geoscene generation and said georegistration may be performed on an image having pixels on both the north side and the south side of the equator.

15. The method of generating a VGIS database as recited in claim 1, wherein said generating step (b) comprises at least one of the sub-steps:

geolocation, geomosaicking, geogridding, geooverlaying, geomasking, geoindexing, geocontour, geocontour interval, geo-3D-fly-through, and geo-4D-fly-through.

16. The method of generating a VGIS database as recited in claim 1, wherein said outputting step (d) comprises normalizing a registered image such that a background pixel is assigned a tone value of zero and a minimum tone value for said registered image is one.

17. The method of generating a VGIS database as recited in claim 1, wherein one of said at least two images comprises at least one of:

map data, a geomask, a digital elevation model (DEM), a geocontour interval, and signal data.

18. The method of generating a VGIS database as recited in claim 17, wherein said map data, DEM, geomask, geocontour interval, and signal data are provided by a source independent from a source of others of said at least two images.

47. The method of generating a VGIS database as recited in claim 15, wherein said geoindexing function comprises generating a color composite geomosaic from an output of a geomasking analysis.

48. The method of generating a VGIS database as recited in claim 15, wherein said geoindexing further comprises generating a feature attribute table corresponding to said color composite.

49. The method of generating a VGIS database as recited in claim 15, wherein said geo-3D-fly-through and said geo-3D-fly-through functions each utilize terrain elevation data from at least one of the sources:

full-range digital elevation model (DEM) data, contour interval based elevation data, an unsorted spectral-based terrain feature geoscene, a sorted spectral-based terrain feature geoscene, a terrain-elevation constrained spectral based terrain feature geoscene, a geomasked geoscene, a map-based geoscene, a signal-based geoscene, and a geoindexed geoscene.

19. A method of generating a virtual geospatial information system (VGIS) database, the steps comprising:

a) defining a virtual geoquad (VGQ) system for uniquely identifying geographic regions comprising 7.5 minutes latitude by 7.5 minutes longitude;

b) assigning an identification code to each VGQ;

c) inputting at least two images, one of said images being a base image for georegistration;

d) generating at least one geoscene comprising at least one of the sub-sets;

geolocation, geomosaicking, geogridding, geooverlaying, geomasking, geoindexing by generating a color composite geomosaic from an output of a geomasking analysis, geocontour, geocontour interval, geo-3D-fly-through, and geo-4D-fly-through, said at least one geoscene using at least two ground control points in a lat-long geospatial domain, and wherein geoindexing further comprises generating a feature attribute table corresponding to said color composite;

e) using said unique identification code, georegistering said at least one geoscene in said lat-long geospatial domain to produce a registered geoimage; and

f) outputting a geoimage;

whereby a pixel in said at least one geoscene is represented by;

i) (x,y) coordinates in the image domain;

ii) (z) coordinate in the spectral and elevation/height domains;

iii) UTM representation in the geospatial domain;

iv) latitude/longitude in the geospatial domain; and

v) Virtual Transverse Mercator.

20. The method of generating a VGIS database as recited in claim 19, wherein a VGQ defined in said defining step (a) comprises a VGQ having origins defined at least one of:

the equator, and the Greenwich Central Meridian.

21. The method of generating a VGIS database as recited in claim 20, wherein said VGQ is identified by a numeric code, a value of 0001_—

0001 defining a first possible VGQ and 2880_—

2880 defining a last VGQ.

22. The method of generating a VGIS database as recited in claim 20, wherein each of said VGQs is divided into four geoquarterquads, each of said geoquarterquads being identified by at least one of:

a numeric designation, and a textual designation, and a combination alpha-numeric designation.

23. The method of generating a VGIS database as recited in claim 20, wherein said generating step (d) comprises the sub-step of identifying all of said input images belonging to a particular one of said geoquads.

24. The method of generating a VGIS database as recited in claim 19, wherein said georegistering step (e) comprises locating at least one of:

the intersection, and the union of pixels in said at least two input images, said intersected and said unioned pixels defining a main body of said registered geoimage, wherein pixels other than said pixels of said main body being background pixels.

25. The method of generating a VGIS database as recited in claim 19, wherein said georegistering step (e) is performed in at least two stages.

26. The method of generating a VGIS database as recited in claim 19, wherein said registered geoimage comprises geoscene characteristic information.

27. The method of generating a VGIS database as recited in claim 26, wherein said geoscene characteristic information comprises at least one of:

a scene dimension, pixel resolution in at least one of an x-scale and a y-scale, the central meridian, the geoscene used, degrees of offset from the north, degrees of offset from the east, a geoelevation, a geocontour, a geocontour interval, a UTM meridian, and a VTM meridian.

28. The method of generating a VGIS database as recited in claim 27, wherein said VTM projection comprises a local central meridian determined from a distribution of geolocations of pixels in at least one of said at least two input images.

29. The method of generating a VGIS database as recited in claim 22, wherein at least one of said generating step (d) and said georegistering step (e) utilize a VTM system whereby said geoscene generation and said georegistration may be performed on an image having pixels on both the north side and the south side of the equator.

30. The method of generating a VGIS database as recited in claim 22, wherein said outputting step (f) comprises normalizing a registered image such that a background pixel is assigned a tone value of zero and a minimum tone value for said registered image is one.

31. The method of generating a VGIS database as recited in claim 19, wherein one of said at least two images comprises at least one of:

map data, a geomask, a DEM, a geocontour interval, and signal data.

32. The method of generating a VGIS database as recited in claim 31, wherein said map data, said geomask, said DEM, said geocontour interval, and said signal data are each provided by a source independent from a source of others of said at least two images.

33. A method of generating a virtual geospatial information system (VGIS) database, the steps comprising:

a) defining a Virtual Transverse Mercator (VTM) projection;

b) inputting at least two images, one of said images being a base image for georegistration;

c) generating at least one geoscene using at least two ground control points in a lat-long geospatial domain defined using at least one of said VTM projection and a Universal Transverse Mercator (UTM) projection;

d) using at least one of said VTM and UTM projections, performing at least one of the functions;

geoscene generation, georegistration, geolocation, geomosaicking, geooverlaying, geogridding, geomasking, geoindexing, geocontouring, geocontour intervaling, geo-3D-fly-through, geo-4D-fly-through to produce a registered geoimage; and

e) outputting a geoimage;

whereby a pixel in said at least one geoscene is represented by;

i) (x,y) coordinates in the image domain;

ii) (z) coordinate in the spectral and elevation/height domains;

iii) UTM representation in the geospatial domain;

iv) latitude/longitude in the geospatial domain; and

v) Virtual Transverse Mercator.

34. The method of generating a VGIS database as recited in claim 33, wherein in at least one of said generating step (c) and said performing step (d), only said UTM projection is used when all pixels in all of said at least two images fall on a single side of the equator and fall within a single UTM zone.

35. The method of generating a VGIS database as recited in claim 33, wherein said generating step (c) comprises performing an Affine transform for coordinate transformation.

36. The method of generating a VGIS database as recited in claim 33, wherein said performing step (d) comprises locating at least one of:

intersecting, unioning, geogridding, geooverlaying, geomasking, geoindexing, geocontouring, and geocontour-intervaling of pixels in said at least two input images to define a main body of said registered geoimage.

37. The method of generating a VGIS database as recited in claim 36, wherein pixels other than pixels of said main body are background pixels.

38. The method of generating a VGIS database as recited in claim 33, wherein said performing step (d) is performed in at least two stages using at least one of said UTM and said VTM projections.

39. The method of generating a VGIS database as recited in claim 33, wherein said registered geoimage comprises geoscene characteristic information defined in at least one of said UTM and said VTM projections.

40. The method of generating a VGIS database as recited in claim 33, wherein said geoscene characteristic information comprises at least one of:

a scene dimension, pixel resolution in at least one of an x-scale and a y-scale, the central meridian, the geoscene used, degrees of offset from the north, degrees of offset from the east, a geogrid, geotext, a geoindex, a geocontour, and a geocontour interval for at least one of said UTM and said VTM projections, a UTM meridian, a VTM meridian.

41. The method of generating a VGIS database as recited in claim 33, wherein said VTM projection comprises a local central meridian determined from a distribution of geolocations of pixels in at least one of said at least two input images.

42. The method of generating a VGIS database as recited in claim 33, wherein at least one of said generating step (c) and said performing step (d) utilizes a VTM system whereby said functions of geoscene generation, georegistration, geolocation, geomosaicking, geooverlaying, geogridding, geomasking, geoindexing, geocontouring, geocontour intervaling, geo-3D fly-through, and geo-4D fly-through geoscene generation and georegistration may be performed on an image having pixels on both the north side and the south side of the equator.

43. The method of generating a VGIS database as recited in claim 33, wherein said outputting step (e) comprises normalizing a registered image such that a background pixel is assigned a tone value of zero and a minimum tone value for said registered image is one.

44. The method of generating a VGIS database as recited in claim 33, wherein one of said at least two images comprises at least one of:

map data, a geomask, a DEM, a geocontour, a geocontour interval, and signal data in at least one of said UTM and said VTM projections.

45. The method of generating a VGIS database as recited in claim 33, wherein said map data, geomask, DEM, geocontour, geocontour interval, and signal data are provided by a source independent from a source of others of said at least two images.

46. A method of generating a VGIS database, the steps comprising:

a) inputting at least two images, one of said images being a base image for georegistration;

b) generating at least one geoscene comprising at least one of the sub-sets;

geolocation, geomosaicking, geogridding, geooverlaying, geomasking, geoindexing, geocontour, geocontour interval, geo-3D-fly-through, and geo-4D-fly-through, said at least one geoscene using at least two ground control points in a lat-long geospatial domain, wherein said geo-3D-fly-through and said geo-4D-fly-through functions each utilize terrain elevation data from at least one of the sources;

full-range digital elevation model (DEM) data, contour interval based elevation data, an unsorted spectral-based terrain feature geoscene, a sorted spectral-based terrain feature geoscene, a terrain-elevation constrained spectral based terrain feature geoscene, a geomasked geoscene, a map-based geoscene, a signal-based geoscene, and a geoindexed geoscene, and wherein geoindexing further comprises generating a feature attribute table corresponding to said color composite;

c) georegistering said at least one geoscene in said lat-long geospatial domain to produce a registered geoimage; and

d) outputting a geoimage comprising at least one of a non-georegistered image, a registered geoimage, a geomosaic, a geogridded geoimage, a geooverlayed geoscene, a geomasked geoscene, a geoindex geoscene, a geocontour, a geocontour interval, a geo-3D-fly-through geoscene, and a geo-4D-fly-through geoscene;

whereby a pixel in said at least one geoscene is represented by;

i) (x,y) coordinates in the image domain;

ii) (z) coordinate in the spectral and elevation/height domains;

iii) UTM representation in the geospatial domain;

iv) latitude/longitude in the geospatial domain; and

v) Virtual Transverse Mercator.