User interface mechanism for interactively manipulating displayed registered images obtained from multiple sensors having diverse image collection geometries

• US 5,422,989 A
• Filed: 11/23/1992
• Issued: 06/06/1995
• Est. Priority Date: 11/23/1992
• Status: Expired due to Term

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Bell, David M., Bryan, Joseph K.
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US07/980,501
• Time in Patent Office: 925 Days
• Field of Search: 395/136, 395/137, 395/138, 395/139, 395/133, 395/135, 345/118, 345/113, 345/117
• US Class Current: 345/689
• CPC Class Codes: G06T 17/05 Geographic models

Abstract

A screen-to-registration surface interface translates coordinate system parameters associated with screen-base image manipulation software to latitude and longitude parameters of a co-registration surface-based coordinate system. By invoking a data file linking mechanism of the `windows`-based display and manipulation software, the user may specify a given image manipulation operation to be executed with respect to plural images that have been imported into respective windows of the display. The registration surface coordinate system parameters are compatible with image collection geometry models. These values are then transformed by the respective collection geometry models to the respective digital image files of the linked images. To display these new pixel values, the geometry models associated with the respective digital images imported to the display screen are mapped by a cascaded mapping function, which cascades the image database-to-registration surface translation of the collection geometry model with the registration surface-to-screen coordinate transform, thereby coupling the new pixel values at the image array locations in the database specified by the manipulation parameters to the output display for exploitation by the user.

Claims

1. For use with a digital image processing system in which plural images of a common scene are derived by a plurality of sensors, observation and image collection parameters of which may be different from one another, and wherein said plural images of said common scene obtained by said plurality of sensors are digitized to form a plurality of digital images, each digital image containing an array of pixels of a prescribed spatial resolution, and wherein plural digital images are translated to a prescribed registration surface in accordance with respective geometric models of said sensors through which said plural digital images were derived, a method of controlling the manner in which plural digital images are displayed by way of an output display device comprising the steps of:

(a) providing a display screen-based user interface which is operative by a user to input data representative of one or more prescribed image manipulation functions;

(b) transforming user input data representative of a selected one of said one or more prescribed image manipulation functions to a set of spatial coordinates located in said prescribed registration surface;

(c) applying said set of spatial coordinates located in said prescribed registration surface to respective ones of said geometric models, so as to convert said set of spatial coordinates to respective sets of pixel array coordinates within said plural digital images; and

(d) mapping image pixel values associated with said respective sets of pixel array coordinates within said plural digital images to pixels of said output display device.

2. A method according to claim 1, wherein step (d) comprises applying said respective sets of pixel array coordinates within said plural digital images to respective ones of said geometric models, so as to convert said respective sets of pixel array coordinates to said set of spatial coordinates located in said prescribed registration surface, and transforming said set of spatial coordinates located in said prescribed registration surface to screen coordinates of said output display device.

3. A method according to claim 2, wherein said one or more prescribed image manipulation functions for which user input data is input in step (a) include at least one of zoom, roam and rotate functions.

4. A method according to claim 3, wherein said prescribed image manipulation functions for which user input data is input in step (a) include a propagate function which links said prescribed image manipulation function to each of plural digital images displayed on said output display device.

5. A method according to claim 1, wherein said one or more prescribed image manipulation functions for which user input data is input in step (a) include at least one of zoom, roam and rotate functions.

6. A method according to claim 1, wherein said user input data is defined in terms of a display screen-based coordinate system, and wherein step (b) comprises transforming said display screen-based user input data to a set of latitude and longitude coordinates of said prescribed registration surface.

7. A method according to claim 6, wherein step (c) comprises applying latitude, longitude and elevation coordinates of said prescribed registration surface to said respective ones of said geometric models, so as to convert said latitude, longitude and elevation coordinates to respective sets of line and pixel locations within respective pixel arrays of said plural digital images.

8. A method according to claim 7, wherein step (d) comprises applying said respective sets of line and pixel locations within respective pixel arrays of said plural digital images to said models, so as to convert said respective sets of line and pixel locations to a set of latitude and longitude coordinates of said prescribed registration surface, and transforming said set of latitude and longitude coordinates of said prescribed registration surface to screen coordinates of said output display device.

9. A method according to claim 1, wherein said respective geometric models of sensors through which said plural digital images were derived have been iteratively refined in accordance with a measure of misregistration of said plural digital images on said prescribed registration surface.

10. A method according to claim 9, wherein respective geometric models of first and second sensors through which first and second ones of said plural digital images were derived have been iteratively refined by processing said first and second digital images to obtain plural versions of each of said first and second digital images, having successively reduced spatial resolutions from said prescribed spatial resolution to a prescribed base resolution, correlating edge contents of respective ones of first neighborhoods of pixels associated with a first array of matching points within a first selected resolution version of said first digital image with edge contents of respective ones of second neighborhoods of pixels associated with a second array of matching points within said first selected resolution version of said second digital image, so as to derive a measure of misregistration between said first and second images upon said prescribed registration surface, the contents of each of said first and second neighborhoods of pixels being defined with respect to said prescribed registration surface in accordance with respective first and second geometric models of said first and second sensors, and modifying at least one of said first and second geometric models in accordance with said measure of misregistration, so as to effectively decrease misregistration between said first and second images on said prescribed registration surface.

11. A method according to claim 1, wherein step (a) comprises providing said display screen-based user interface as a windows-based user interface icon, which is operative to enable an operator to interactively input said data representative of said one or more prescribed image manipulation functions.

12. A method according to claim 11, wherein said one or more prescribed image manipulation functions for which user input data is input in step (a) include at least one of zoom, roam and rotate functions.

13. A method according to claim 12, wherein said prescribed image manipulation functions for which user input data is input in step (a) include a propagate function which links said prescribed image manipulation function to each of plural digital images displayed on said output display images.

14. For use with a digital image processing system in which plural images of a common scene are derived by a plurality of sensors, observation and image collection parameters of which may be different from one another, and wherein said plural images of said common scene obtained by said plurality of sensors are digitized to form a plurality of digital images, each digital image containing an array of pixels of a prescribed spatial resolution, and wherein plural digital images are translated to a prescribed registration surface in accordance with respective geometric models of sensors through which said plural digital images were derived, an arrangement for controlling the manner in which plural digital images are displayed by way of an output display device comprising:

a display screen-based user interface which is operative to accept user input data representative of one or more prescribed image manipulation functions;

a first coordinate system translator which is operative to transform said user input data representative of a selected one of said one or more prescribed image manipulation functions to a set of spatial coordinates located in said prescribed registration surface;

a second coordinate system translator which is operative to convert said set of spatial coordinates located in said prescribed registration surface to respective sets of pixel array coordinates within said plural digital images in accordance with respective ones of said geometric models; and

an image database-to-display screen mapping mechanism which is operative to map image pixel values associated with said respective sets of pixel array coordinates within said plural digital images to pixels of said output display device.

15. An arrangement according to claim 14, wherein said image database-to-display screen mapping mechanism is operative to convert respective ones of said plural sets of pixel array coordinates to said set of spatial coordinates located in said prescribed registration surface, in accordance with respective ones of said geometric models, and to transform said set of spatial coordinates located in said prescribed registration surface to screen coordinates of said output display device.

16. An arrangement according to claim 15, wherein said one or more prescribed image manipulation functions for which user data is input to said display screen-based user interface include at least one of zoom, roam and rotate functions.

17. An arrangement according to claim 16, wherein said display screen-based user interface contains a propagate function icon the invoking of which links said prescribed image manipulation function to each of plural digital images displayed on said output display device.

18. An arrangement according to claim 14, wherein said one or more prescribed image manipulation functions for which user data is input to said display screen-based user interface include at least one of zoom, roam and rotate functions.

19. An arrangement according to claim 14, wherein said user input data is defined in terms of a display screen-based coordinate system, and wherein said first coordinate system translator is operative to transform said display screen-based user input data to a set of latitude and longitude coordinates of said prescribed registration surface.

20. An arrangement according to claim 19, wherein said second coordinate system translator is operative to convert latitude, longitude and elevation coordinates of said prescribed registration surface in accordance with said respective ones of said geometric models, to respective sets of line and pixel locations within respective pixel arrays of said plural digital images.

21. An arrangement according to claim 20, wherein said image database-to-display screen mapping mechanism is operative to convert a respective set of line and pixel locations of an image database to a set of latitude and longitude coordinates of said prescribed registration surface, and to transform said set of latitude and longitude coordinates of said prescribed registration surface to screen coordinates of said output display device.

22. An arrangement according to claim 14, wherein said display screen-based user interface is operative to display at least one windows-based user interface icon to enable an operator to interactively input said data representative of said one or more prescribed image manipulation functions.

23. An arrangement according to claim 20, wherein said one or more prescribed image manipulation functions for which user input data is input include at least one of zoom, roam and rotate functions.

24. An arrangement according to claim 23, wherein said prescribed image manipulation functions for which user input data is input include a propagate function which links said prescribed image manipulation function to each of plural digital images displayed on said output display images.

25. An arrangement according to claim 14, wherein said respective geometric models of sensors through which said plural digital images were derived have been iteratively refined in accordance with a measure of misregistration of said plural digital images on said prescribed registration surface.

26. An arrangement to claim 25, wherein respective geometric models of first and second sensors through which first and second ones of said plural digital images were drived have been iteratively refined by processing said first and second digital images to obtain plural versions of each of said first and second digital images, having successively reduced spatial resolutions from said prescribed spatial resolution to a prescribed base resolution, correlating edge contents of respective ones of first neighborhoods of pixels associated with a first array of matching points within a first selected resolution version of said first digital image with edge contents of respective ones of second neighborhoods of pixels associated with a second array of matching points within said first selected resolution version of said second digital image, so as to derive a measure of misregistration between said first and second images upon said prescribed registration surface, the contents of each of said first and second neighborhoods of pixels being defined with respect to said prescribed registration surface in accordance with respective first and second geometric models of said first and second sensors, and modifying at least one of said first and second geometric models in accordance with said measure of misregistration, so as to effectively decrease misregistration between said first and second images on said prescribed registration surface.