Methods for transferring points of interest between images with non-parallel viewing directions

• US 8,547,375 B2
• Filed: 03/19/2009
• Issued: 10/01/2013
• Est. Priority Date: 03/31/2008
• Status: Active Grant

• Current Assignee: Rafael Advanced Defense Systems Limited (Government of Israel)
• Original Assignee: Rafael Advanced Defense Systems Limited (Government of Israel)
• Inventors: Rotem, Efrat, Adiv, Gilad
• Primary Examiner(s): Hajnik, Daniel
• Assistant Examiner(s): PRINGLE-PARKER, JASON A
• Application Number:
  US12/922,855
• Publication Number:
  US 20110012900A1
• Time in Patent Office: 1,657 Days
• Field of Search: None
• US Class Current: 345/419
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A method for identifying within a working image of a scene a point of interest designated within a designation image of a scene, the designation image being taken along a first viewing direction and the working image being taken along a second viewing direction, the second viewing direction being significantly non-parallel to the first viewing direction, the method comprising the steps of: obtaining a designation image of the scene; obtaining a working image of the scene; correlating said designation image and said working image with each other, directly or by correlating each of said designation image and said working image with a common reference image, so as to derive an interrelation between said designation image and said working image; and employing said interrelation between said designation image and said working image to derive a location within said working image of a point of interest designated within said designation image, characterized in that the method further comprises: obtaining a secondary designation image taken along a viewing direction displaced from, but similar to, the first viewing direction; and co-processing said designation image and said secondary designation image to derive range information relating to a plurality of pixels in said designation image, thereby defining a partial relative three-dimensional model of a portion of the scene relative to the viewing direction of said designation image, wherein said partial relative three-dimensional model is used in said step of correlating.

Claims

1. A method for identifying within a working image of a scene a point of interest designated within a designation image of the scene, the designation image being taken along a first viewing direction and the working image being taken along a second viewing direction, the second viewing direction being significantly non-parallel to the first viewing direction, the method comprising the steps of:

(a) obtaining a designation image of the scene;

(b) obtaining a working image of the scene;

(c) correlating said designation image and said working image with each other, directly or by correlating each of said designation image and said working image with a common reference image, so as to derive an interrelation between said designation image and said working image; and

(d) employing said interrelation between said designation image and said working image to derive a location within said working image of a point of interest designated within said designation image, characterized in that the method further comprises;

(e) obtaining a secondary designation image taken along a viewing direction displaced from, but similar to, the first viewing direction, said designation image and said secondary designation image being a pair of similar images with similar resolution and similar angular field of view; and

(f) co-processing said designation image and said secondary designation image to derive range information relating to a plurality of pixels in said designation image, thereby defining a partial relative three-dimensional model of a portion of the scene relative to the viewing direction of said designation image, wherein said step of correlating includes employing said partial relative three-dimensional model to generate a simulated image generated by warping said designation image to approximate a view taken along said second viewing direction and correlating said working image with said simulated image,

and wherein said steps of correlating, employing and co-processing are implemented using at least one processing system comprising at least one processor.

2. The method of claim 1, wherein said designation image and said working image are derived from different imaging sensors.

7. The method of claim 1, wherein said simulated image is a partial image containing data only for regions of said designation image expected to be visible in said second viewing direction.

8. The method of claim 1, wherein the second viewing direction differs from said first viewing direction by at least 60 degrees.

9. The method of claim 1, wherein the second viewing direction differs from said first viewing direction by at least 90 degrees.

10. The method of claim 1, wherein said designation image and said working image are views of buildings from different directions and wherein the first and second viewing directions are shallow angle viewing directions.

11. The method of claim 1, wherein a majority of features in said working image do not have corresponding features in said designation image.

12. The method of claim 1, wherein said designation image and said secondary designation image are sampled sequentially by a first camera.

3. A method for identifying within a working image of a scene a point of interest designated within a designation image of the scene, the designation image being taken along a first viewing direction and the working image being taken along a second viewing direction, the second viewing direction being significantly non-parallel to the first viewing direction, the method comprising the steps of:

(a) obtaining a designation image of the scene taken along the first viewing direction;

(b) obtaining a secondary designation image taken along a viewing direction displaced from, but similar to, the first viewing direction, said designation image and said secondary designation image being a pair of similar images with similar resolution and similar angular field of view;

(c) co-processing said designation image and said secondary designation image to derive range information relating to a plurality of pixels in said designation image, thereby defining a partial relative three-dimensional model of a portion of the scene relative to the viewing direction of said designation image;

(d) obtaining a working image of the scene taken along the second viewing direction;

(e) employing said range information to generate a simulated image generated by warping said designation image to approximate a view taken along said second viewing direction;

(f) correlating said working image with said simulated image so as to derive an interrelation between said designation image and said working image; and

(g) employing said interrelation between said designation image and said working image to derive a location within said working image of a point of interest designated within said designation image,

wherein said steps of co-processing, employing said range information, correlating and employing said interrelation are implemented using at least one processing system comprising at least one processor.

13. The method of claim 3, wherein said designation image and said secondary designation image are sampled sequentially by a first camera.

15. The method of claim 3, wherein said designation image and said working image are derived from different imaging sensors.

4. A method for identifying within a working image of a scene a point of interest designated within a designation image of the scene, the designation image being taken along a first viewing direction and the working image being taken along a second viewing direction, the second viewing direction being significantly non-parallel to the first viewing direction, the method comprising the steps of:

(a) obtaining a designation image of the scene taken along the first viewing direction;

(b) obtaining a secondary designation image taken along a viewing direction displaced from, but similar to, the first viewing direction, said designation image and said secondary designation image being a pair of similar images with similar resolution and similar angular field of view;

(c) co-processing said designation image and said secondary designation image to derive range information relating to a plurality of pixels in said designation image;

(d) correlating said designation image with a reference image taken along a reference viewing direction so as to derive an interrelation between said designation image and said reference image, said range information being used to derive a local relative depth map for at least a subregion of said reference image;

(e) correlating said working image with said reference image so as to derive an interrelation between said reference image and said working image, said correlating being performed using said local relative depth map; and

(f) employing said interrelations between said designation image and said reference image and between said working image and said reference image to derive a location within said working image of a point of interest designated within said designation image,

wherein said steps of co-processing, correlating said designation image, correlating said working image and employing are implemented using at least one processing system comprising at least one processor.

5. The method of claim 4, wherein said reference viewing direction is intermediate to said first and second viewing directions.

6. The method of claim 4, wherein said reference image is a geographically anchored reference image, and wherein said local relative depth map is a local relative digital elevation map for at least a subregion of said geographically anchored reference image.

14. The method of claim 4, wherein said designation image and said secondary designation image are sampled sequentially by a first camera.

16. The method of claim 4, wherein said designation image and said working image are derived from different imaging sensors.