Method and system of generating a three-dimensional view of a real scene for military planning and operations

• US 9,001,116 B2
• Filed: 11/30/2010
• Issued: 04/07/2015
• Est. Priority Date: 12/01/2009
• Status: Active Grant

• Current Assignee: Rafael Advanced Defense Systems Limited (Government of Israel)
• Original Assignee: Rafael Advanced Defense Systems Limited (Government of Israel)
• Inventors: Ben-David, Amit, Peleg, Omri, Briskin, Gil, Shimonovitch, Eran
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US13/512,599
• Publication Number:
  US 20130009950A1
• Time in Patent Office: 1,589 Days
• Field of Search: 345/419, 345/633, 348/143
• US Class Current: 345/419
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/05 Geographic models

Abstract

Generation of views of a real scene is based on selection of view criteria and the content of multiple types of source data. The viewpoint and at least one other criterion area used to determine which texture to sue for each surface, or portion of a surface, of a three-dimensional model. Regions of the images of the scene are associated with at least one portion of at least one surface of the three-dimensional model of the scene, wherein at least one of the portions is associated with the regions of at least two of the images. View selection criteria includes a criterion other than viewpoint. Based on the view selection criteria, selecting at least one of the regions for at least one of the portions and rendering a multi-view using the selected regions in combination with the three-dimensional model of the scene.

Claims

1. A method for rendering views, comprising the steps of:

(a) providing a plurality of images of a scene;

(b) providing a three-dimensional model of the scene;

(c) associating regions of said plurality of images of the scene with at least one portion of at least one surface of said three-dimensional model of the scene, wherein at least one of the portions is associated with said regions of at least two of said plurality of images;

(d) providing at least one view selection criterion, wherein said view selection criteria includes a criterion other than viewpoint;

(e) selecting, for each of at least one portions, at least one of said regions, said selecting based on prioritizing regions for each portion to satisfy said view selection criteria as a function of a current viewpoint;

(f) rendering a multi-view using the selected regions in combination with said three-dimensional model of the scene; and

(g) dynamically repeating steps (e) and (f) such that the selected regions for each of said at least one portions varies as a function of the current viewpoint.

2. The method of claim 1 wherein said plurality of images of a scene are provided from the group consisting of:

(a) still images;

(b) video images;

(c) visible images;

(d) infrared (IR) images;

(e) two-dimensional (2D) images; and

(f) three-dimensional (3D) images.

3. The method of claim 1 wherein providing a three-dimensional model of a scene includes generating a three-dimensional model of the scene using said plurality of images of a scene.

4. The method of claim 1 wherein associating each of said regions of said images of the scene with at least one portion of at least one surface of said three-dimensional model of the scene further includes registering said plurality of images of a scene to said three-dimensional model of the scene.

5. The method of claim 4 wherein registering said plurality of images of a scene to said three-dimensional model of the scene further includes determining the camera angle for each of said images.

6. The method of claim 1 wherein at least one view selection criterion is selected from the group consisting of:

(a) a given time period;

(b) registration accuracy; and

(c) resolution.

7. The method of claim 1 further including deriving features from said plurality of images of a scene and said three-dimensional model of the scene to generate one or more maps, and wherein rendering a multi-view further includes using said maps in combination with said three-dimensional model of the scene to render said multi-view.

8. The method of claim 7 wherein said maps is selected from the group consisting of:

(a) a depth map;

(b) an occlusion map; and

(c) a silhouette map.

9. The method of claim 1 further including providing one or more maps, said one or more maps including the scene, and wherein rendering a multi-view further includes using said one or more maps in combination with said three-dimensional model of the scene to render said multi-view.

10. The method of claim 9 wherein said one or more maps is selected from the group consisting of:

(a) a digital terrain map (DTM);

(b) a digital surface map (DSM); and

(c) satellite imagery, said satellite imagery including raw images with projection information and raw images orthorectified to orthophotos.

11. The method of claim 1 further including providing a collection of information, other than said plurality of images of a scene, as an information layer and further including using said information layers in combination with said multi-view to generate a combined-view.

12. The method of claim 11 wherein said information layer includes a computer vision layer calculated from the features from said plurality of images of a scene and said three-dimensional model of the scene.

13. The method of claim 11 wherein said information layer includes a dynamic vector layer, said dynamic vector layer including information associated with said scene.

14. A system for rendering views, comprising:

(a) one or more image providing devices configured for providing a plurality of images of a scene; and

(b) a processing system containing at least one processor and configured for;

(i) providing a three-dimensional model of the scene;

(ii) associating regions of said plurality of images of the scene with at least one portion of at least one surface of said three-dimensional model of the scene, wherein at least one of the portions is associated with said regions of at least two of said plurality of images;

(iii) providing at least one view selection criterion, wherein said view selection criteria includes a criterion other than viewpoint;

(iv) selecting at least one of said regions for each of at least one portions, said selecting based on prioritizing regions for each portion to satisfy said view selection criteria as a function of a current viewpoint;

(v) rendering a multi-view using the selected regions in combination with said three-dimensional model of the scene; and

(vi) dynamically repeating steps (iv) and (v) such that the selected regions for each of said at least one portions varies as a function of the current viewpoint.

15. The system of claim 14 wherein said one or more image providing devices is selected from the group consisting of:

(a) an image capture device; and

(b) a storage system.

16. The system of claim 14 wherein said plurality of images of a scene are selected from the group consisting of:

(a) still images;

(b) video images;

(c) visible images;

(d) infrared (IR) images;

(e) two-dimensional (2D) images; and

(f) three-dimensional (3D) images.

17. The system of claim 14 wherein said processing system is further configured to provide said three-dimensional model of the scene from a storage system.

18. The system of claim 14 wherein said wherein at least one view selection criterion is selected from the group consisting of:

(a) a given time period;

(b) registration accuracy; and

(c) resolution.

19. The system of claim 14 wherein said processing system is further configured to derive features from said plurality of images of the scene and said three-dimensional model of the scene to generate one or more maps, and further configured to render said multi-view from said three-dimensional model of the scene in combination with said maps.

20. The system of claim 14 wherein said processing system is further configured to render said multi-view from said three-dimensional model of the scene in combination with one or more provided maps.