System for viewing a collection of oblique imagery in a three or four dimensional virtual scene

• US 20070024612A1
• Filed: 07/27/2005
• Published: 02/01/2007
• Est. Priority Date: 07/27/2005
• Status: Active Grant

• Current Assignee: Balfour Technologies LLC
• Original Assignee: Balfour Technologies LLC
• Inventors: Balfour, Robert
• Granted Patent: US 7,554,539 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06T 17/05 Geographic models

Abstract

A system for selecting oblique images from a collection of geo-referenced oblique images and viewing them within the context of a virtual, three- or four-dimensional (3D space and time) geographic scene, providing the ability to analyze and interact with the oblique image being viewed. The system automatically selects and displays the best fit oblique image from an image warehouse based on the user'"'"'s current 3D/4D viewpoint, and continuously maintains the geo-registration of the oblique image as the user adjusts the viewpoint.

Claims

1. A system to interactively view and analyze a collection of oblique images within a three-dimensional or four-dimensional virtual scene, the system comprising:

a plurality of electronic geo-registered oblique images stored in an image warehouse,

electronic meta-data describing the plurality of oblique images;

electronic 3D virtual scene definition data that describe the contents of geographic areas depicted by the plurality of oblique images;

an electronic 3D/4D viewer program operable to select and display at least one of the plurality of oblique images in the image warehouse, wherein the images are geo-registered within a 3D virtual scene; and

a display of the 3D/4D viewer program operable for a user to view and interact with at least one of the plurality of oblique images that is geo-registered within the 3D virtual scene.

2. The system according to claim 1, further comprising:

one or more remote video stream generator devices operable to provide geo-registered, time-stamped oblique images to be stored in the image warehouse.

3. The system according to claim 1, wherein the 3D/4D viewer program includes components for an oblique image motion model, oblique image selection and oblique image display during a display update cycle.

4. The system according to claim 3, wherein the oblique image motion model updates a current viewpoint location based on input from the user, and calculates an associated surface reference point.

5. The system according to claim 4, wherein a new viewpoint is constrained to maintain an angle with the reference surface in the 3D scene that stays within a user-defined deviance with respect to the image angle of the currently displayed oblique image.

6. The system according to claim 4, wherein the oblique image motion model is operable to:

determine if the proximity of the current viewpoint location to a currently displayed oblique image is within a user-defined threshold;

,

lock the oblique image in place by disabling auto image selection and continuous geo-registration functions; and

unlock the oblique image when the current viewpoint location changes to be outside the user-defined threshold.

7. The system according to claim 6, wherein the oblique motion model automatically switches to a maneuvering mode for navigating in a flat, 2D oblique image during a period of time when the oblique image is locked.

8. The system according to claim 3, wherein the oblique image selection is determined automatically based on a current viewpoint location and a surface reference point, wherein a user-configurable best-fit heuristic function is utilized based on a current viewpoint heading with respect to available oblique image heading meta-data and a distance from the surface reference point with respect to all available oblique image geo-reference point meta-data.

9. The system according to claim 8, wherein the oblique image meta-data includes a list of geo-referenced polygons and neighbor lists with respect to each available oblique image enabling a user to logically navigate through the available oblique images, which may be dynamically updated as new oblique images become available in the image warehouse.

10. The system according to claim 8, wherein the best fit heuristic function is further based on a current simulated time with respect to all available oblique image time-stamp meta-data, within a 4D viewer and an image warehouse containing time-stamped oblique images.

11. The system according to claim 8, wherein the automatic oblique image selection function operates in a background process with respect to the 3D/4D viewer display update cycle.

12. The system according to claim 3, wherein the oblique image display function is based on an updated viewpoint and continuously re-geo-registers a currently displayed oblique image by re-applying the currently selected oblique image to a re-calculated oblique image rendering plane within a re-calculated oblique image view frustrum.

13. The system according to claim 12, wherein the oblique image rendering plane is positioned in an oblique image view frustrum foreground of all 3D model surfaces in the 3D scene which are contained within the currently selected oblique image.

14. The system according to claim 3, wherein the oblique image selection component is operable to retrieve an oblique image from the image warehouse when a new image is selected by a user.

15. The system according to claim 14, further comprising an image retrieval and conditional un-compression function that operates in a background process with respect to the display update cycle.

16. The system according to claim 14, wherein the oblique image may be retrieved from the image warehouse in whole or in sections, wherein each section incorporates increased image detail with respect to each image section.

17. The system according to claim 12, wherein a mathematical function is operable to approximate a non-linearity of the oblique image contents and to segment the oblique image rendering plane into a non-linear grid of rectangles to which the appropriate sections of the oblique image is applied.

18. The system according to claim 12, wherein the oblique image is gridded to employ a level-of-detail rendering technique, with the level-of-detail grid constrained to respect the segmentation boundaries of the non-linear grid also applied to the oblique image.

19. The system according to claim 12, wherein decorations are selectively added to the oblique image being rendered, based on its state of locked, unlocked, old or new, including colored border lines, transparent pixels and translucent oblique image frustrum sides.

20. The system according to claim 12, wherein 2D pixel representations of dynamically moving 3D objects in the 3D scene contained within the oblique image view frustrum behind the currently display oblique image are added to the oblique image as the currently displayed oblique image.

21. The system according to claim 20, wherein the 2D pixel representations of dynamic 3D objects account for parts of the 3D object being obscured by 3D objects contained within the currently displayed oblique image.

22. The system according to claim 12, wherein the user can pick a point on the currently displayed oblique image that can be identified as belonging to a specific 3D object in the 3D scene.

23. The system according to claim 22, wherein multiple points on the currently displayed oblique image can be picked by the user, with the individual or cumulative distance between the points automatically calculated.

24. The system according to claim 1, wherein the image warehouse is co-located on a local storage medium on a computer platform containing the 3D/4D viewer adapted for oblique image display and operated by the user.

25. The system according to claim 1, wherein the image warehouse is located on a remote storage medium which transmits oblique image data and meta-data to the adapted 3D/4D viewer program via a communications medium.

26. The system according to claim 1, wherein the meta-data for all the oblique images contained in the image warehouse, including geo-reference points, logical center point and heading and neighbor list are pre-computed and updated as new images are added to the image warehouse.

27. The system according to claim 1, wherein the oblique images in the image warehouse are stored and transmitted in a compressed format.

28. The system according to claim 1, wherein the 3D virtual scene definition is a scene graph.

29. The system according to claim 1, further comprising a computer graphics platform that is a mobile wireless device comprising at least one of a laptop, handheld and wearable computing device.