Method, system and program product for augmenting an image of a scene with information about the scene

• US 6,930,715 B1
• Filed: 07/21/2000
• Issued: 08/16/2005
• Est. Priority Date: 07/21/2000
• Status: Expired due to Term

• Current Assignee: The Research Foundation for The State University of New York (State University of New York)
• Original Assignee: The Research Foundation for The State University of New York (State University of New York)
• Inventors: Mower, James E.
• Primary Examiner(s): Ho, Tuan
• Assistant Examiner(s): JONES, HEATHER RAE
• Application Number:
  US09/621,513
• Time in Patent Office: 1,852 Days
• Field of Search: 701/208, 701/201, 701/207, 345/247, 348/148, 348/239, 348/207.99
• US Class Current: 348/239
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A fixed or mobile image capturing device, coupled to a first computer, is remotely controlled over a global computer network by a second computer, and obtains a digital image of a scene. The first computer acquires a digital elevation model of the scene, which is data for creating a three-dimensional perspective model of the scene in two dimensions based on data corresponding, and registers the image with the digital elevation model. The registered digital image is provided to the second computer over the global computer network. When a user at the second computer points to an area of the registered image, the first computer augments the image provided with at least some information corresponding to the area pointed to.

Claims

1. A method of augmenting an image of a scene, comprising:

obtaining by a first computing unit a digital image of a scene;

acquiring by the first computing unit a digital elevation model of the scene, wherein the digital elevation model comprises data for creating a three-dimensional perspective model of the scene in two dimensions;

registering by the first computing unit the digital image with the digital elevation model to create a registered digital image;

providing the registered digital image from the first computing unit to a second computing unit coupled to the first computing unit by a communication network; and

augmenting the registered digital image provided to the second computing unit with at least some information about the scene in response to input from the second computing unit.

2. The method of claim 1, wherein obtaining the digital image comprises obtaining an image of the scene in real time with a fixed image capturing device.

3. The method of claim 1, wherein obtaining the digital image comprises obtaining an image of the scene in real time with a mobile image capturing device.

4. The method of claim 1, wherein obtaining the digital image comprises obtaining a preexisting image of the scene.

5. The method of claim 1, wherein the obtaining comprises obtaining an analog image and digitizing the analog image.

6. The method of claim 1, wherein the acquiring comprises acquiring a preexisting digital elevation model.

7. The method of claim 1, wherein the information comprises a location of at least one above-ground feature within the scene.

8. The method of claim 1, wherein the information comprises a location of at least one underground feature within the scene.

9. The method of claim 1, wherein the obtaining comprises obtaining an image of the scene from an image-capturing device, the method further comprising determining an orientation and a zoom factor for the image-capturing device.

10. The method of claim 9, wherein the image-capturing device is fixed, and wherein the determining comprises reading positional and zoom factor information from the image-capturing device.

11. The method of claim 10, wherein the positional information comprises latitude, longitude and elevation above ground.

12. The method of claim 9, wherein the image-capturing device is mobile, and wherein the determining comprises reading a zoom factor from the image-capturing device and obtaining the orientation from equipment positionally fixed relative to the image-capturing device.

13. The method of claim 12, wherein the equipment comprises a global positioning system, a digital compass and a digital inclinometer.

14. The method of claim 1, further comprising rendering the registered digital image at the second computing unit to produce a rendered image, wherein the augmenting comprises:

identifying at the second computing unit a point of interest in the rendered image;

providing the identified point of interest from the second computing unit to the first computing unit;

reverse projecting the identical point of interest at the first computing unit to determine coordinates therefor; and

accessing the information at the first computing unit corresponding to the coordinates.

15. The method of claim 14, wherein the augmenting further comprises overlaying the information corresponding to the coordinates on the registered digital image.

16. The method of claim 15, wherein the information comprises textual cartographic data.

17. The method of claim 15, wherein the information comprises graphical cartographic data.

18. The method of claim 1, further comprising rendering the registered digital image at the second computing unit to create a displayed image, wherein the input comprises pointing by a user to at least one area of the displayed image.

19. The method of claim 1, wherein the communications network comprises a global computer network.

20. A system for augmenting an image of a scene, comprising:

a computing unit programmable for;

obtaining a digital image of a scene;

acquiring a digital elevation model of the scene, wherein the digital elevation model comprises data for creating a three-dimensional perspective model of the scene in two dimensions;

registering the digital image with the digital elevation model to create a registered digital image;

providing the registered digital image over a communications network; and

augmenting the registered digital image with at least some of information about the scene in response to input.

21. The system of claim 20, further comprising a fixed image capturing device coupled to the computing unit for capturing an image of the scene in real time.

22. The system of claim 20, further comprising a mobile image capturing device coupled to the computing unit for capturing an image of the scene in real time.

23. The system of claim 20, further comprising a scanner for digitizing an analog image of the scene.

24. The system of claim 20, wherein the acquiring comprises acquiring a preexisting digital elevation model.

25. The system of claim 20, wherein the information comprises a location of at least one above-ground feature within the scene.

26. The system of claim 20, wherein the information comprises a location of at least one underground feature within the scene.

27. The system of claim 20, further comprising an image-capturing device, the computing unit determining an orientation and a zoom factor for the image-capturing device.

28. The system of claim 27, wherein the image-capturing device is fixed, and wherein the determining comprises reading positional and zoom factor information from the image-capturing device.

29. The system of claim 28, wherein the positional information comprises latitude, longitude and elevation above ground.

30. The system of claim 27, further comprising orientation equipment positionally fixed relative to the image-capturing device, wherein the image-capturing device is mobile, and wherein the determining comprises reading a zoom factor from the image-capturing device.

31. The system of claim 30, wherein the orientation equipment comprises a global positioning system, a digital compass and a digital inclinometer.

32. The system of claim 20, further comprising another computing unit coupled to the computing unit via the communication network, the another computing unit comprising:

a display for rendering the registered digital image to produce a rendered image; and

a pointing device for identifying a point of interest in the rendered image;

wherein the augmenting comprises reverse projecting the identified point of interest to determine coordinates therefor, and accessing the information corresponding to the coordinates.

33. The system of claim 32, wherein the augmenting further comprises overlaying the information corresponding to the coordinates on the rendered image.

34. The system of claim 33, wherein the information comprises textual cartographic data.

35. The system of claim 33, wherein the information comprises graphical cartographic data.

36. The system of claim 32, wherein the communication network comprises a global computer network.

37. At least one program storage device readable by a machine, tangibly embodying at least one program of instruction executable by the machine to perform a method of augmenting an image of a scene, the method comprising:

obtaining by a first computing unit a digital image of a scene;

acquiring by the first computing unit a digital elevation model of the scene, wherein the digital elevation model comprises data for creating a three-dimensional perspective model of the scene in two dimensions;

registering by the first computing unit the digital image with the digital elevation model to create a registered digital image;

providing the registered digital image from the first computing unit to second computing unit coupled to the first computing unit by a communications network; and

augmenting the registered digital image provided to the second computing unit with at least some of information about the scene in response to input from the second computing unit.

38. The at least one program storage device of claim 37, wherein obtaining the digital image comprises obtaining a digital image of the scene in real time from a fixed digital image capturing device.

39. The at least one program storage device of claim 37, wherein obtaining the digital image comprises obtaining a digital image of the scene in real time from a mobile digital image capturing device.

40. The at least one program storage device of claim 37, wherein obtaining the digital image comprises obtaining a preexisting digital image of the scene.

41. The at least one program storage device of claim 37, wherein acquiring the digital elevation model comprises acquiring a preexisting digital elevation model.

42. The at least one program storage device of claim 37, wherein the information comprises a location of at least one above-ground feature within the scene.

43. The at least one program storage device of claim 37, wherein the information comprises a location of at least one underground feature within the scene.

44. The at least one program storage device of claim 37, wherein the obtaining comprises obtaining the digital image from a digital image-capturing device, the method further comprising determining an orientation and a zoom factor for the digital image-capturing device.

45. The at least one program storage device of claim 44, wherein the digital image-capturing device is fixed, and wherein the determining comprises reading positional and zoom factor information from the digital image-capturing device.

46. The at least one program storage device of claim 45, wherein the positional information comprises latitude, longitude and elevation above ground.

47. The at least one program storage device of claim 44, wherein the digital image-capturing device is mobile, and wherein the determining comprises reading a zoom factor from the digital image-capturing device and obtaining the orientation from equipment positionally fixed relative to the digital image-capturing device.

48. The at least one program storage device of claim 47, wherein the equipment comprises a global positioning system, a digital compass and a digital inclinometer.

49. The at least one program storage device of claim 37, further comprising rendering the registered digital image at the second computing unit to produce a rendering image, wherein the augmenting comprises:

identifying at the second computing unit a point of interest in the rendered image;

providing the identified point of interest from the second computing unit to the first computing unit;

reverse projecting the identified point of interest at the first computing unit to determine coordinates therefor; and

accessing the information at the first computing unit corresponding to the coordinates.

50. The at least one program storage device of claim 49, wherein the augmenting further comprises overlaying the information corresponding to the coordinates on the registered digital image.

51. The at least one program storage device of claim 50, wherein the information comprises textual cartographic data.

52. The at least one program storage device of claim 50, wherein the information comprises graphical cartographic data.

53. The at least one program storage device of claim 37, wherein the communications network comprise a global computer network.

54. A system of augmenting an image of a scene, comprising:

means for obtaining by a first computing unit a digital image of a scene;

means for acquiring by the first computing unit a digital elevation model of the scene, wherein the digital elevation comprises data for creating a three-dimensional perspective model of the scene in two dimensions;

means for registering by the first computing unit the digital image with the digital elevation model to create a registered digital image;

means for providing the registered digital image from the first computing unit to a second computing unit coupled to the first computing unit by a communications network; and

means for augmenting the registered digital image provided to the second computing unit with at least some information about the scene in response to input from the second computing unit.