System and method for creating, storing, and utilizing composite images of a geographic location

• US 7,239,760 B2
• Filed: 05/16/2005
• Issued: 07/03/2007
• Est. Priority Date: 10/06/2000
• Status: Expired due to Term

• Current Assignee: Vederi LLC
• Original Assignee: Enrico Di Bernardo, Luis F. Goncalves
• Inventors: Di Bernardo, Enrico, Goncalves, Luis F.
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US11/130,004
• Publication Number:
  US 20050207672A1
• Time in Patent Office: 778 Days
• Field of Search: 382/103, 382/104, 382/113, 382/305, 382/284, 382/285, 340/990, 340/995.1, 340/995.11, 340/995.17, 348113-118, 701200-213, 707/1, 707/100
• US Class Current: 382/305
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G01C 21/387 Organisation of map data, e.g. version management or database structures
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

A system and method synthesizing images of a locale to generate a composite image that provide a panoramic view of the locale. A video camera moves along a street recording images of objects along the street. A GPS receiver and inertial navigation system provide the position of the camera as the images are being recorded. The images are indexed with the position data provided by the GPS receiver and inertial navigation system. The composite image is created on a column-by-column basis by determining which of the acquired images contains the desired pixel column, extracting the pixels associated with the column, and stacking the columns side by side. The composite images are stored in an image database and associated with a street name and number range of the street being depicted in the image. The image database covers a substantial amount of a geographic area allowing a user to visually navigate the area from a user terminal.

Claims

1. In a system including an image source and a user terminal having a screen and an input device, a method for enabling visual navigation of a geographic area from the user terminal, the method comprising:

receiving a first user input specifying a first location in the geographic area;

retrieving from the image source a first image associated with the first location, the image source providing a plurality of images depicting views of objects in the geographic area, the views being substantially elevations of the objects in the geographic area, wherein the images are associated with image frames acquired by an image recording device moving along a trajectory;

receiving a second user input specifying a navigation direction relative to the first location in the geographic area;

determining a second location based on the user specified navigation direction; and

retrieving from the image source a second image associated with the second location.

2. The method of claim 1, wherein image source resides at a remote site and receives a request via a communications network for an image corresponding to the first or second location, and transmits the retrieved image to the user terminal via the communications network.

3. The method of claim 1, wherein the image source resides at the user terminal and the method further comprises:

displaying the image corresponding to the first location on the screen of the user terminal; and

updating the image on the screen with the image corresponding to the second location.

4. The method of claim 1 further comprising retrieving a map of a portion of the geographic area for displaying the map on the screen of the user terminal.

5. The method of claim 4 further comprising:

displaying an icon on the map associated with an object in the portion of the geographic area covered by the map;

receiving a user selection of the icon; and

identifying the second location based on the user selection.

6. The method of claim 4, wherein the first or second location is identified by a user selection of the location on the map using the input device.

7. The method of claim 1, wherein the first location is identified by a specific address entered by the user using the input device.

8. The method of claim 1, wherein the retrieving of the image corresponding to the first or second location comprises:

identifying a street segment including the first or second location;

identifying a position on the street segment corresponding to the first or second location; and

identifying an image associated with said position.

9. The method of claim 8, wherein the image simulates a view of objects on one side of the street segment and the method further comprises retrieving a second image depicting a view of objects on an opposite side of the street segment in response to a user request.

10. The method of claim 1 further comprising:

receiving a user selection of a portion of the image; and

displaying an alternate view of the selected portion.

11. The method of claim 1 further comprising:

displaying an indicia that information is available about an object in the image; and

retrieving information about the object in response to a user request to display the information on the screen of the user terminal.

12. The method of claim 1 further comprising:

displaying a navigation button on the screen of the user terminal; and

retrieving the image associated with the second location from the image source upon actuation of the navigation button using the user input device.

13. The method of claim 12, wherein the navigation button indicates direction of motion within a displayed map.

14. The method of claim 12, wherein the navigation button indicates direction of motion with respect to the displayed image.

15. The method of claim 14, wherein the direction of motion includes one of panning left or right, rotating left or right, and viewing a direction opposite of a displayed direction.

16. The method of claim 1 further comprising:

displaying a map of a portion of the geographic area;

identifying the first location with an identifier on the map; and

advancing the identifier on the map from the first location to the second location.

17. The method of claim 1, wherein the plurality of images are substantially frontal views of the objects within the geographic area.

18. The method of claim 1, wherein the image source is an image database.

19. In a system including an image source and a user terminal having a screen and an input device, a method for enabling visual navigation of a geographic area from the user terminal, the method comprising:

accessing a plurality of images from the image source, the images representing objects within the geographic area;

identifying a current location in the geographic area;

retrieving from the image source an image corresponding to the current location;

monitoring a change of the current location in the geographic area; and

retrieving from the image source an image corresponding to the changed location, wherein each retrieved image is a composite image created by processing a plurality of image frames acquired from an image recording device moving through the geographic area.

20. A system for enabling visual navigation of a geographic area from a user terminal, the system comprising:

means for accessing an image source providing a plurality of images depicting views of objects in the geographic area, the views being substantially elevations of the objects in the geographic area, wherein the images are associated with image frames acquired by an image recording device moving along a trajectory;

means for receiving a first user input specifying a first location in the geographic area;

means for retrieving from the image source a first image associated with the first location;

means for receiving a second user input specifying a navigation direction relative to the first location in the geographic area;

means for determining a second location based on the user specified navigation direction; and

means for retrieving from the image source a second image associated with the second location.

21. The system of claim 20, wherein the image source resides at a remote site and includes means for receiving a request for an image corresponding to the first or second location, and means for transmitting the retrieved image to the user terminal.

22. The system of claim 20, wherein the image source resides at the user terminal and the system further comprises:

a display screen for displaying the image of the first location; and

means for updating the image on the screen with the image corresponding to the second location.

23. The system of claim 20 further comprising means for retrieving a map of a portion of the geographic area for displaying the map on the screen of the user terminal.

24. The system of claim 23 further comprising:

means for displaying an icon on the map associated with an object in the portion of the geographic area covered by the map;

means for receiving a user selection of the icon; and

means for identifying the second location based on the user selection.

25. The system of claim 23, wherein the first or second location is identified by a user selection of a portion of the map.

26. The system of claim 20, wherein the first location is identified by a specific address entered by the user using an input device.

27. The system of claim 20, wherein the means for retrieving the image corresponding to the first or second location comprises:

means for identifying a street segment including the first or second location;

means for identifying a position on the street segment corresponding to the first or second location; and

means for identifying an image associated with said position.

28. The system of claim 27, wherein the image simulates a view of objects on one side of the street segment and the system further comprises means for retrieving a second image depicting a view of objects on an opposite side of the street segment in response to a user request.

29. The system of claim 20 further comprising means for processing a plurality of image frames acquired from the image recording device moving through the geographic area.

30. The system of claim 20 further comprising:

means for receiving a user selection of a portion of the image; and

a display displaying an alternate view of the selected portion.

31. The system of claim 20 further comprising:

means for indicating that information is available about an object in the image;

means for retrieving information about the object in response to a user request to display the information on the screen of the user terminal.

32. The system of claim 20 further comprising:

a navigation button indicating the navigation direction.

33. The system of claim 32, wherein the navigation button indicates the navigation direction within a displayed map.

34. The system of claim 32, wherein the navigation button indicates the navigation direction with respect to the displayed image.

35. The system of claim 34, wherein the navigation direction includes one of panning left or right, rotating left or right, and viewing a direction opposite of a displayed direction.

36. The system of claim 20 further comprising:

means for displaying a map of a portion of the geographic area;

means for identifying the first location on the map; and

means for identifying the second location on the map based on the user-selected navigation direction.

37. The system of claim 20, wherein the plurality of images are substantially frontal views of the objects within the geographic area.

38. In a system including an image database and a user terminal having a screen and an input device, a method for enabling visual navigation of a geographic area from the user terminal, the method comprising:

receiving a first user input specifying a first location in the geographic area;

retrieving from the image database a first set of image frames associated with the first location, the image database storing a plurality of image frames depicting views of objects in the geographic area, the views being substantially elevations of the objects in the geographic area, wherein the image frames are acquired by an image recording device moving along a trajectory;

generating a first image of the first location based on the first set of image frames;

receiving a second user input specifying a navigation direction relative to the first location in the geographic area;

determining a second location based on the user specified navigation direction;

retrieving from the image database a second set of image frames associated with the second location; and

generating a second image of the second location based on the second set of image frames.