System and method for creating, storing and utilizing images of a geographical location

• US 8,818,138 B2
• Filed: 06/01/2012
• Issued: 08/26/2014
• Est. Priority Date: 10/06/2000
• Status: Expired due to Fees

• Current Assignee: Enrico Di Bernardo, Luis F. Goncalves
• Original Assignee: Enrico Di Bernardo, Luis F. Goncalves
• Inventors: Di Bernardo, Enrico, Goncalves, Luis F.
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US13/486,992
• Publication Number:
  US 20130107034A1
• Time in Patent Office: 816 Days
• Field of Search: 382/104, 382/113, 382/291, 382/305, 382/312, 715/850, 715/851, 715/854, 715/855, 701408-418, 3409951-99526, 348/113, 348/116
• US Class Current: 382/305
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G01C 21/387 Organisation of map data, e.g. version management or database structures
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

A system and method synthesizing images of a locale to generate a composite image that provide a panoramic view of the locale. A video camera moves along a street recording images of objects along the street. A GPS receiver and inertial navigation system provide the position of the camera as the images are being recorded. The images are indexed with the position data. The composite image is created on a column-by-column basis by determining which of the acquired images contains the desired pixel column, extracting the pixels associated with the column, and stacking the columns side by side. The composite images are stored in an image database and associated with a street name and number range of the street being depicted in the image. The image database covers a substantial amount of a geographic area allowing a user to visually navigate the area from a user terminal.

Claims

1. A server configured for use in a system that enables visual navigation of a geographic area, the system further comprising an image source and a user terminal, the user terminal comprising a screen and an input device, the server comprising:

at least one processor; and

a memory coupled to the at least one processor and configured to store program instructions,

wherein the at least one processor is configured to execute the program instructions, the program instructions comprising instructions for;

receiving, by the at least one processor from the user terminal, a first user input specifying a first location in the geographic area;

retrieving, by the at least one processor from the image source, first image data associated with the first location, the image source providing a plurality of image data for displaying a plurality of images depicting front and/or side views of objects in the geographic area, wherein each of the images is a composite image generated based on a plurality of image frames acquired by an image recording device when the image recording device is in motion along a trajectory;

sending, by the at least one processor, instructions to the user terminal to display on the screen a first composite image based on the first image data;

receiving, by the at least one processor from the user terminal, a second user input specifying a navigation direction relative to the first location in the geographic area;

determining, by the at least one processor, a second location based on the user specified navigation direction; and

retrieving, by the processor from the image source, second image data associated with the second location.

2. The server of claim 1, wherein the retrieving the first image data associated with the first location comprises:

identifying a street segment including the first location;

identifying a position on the street segment corresponding to the first location; and

identifying an image associated with the position.

3. The server of claim 1, wherein the program instructions further comprise instructions for:

sending, by the at least one processor, instructions to the user terminal to display on the screen a map of a portion of the geographic area;

sending, by the at least one processor, an identifier of the first location on the map; and

sending, by the at least one processor, instructions to the user terminal to advance the identifier on the map from the first location to the second location.

4. The server of claim 1, wherein the program instructions further comprise instructions for:

receiving, by the at least one processor, a selection of an object of the objects;

identifying, by the at least one processor, a hyperlink associated with the object; and

sending, by the at least one processor to the user terminal, the hyperlink associated with the object.

5. The server of claim 1, wherein the first composite image is created by combining pixel data of a plurality of the image frames to create a new image depicting objects of the plurality of image frames over a wider field of view as compared to the field of view of any one of the image frames.

6. The server of claim 5, wherein the first composite image is created based on a first one of the image frames acquired at a first location along the trajectory and a second one of the image frames acquired at a second location along the trajectory.

7. The server of claim 5, wherein the first composite image is created based on at least two image frames acquired at the same location along the trajectory.

8. The server of claim 1, wherein the image recording devices captures images in all directions.

9. The server of claim 1, wherein the image recording devices captures images in 360 degrees of view.

10. In a system including an image source and a portable communication device having a screen and an input device, a method for enabling visual navigation of a geographic area from the portable communication device, the method comprising:

receiving, from the portable communication device, a first user input specifying a first location in the geographic area;

retrieving, from the image source, first image data associated with the first location, the image source providing a plurality of image data for displaying a plurality of images depicting front and/or side views of objects in the geographic area, wherein each of the images is a composite image generated based on image frames acquired by an image recording device when the image recording device is in motion along a trajectory;

sending instructions to the portable communication device to display, on the screen, a first composite image based on the first image data;

receiving, from the portable communication device, a second user input specifying a navigation direction relative to the first location in the geographic area;

determining a second location based on the user specified navigation direction; and

retrieving, from the image source, second image data associated with the second location.

11. The method of claim 10, wherein the image recording device comprises:

a front facing camera;

a rear facing camera;

a first side facing camera; and

a second side facing camera.

12. The method of claim 10, wherein the composite image is created by combining pixel data of a plurality of the image frames to create a new image depicting objects of the plurality of image frames over a wider field of view as compared to the field of view of any one of the image frames.

13. The method of claim 10, further comprising:

receiving, from the portable communication device, a selection of an object of the objects;

identifying a hyperlink associated with the object; and

sending, to the portable communication device, the hyperlink associated with the object.

14. The method of claim 10, further comprising:

sending instructions to the portable communication device to display on the screen a map of a portion of the geographic area;

sending instructions to the portable communication device to display an identifier of the first location on the map; and

sending instructions to the portable communication device instructions to advance the identifier on the map from the first location to the second location.

15. In a system including an image source providing a plurality of image data for displaying a plurality of images depicting front and/or side views of objects in a geographic area, each of the images being a composite image generated based on image frames acquired by an image recording device when the image recording device is in motion along a trajectory, and including a user terminal comprising a screen and an input device, a method for enabling visual navigation of a geographic area from the user terminal, the method comprising:

segmenting the trajectory on which the image recording device moved into a plurality of segments;

correlating the plurality of segments to a plurality of street segments in a geographic information database;

receiving, from the user terminal, a first user input specifying a first location in the geographic area;

identifying one of the street segments based on the first user input specifying the first location;

retrieving, from the image source, first image information associated with the first location based on the identified one of the plurality of street segments; and

sending the first image information to the user terminal to display on the screen a first image based on the first image information.

16. The method of claim 15, wherein the composite image is created by combining pixel data of a plurality of the image frames to create a new image depicting objects of the plurality of image frames over a wider field of view as compared to the field of view of any one of the image frames.

17. The method of claim 16, wherein the composite image depicts a wider field of view than is depicted in any one of the image frames.

18. The method of claim 15, further comprising:

receiving, from the user terminal, a selection of an object of the objects;

identifying a hyperlink associated with the object; and

sending, to the user terminal, the hyperlink associated with the object.

19. The method of claim 15, further comprising:

sending instructions to the user terminal to display on the screen a map of a portion of the geographic area;

sending instructions to the user terminal to display an identifier of the first location on the map;

receiving, from the user terminal, a second user input specifying a second location in the geographic area; and

sending instructions to the user terminal to advance the identifier on the map from the first location to the second location.