Method for incorporating images with a user perspective in navigation

• US 7,272,498 B2
• Filed: 09/30/2004
• Issued: 09/18/2007
• Est. Priority Date: 09/30/2004
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Scenera Technologies LLC (Scenera Research LLC)
• Inventors: Singh, Mona
• Primary Examiner(s): Beaulieu, Y.
• Application Number:
  US10/954,442
• Publication Number:
  US 20070198182A1
• Time in Patent Office: 1,083 Days
• Field of Search: 701200-202, 701/211, 701/213, 701/24, 701/28, 701/208, 348/148, 340/988, 340/990, 340/991, 340/995.1, 340/995.19, 342/357.09
• US Class Current: 701/211
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams

Abstract

An image-based navigation method and system are provided. In response to a user request submitted to a navigation system, the image-based navigation system provides directions to the user, wherein the directions include one or more route segments, and at least one digital image of a view along at least one of the route segments. In a further aspect, the directions are provided to the user by displaying a map of the route along with the route segments, wherein when each of the route segments is displayed, any images associated with the corresponding route segment are also displayed.

Claims

1. A method for providing an online image-based navigation directory service, comprising:

receiving by a server uploaded digital images, wherein each of the uploaded images is tagged with location data and metadata, wherein the location data specifies geographic coordinates of the image capture, and the metadata includes a time and date of the image capture and optionally user annotations describing the image;

storing the digital images in an image database that is indexed by the location data;

allowing a user of a navigation system to log into the server and to submit a request for directions that is wirelessly transmitted to the server;

in response to receiving the request for directions, accessing a map database and calculating directions for the route, including route segments;

for each segment identified along the route, searching the image database for any digital images that have location data that matches the location of the segment; and

transmitting the set of images found for the directions from the server to the navigation system for display to the user along with the directions.

2. The method of claim 1 wherein the directions are provided to the user by displaying a map of the route along with the route segments, the method further including:

when each of the route segments is displayed, also displaying any images associated with the corresponding route segment.

3. The method of claim 2 wherein when each of the images are displayed, displaying metadata corresponding to each image.

4. The method of claim 2 further including:

allowing the user of the navigation system to cycle through the display of route segments and indicating to the user whether images are available for each segment.

5. The method of claim 2 further including:

displaying the map of the route with points of interest identified, and in response to the user selecting one or more of these points, making any images corresponding to those locations available to the user.

6. The method of claim 2 further including:

using the user'"'"'s current time and date to filter a set of available images for a segment such that only the images that have a time and date matching the user'"'"'s current time and date within a predetermined threshold are displayed to the user.

7. The method of claim 1 wherein the navigation system sends its current heading to the server at a beginning of each segment along the route, and the server only transmits those images for a segment that match the heading transmitted by the navigation system.

8. The method of claim 1 wherein the server transmits the images found for all of the segments along the route to the navigation system, regardless of heading, and the navigation system determines which of those images match a current heading of the navigation system and only displays the matching images for each segment.

9. The method of claim 1 further including:

requiring that the location data uploaded with the digital images include a heading indicating a direction of view of the image.

10. The method of claim 9 wherein the location data comprises GPS data.

11. The method of claim 9 wherein the location data comprises a manually entered street address and a direction of view from a user of an image capture device.

12. The method of claim 1 wherein the request includes a current heading of the navigation system, the method further including:

searching the image database for digital images that have both location data and a heading that matches the location and the heading of the navigation system.

13. The method of claim 12 further including:

charging the user of the navigation system a fee for use of the image-based navigation directory service.

14. An image-based navigation system, comprising:

at least one server;

an image database for storing digital images uploaded from image capture devices to the server, wherein each of the uploaded images is tagged with location data and metadata, the location data specifies geographic coordinates of the image capture, the metadata includes a time and date of the image capture and optionally user annotations describing the image, and the image database is indexed by the location data; and

a map database for storing map and directions data;

wherein the server is configured to perform functions for

receiving a request for directions from a user of a navigation system,

in response to receiving the request for directions, accessing the map database to calculate directions based on the request, wherein the directions include one or more route segments;

searching the image database for any digital images that have location data that match locations of the route segments; and

returning the route segments and corresponding images found to the navigation system for display along with the directions.

15. The system of claim 14 wherein the request includes a current heading of the navigation system, the server further configured to:

search the image database for digital images that have both location data and a heading that matches the location and the heading of the navigation system.

16. The system of claim 14 wherein the image-based navigation system further includes a user account database for storing user account information;

such that when the user request for directions is received, the user is validated in the account database.

17. A method for operating a navigation system comprising:

enabling a user to submit a request for directions;

transmitting the request to an online navigation directory service for processing the request;

receiving the directions from the online navigation directory service, wherein the directions include one or more route segments and at least one digital image uploaded from an image capture device to the navigation directory service, the at least one digital image tagged with location data and metadata, the location data specifying geographic coordinates of the image capture, the metadata including a time and date of the image capture and optionally user annotations describing the image, and the at least one digital image has location data that matches a location of at least one of the route segments; and

displaying the directions to the user by sequentially displaying the route segments along with any images having location data that matches the locations of the route segments.

18. A computer readable medium containing programming instructions, executable by a machine, for providing image-based navigation, the instructions for:

receiving by a server uploaded digital images, wherein each of the uploaded images is tagged with location data and metadata, wherein the location data specifies geographic coordinates of the image capture, and the metadata includes a time and date of the image capture and optionally user annotations describing the image;

storing the digital images in an image database that is indexed by the location data;

allowing a user of a navigation system to log into the server and to submit a request for directions that is wirelessly transmitted to the server;

in response to receiving the request for directions, accessing a map database and calculating directions for the route, including route segments;

for each segment identified along the route, searching the image database for any digital images that have location data that matches the location of the segment; and

transmitting the set of images found for the directions from the server to the navigation system for display to the user along with the directions.

19. The computer readable medium of claim 18 wherein the directions are provided to the user by displaying a map of the route along with the route segments, the computer readable medium further including instructions for:

when each of the route segments is displayed, also displaying any images associated with the corresponding route segment.

20. The computer readable medium of claim 18 including instructions for when each of the images are displayed, displaying metadata corresponding to each image.

21. The computer readable medium of claim 18 further including instructions for:

allowing the user of the navigation system to cycle through the display of route segments and indicating to the user whether images are available for each segment.

22. The computer readable medium of claim 18 further including instructions for:

displaying the map of the route with points of interest identified, and in response to the user selecting one or more of these points, making any images corresponding to those locations available to the user.

23. The computer readable medium of claim 18 further including instructions for:

using the user'"'"'s current time and date to filter a set of available images for a segment such that only the images that have a time and date matching the user'"'"'s current time and date within a predetermined threshold are displayed to the user.

24. The computer readable medium of claim 18 further including instructions for:

searching the image database for digital images that have both location data and a heading that matches a location of the segment and the heading of the navigation system.

25. The computer readable medium of claim 18 including instructions for the navigation system sending its current heading to the server at a beginning of each segment along the route, and instructions for the server only transmitting those images for a segment that match the heading transmitted by the navigation system.

26. The computer readable medium of claim 18 including instructions for the server transmitting the images found for all of the segments along the route to the navigation system, regardless of heading, and instructions for the navigation system determining which of those images match a current heading of the navigation system and only displaying the matching images for each segment.

27. The computer readable medium of claim 18 further including instructions for:

requiring that the location data uploaded with the digital images include a heading indicating a direction of view of the image.

28. The computer readable medium of claim 27 wherein the location data comprises GPS data.

29. The computer readable medium of claim 27 wherein the location data comprises a manually entered street address and a direction of view from a user of an image capture device.