Method of obtaining geographically related images using a vehicle

• US 9,726,485 B2
• Filed: 04/22/2008
• Issued: 08/08/2017
• Est. Priority Date: 04/22/2007
• Status: Active Grant

• Current Assignee: iLOOKabout, Inc.
• Original Assignee: iLOOKabout, Inc.
• Inventors: Hack, Jeffrey
• Primary Examiner(s): Philippe, Gims
• Assistant Examiner(s): Mung, On S
• Application Number:
  US12/596,553
• Publication Number:
  US 20100141736A1
• Time in Patent Office: 3,395 Days
• Field of Search: 348 36, 348148
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 22/02 by conversion into electric waveforms and subsequent integration, e.g. using tachometer generator G01C22/002, G01C22/004, G01C22/006 take precedence

Abstract

The method employs a computer interconnected with a GPS transceiver and a distance measurement means and operatively interconnected with a digital camera, all provided on a moving vehicle, in order to acquire a plurality of geographic location related images. A first geographic position is obtained from the GPS transceiver and a plurality of distance measurements are then received before a second geographic position is taken. A photo request is generated after a pre-determined distance has been traveled and is transmitted to the digital camera. The camera takes a photo based on the request and the photos, photo requests, first and second positions and distance measurements are stored for post-processing. The post-processing determines a path traveled by the vehicle using the positions and distance measurements and assigns an accurate geographic location and orientation to each photo. The method is useful in building a database of geographically related images.

Claims

1. A method of obtaining digital images related to geographic locations, the method comprising:

providing a moving vehicle having thereon a computer having a computer clock interconnected with a GPS transceiver and distance measurement structure configured to measure a distance traveled by the vehicle, the distance measurement structure comprising a pair of laterally spaced apart independently rotating wheels, the wheels spaced apart by a track width, the distance measurement structure simultaneously providing wheel speed measurements from each wheel, the wheel speed measurements and a diameter of each wheel being used to calculate a distance traveled by each wheel, the computer operatively interconnected with a digital camera having a camera clock mounted on the vehicle;

receiving into the computer a first geographic position from the GPS transceiver;

receiving into the computer a measurement of distance traveled by the vehicle relative to the first geographic position from the distance measurement structure;

selecting a pre-determined distance based on a distance between a subject of a photo to be acquired and the camera;

determining when the pre-determined distance has been traveled by the vehicle based on the measurement of distance;

generating a photo request using the computer on the vehicle for the digital camera on the vehicle at the pre-determined distance, the photo request having an associated photo request time comprising a computer time value obtained from the computer clock, the photo request resulting in a shutter release by the digital camera;

taking a digital photo using the camera on the vehicle based on the shutter release, the digital photo having an associated photo time comprising a camera time value obtained from the camera clock;

repeating the steps of receiving into the computer the measurement of distance traveled by the vehicle, determining when the pre-determined distance has been traveled, generating a photo request using the computer resulting in a shutter release by the digital camera and taking a digital photo using the camera on the vehicle based on the shutter release;

receiving into the computer a second geographic position from the GPS transceiver;

storing the photos, associated photo times, photo requests, associated photo request times, first and second geographic positions and distance measurements;

conducting a post-processing step remotely from the vehicle, the post-processing step comprising

calculating a difference in the distance measurements from the two wheels;

calculating a change in vehicle orientation based on the difference;

using the change in orientation and distance traveled, iteratively fitting a plurality of path segments, each having a fixed length corresponding to the pre-determined distance and having an associated vehicle orientation comprising the direction in which the vehicle has turned and the number of degrees in the turn, to a path traveled by the vehicle between the first and second geographic positions, the path iteratively fit in three dimensional Cartesian space by an error minimization solver using at least i) the diameter of each wheel or the ratio of the diameters of the two wheels, ii) the wheel speed of each wheel, iii) the track width, iv) the three components of an initial position in the Cartesian space corresponding to the first geographic position, and v) an initial heading of the vehicle;

associating each photo with the photo request used to generate that photo by comparing camera time values of a plurality of photos with computer time values of a plurality of photo requests using a piecewise affine transformation;

determining a computer time value associated with each photo based on the computer time value of the photo request plus a lag value that accounts for time differences between the generation of the photo request and the taking of the photo, the lag value quantified by comparing camera time values of a plurality of photos with computer time values of a plurality of photo requests;

obtaining a location of each photo on the path based on the computer time value of the photo; and

,

correcting each photo to a rectilinear perspective based on the associated vehicle orientation of the path segment corresponding to the photo and a field of view of the lens to provide a centered perpendicular image orientation for the photo.

2. The method of claim 1, wherein the vehicle is land-based.

3. The method of claim 1, wherein the vehicle comprises an automobile and wherein the distance measurement structure comprises a wheel speed sensor connected to each of the automobile'"'"'s rear wheels.

4. The method of claim 1, wherein the digital camera is a still-image camera with a resolution of at least 6 megapixels.

5. The method of claim 1, wherein the digital camera is equipped with a lens having a field of view of at least 160°

.

6. The method of claim 1, wherein the digital photos are adjacent to one another and share overlapping visual elements.

7. The method of claim 1, wherein the vehicle travels at a speed of at least 35 km/h while photos are being taken.

8. The method of claim 1, wherein the vehicle is equipped with at least two digital cameras oriented in opposite directions to one another and wherein each photo request is directed to all cameras.

9. The method of claim 1, wherein the error minimization solver comprises a Levinburg-Marquardt algorithm.

10. A method of obtaining digital images related to geographic locations, the method comprising:

providing a moving vehicle having thereon a computer having a computer clock interconnected with a GPS transceiver and distance measurement structure configured to measure a distance traveled by the vehicle, the distance measurement structure comprising a pair of laterally spaced apart independently rotating wheels, the wheels spaced apart by a track width, the distance measurement structure simultaneously providing wheel speed measurements from each wheel, the wheel speed measurements and a diameter of each wheel being used to calculate a distance traveled by each wheel, the computer operatively interconnected with a digital camera having a camera clock mounted on the vehicle;

receiving into the computer a first geographic position from the GPS transceiver;

receiving into the computer a measurement of distance traveled by the vehicle relative to the first geographic position from the distance measurement structure;

determining when a pre-determined distance has been traveled by the vehicle based on the measurement of distance;

generating a photo request using the computer on the vehicle for the digital camera on the vehicle at the pre-determined distance, the photo request having an associated photo request time comprising a computer time value obtained from the computer clock, the photo request resulting in a shutter release by the digital camera;

taking a digital photo using the camera on the vehicle based on the shutter release, the digital photo having an associated photo time comprising a camera time value obtained from the camera clock;

repeating the steps of receiving into the computer the measurement of distance traveled by the vehicle, determining when the pre-determined distance has been traveled, generating a photo request using the computer resulting in a shutter release by the digital camera and taking a digital photo using the camera on the vehicle based on the shutter release;

receiving into the computer a second geographic position from the GPS transceiver;

storing the photos, associated photo times, photo requests, associated photo request times, first and second geographic positions and distance measurements;

conducting a post-processing step remotely from the vehicle, the post-processing step comprising

calculating a change in vehicle orientation;

using the change in orientation and distance traveled, iteratively fitting a plurality of path segments, each having a fixed length corresponding to the pre-determined distance and having an associated vehicle orientation comprising the direction in which the vehicle has turned and the number of degrees in the turn, to a path traveled by the vehicle between the first and second geographic positions;

associating each photo with the photo request used to generate that photo by comparing camera time values of a plurality of photos with computer time values of a plurality of photo requests;

determining a computer time value associated with each photo based on the computer time value of the photo request plus a lag value that accounts for time differences between the generation of the photo request and the taking of the photo;

obtaining a location of each photo on the path based on the computer time value of the photo; and

,

correcting each photo to a rectilinear perspective.

11. The method of claim 10, wherein the pre-determined distance is determined based on a distance between a subject of a photo to be acquired and the camera.

12. The method of claim 10, wherein the path is iteratively fit in three-dimensional Cartesian space by an error minimization solver using at least i) the diameter of each wheel or the ratio of the diameters of the two wheels, ii) the wheel speed of each wheel, iii) the track width, iv) the three components of an initial position in the Cartesian space corresponding to the first geographic position, and v) an initial heading of the vehicle.

13. The method of claim 10, wherein associating each photo with the photo request used to generate that photo by comparing camera time values of a plurality of photos with computer time values of a plurality of photo requests comprises using a piecewise affine transformation.

14. The method of claim 10, wherein the lag value is quantified by comparing camera time values of a plurality of photos with computer time values of a plurality of photo requests.

15. The method of claim 10, wherein each photo is corrected to a rectilinear perspective based on the associated vehicle orientation of the path segment corresponding to the photo and a field of view of the lens to provide a centered perpendicular image orientation for the photo.

16. A method of obtaining digital images related to geographic locations, the method comprising:

providing a moving vehicle having thereon a computer having a computer clock interconnected with a GPS transceiver and distance measurement structure configured to measure a distance traveled by the vehicle, the distance measurement structure comprising a pair of laterally spaced apart independently rotating wheels, the computer operatively interconnected with a digital camera having a camera clock mounted on the vehicle;

receiving into the computer a first geographic position from the GPS transceiver;

receiving into the computer a measurement of distance traveled by the vehicle relative to the first geographic position from the distance measurement structure;

determining when a pre-determined distance has been traveled by the vehicle based on the measurement of distance;

generating a photo request using the computer on the vehicle for the digital camera on the vehicle at the pre-determined distance, the photo request having an associated photo request time comprising a computer time value obtained from the computer clock, the photo request resulting in a shutter release by the digital camera;

taking a digital photo using the camera on the vehicle based on the shutter release, the digital photo having an associated photo time comprising a camera time value obtained from the camera clock;

repeating the steps of receiving into the computer the measurement of distance traveled by the vehicle, determining when the pre-determined distance has been traveled, generating a photo request using the computer resulting in a shutter release by the digital camera and taking a digital photo using the camera on the vehicle based on the shutter release;

receiving into the computer a second geographic position from the GPS transceiver;

storing the photos, associated photo times, photo requests, associated photo request times, first and second geographic positions and distance measurements; and

,

conducting a post-processing step remotely from the vehicle.

17. The method of claim 16, wherein the post-processing step comprises calculating a change in vehicle orientation and, using the change in orientation and distance traveled, iteratively fitting a plurality of path segments, each having a fixed length corresponding to the pre-determined distance and having an associated vehicle orientation comprising the direction in which the vehicle has turned and the number of degrees in the turn, to a path traveled by the vehicle between the first and second geographic positions.

18. The method of claim 17, wherein the post-processing step further comprises associating each photo with the photo request used to generate that photo by comparing camera time values of a plurality of photos with computer time values of a plurality of photo requests.

19. The method of claim 18, wherein the post-processing step further comprises determining a computer time value associated with each photo based on the computer time value of the photo request plus a lag value that accounts for time differences between the generation of the photo request and the taking of the photo.

20. The method of claim 19, wherein the post-processing step further comprises obtaining a location of each photo on the path based on the computer time value of the photo.

21. The method of claim 20, wherein the post-processing step further comprises correcting each photo to a rectilinear perspective.