Performing data collection based on internal raw observables using a mobile data collection platform

• US 9,538,336 B2
• Filed: 10/15/2014
• Issued: 01/03/2017
• Est. Priority Date: 12/28/2012
• Status: Active Grant

• Current Assignee: Trimble Inc.
• Original Assignee: Trimble Inc.
• Inventors: Rudow, Richard, Kasirajan, Venkateswaran, Wold, Robert, Talbot, Nicholas C., Loomis, Peter Van Wyck, Weisenburger, Shawn D., Janky, James M., McCusker, Michael V.
• Primary Examiner(s): Hong, Dung
• Application Number:
  US14/515,317
• Publication Number:
  US 20150045058A1
• Time in Patent Office: 811 Days
• US Class Current: 1/1
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

An image that includes a point of interest is captured using an image capturing device that is part of the mobile data collection platform. Raw observables are obtained from a GNSS chipset that is internal to the mobile data collection platform. A position fix of the mobile data collection platform is determined based on the raw observables where the position fix defines a location of an antenna. A location of an entrance pupil is calculated as an offset of the location of the antenna. Orientation information comprising a tilt angle and an azimuth angle is determined. The position fix and the orientation information are associated with a three dimensional location that the mobile data collection platform is at when the image was captured. Scale information is captured. The image, the position fix, the scale information, and the orientation information are stored in hardware memory of the mobile data collection platform.

Claims

1. A method of performing data collection using a mobile data collection platform, the method comprising:

capturing an image that includes a point of interest, wherein the capturing is performed by an image capturing device that is an integral part of the mobile data collection platform;

obtaining raw observables from a Global Navigation Satellite System (GNSS) chipset that is internal to the mobile data collection platform;

determining a position fix based on the raw observables, wherein the position fix defines a location of an antenna associated with the GNSS chipset;

calculating a location of an entrance pupil of the image capturing device as an offset from the location of the antenna;

determining orientation information comprising a tilt angle and an azimuth angle of the mobile data collection platform, wherein the tilt angle is between a y platform axis of the mobile data collection platform and a local gravity vector, and the azimuth angle is between a reference direction and a pointing vector of the mobile data collection platform, wherein the pointing vector is in a known orientation relative to a compass heading, and the position fix and the orientation information are associated with a three dimensional location of the mobile data collection platform when the image was captured;

capturing scale information; and

storing the image, the position fix, the scale information, and the orientation information in hardware memory of the mobile data collection platform, wherein the capturing, the obtaining, the determining of the position fix, the calculating, the determining of the orientation information, and the storing are performed by one or more hardware processors that are part of the mobile data collection platform, and wherein the one or more hardware processors are outside of the GNSS chipset of the mobile data collection platform.

2. The method as recited by claim 1, wherein the capturing of the scale information further comprises:

capturing depiction of an object with at least one known dimension, wherein the image depicts the object with the at least one known dimension.

3. The method as recited by claim 1, wherein the capturing of the scale information further comprises:

capturing a first image and a second image that both depict the point of interest, wherein the first image is captured from a first position and the second image is captured from a second position.

4. The method as recited by claim 3, wherein the capturing of the scale information further comprises:

calculating a distance between the first position and the second position.

5. The method as recited by claim 1, wherein the method further comprises:

determining a first position, wherein the first position is selected from a group consisting of a position of a georeference point of interest and a position fix of the mobile data collection platform, where the point of interest is in a field of view of the image capturing device;

determining a second position, wherein the second position is selected from a group consisting of the position of the georeference point of interest and the position fix of the mobile data collection platform, where the point of interest is in the field of view; and

determining a reference distance between the first position and the second position.

6. The method as recited by claim 1, wherein the method further comprises:

capturing angular displacement from a first point on a scalar reference to a second point on the scalar reference visible in the image as given by a pixel count from the first point to the second point.

7. The method as recited by claim 1, wherein the method further comprises:

calibrating the mobile data collection platform by determining a pixel calibration datum providing angular displacement of each pixel, in two dimensions, depicted in a calibration image taken with the image capturing device.

8. The method as recited by claim 1, wherein the method further comprises:

calibrating the mobile data collection platform by determining an acceptable region in a calibration image taken with the image capturing device, wherein the acceptable region includes a subset of pixels of the calibration image where the pixels do not exceed a specified level of distortion.

9. The method as recited by claim 1, wherein the method further comprises:

receiving outline information describing an outline of the point of interest; and

designating the point of interest as a user specified point of interest based on the outline information.

10. The method as recited by claim 1, wherein the method further comprises:

designating the point of interest as a user specified point of interest based on orientation information from an orientation system associated with the mobile data collection platform when crosshairs are aligned with the point of interest and an image capture button is pressed to capture the image.

11. The method as recited by claim 1, wherein the method further comprises:

designating the point of interest as a user specified point of interest based on an annotation from an image editor associated with the mobile data collection platform.

12. The method as recited by claim 1, wherein the method further comprises:

performing feature identification on at least a subset of the image.

13. The method as recited by claim 1, wherein the method further comprises:

performing pattern recognition on at least a subset of the image.

14. The method as recited by claim 1, wherein the method further comprises:

displaying crosshair display overlay on a display of the mobile data collection platform;

displaying a photographic image of the point of interest in relation to the crosshair display overlay; and

positioning the photographic image with respect to the crosshair display overlay based on an alignment of an entrance pupil of the mobile data collection platform with the point of interest.

15. The method as recited by claim 1, wherein the method further comprises:

displaying a bubble level overlay on a display of the mobile data collection platform;

displaying a graphical bubble in relation to the bubble level overlay; and

positioning the graphical bubble with respect to the bubble level overlay based on a degree of tilt of the mobile data collection platform.

16. The method as recited by claim 1, wherein the method further comprises:

determining one or more geometric offsets between the entrance pupil and the antenna.

17. A mobile data collection platform, the mobile data collection platform comprising:

a cellular device that includes;

an antenna;

a display; and

an internal Global Navigation Satellite System (GNSS) chipset, wherein the antenna receives GNSS positioning signals that define a location of the antenna and wherein the display is for displaying images;

an image capturing device that captures an image through an entrance pupil;

an orientation system that includes a tilt sensor and a compass and determines orientation information that includes tilt angle obtained from the tilt sensor and heading information obtained from the compass, wherein the tilt angle is between the mobile data collection platform and a local gravity vector, and the heading information is an azimuth angle between a pointing vector emanating from the entrance pupil of the image capturing device and a reference direction;

hardware memory that stores the image, a position fix, and the orientation information; and

one or more hardware processors that executes instructions which;

receive raw observables from the internal GNSS chipset;

capture the image with the image capturing device, wherein the image depicts a point of interest and wherein the image is captured while the antenna is at the location;

determine the position fix associated with the mobile data collection platform based on the raw observables, wherein the position fix provides the location of the antenna in a GNSS coordinate system;

calculate a location of the entrance pupil as an offset from the location of the antenna;

access the orientation information from the orientation system, wherein accessing the orientation information includes receiving tilt angle information and azimuth angle information, wherein the orientation information and the heading information are associated with a three dimensional location of the mobile data collection platform when the image was captured;

determine a direction of the pointing vector; and

store the image, the position fix, the orientation information, and the heading information in the hardware memory of the mobile data collection platform,

wherein the hardware memory and the one or more hardware processors are outside of the internal GNSS chipset.

18. The mobile data collection platform of claim 17, wherein the raw observables include raw pseudoranges and at least one of real carrier phase information and Doppler Shift Information.

19. The mobile data collection platform of claim 17, wherein the image capturing device is embedded in the cellular device as an integrated subsystem in a known location relative to the antenna of the cellular device.

20. The mobile data collection platform of claim 17, wherein the display emulates a pair of crosshairs indicative of an axial direction of the entrance pupil of the image capturing device when displaying the image.

21. The mobile data collection platform of claim 17, wherein the mobile data collection platform includes a graphical user interface, and wherein the mobile data collection platform further comprises:

a bubble level processing logic that displays a virtual representation of a bubble on the display, wherein the bubble level processing logic is coupled with the orientation system.

22. The mobile data collection platform of claim 21, wherein a position of the virtual representation of the bubble on the display is determined based on the orientation information.

23. The mobile data collection platform of claim 22, wherein the virtual representation of the bubble is visible when the tilt angle from the orientation system is less than a selected number in a range from 1 to 10 degrees from a horizontal reference.

24. The mobile data collection platform of claim 17, wherein the one or more hardware processors execute instructions that:

provide smoothed pseudoranges by smoothing raw pseudoranges based on carrier phase information, wherein the raw observables include the raw pseudoranges;

provide corrected smoothed pseudoranges by correcting the smoothed pseudoranges based on external corrections; and

determine the position fix based on the corrected smoothed pseudoranges.

25. The mobile data collection platform of claim 17, wherein the image is a first image, the location is a first location, the position fix is a first position fix, and the orientation information is first orientation information, wherein:

the one or more hardware processors further executes instructions that;

capture a second image with the image capturing device, wherein the second image includes the point of interest;

determines a second position fix of the mobile data collection platform, wherein the second position fix correlates with a second location of the mobile data collection platform in the GNSS coordinate system;

accesses second orientation information from the orientation system, wherein the second orientation information includes a second tilt angle and a second azimuth angle that correspond with the mobile data collection platform when the second image was captured;

stores the second image, the second position fix, and the second orientation information in the hardware memory;

calculates a distance between the first location and the second location by calculating a difference between the first position fix and the second position fix, wherein the distance is scale information; and

stores the distance in the hardware memory.

26. A non-transitory computer readable storage medium having computer readable instructions stored thereon for causing a computer system to perform a method of performing data collection using a mobile data collection platform, the method comprising:

capturing an image that depicts a point of interest using a cellular device;

designating the point of interest as a user specified point of interest based on information selected from a group consisting of annotation from an image editor, an outline of the point of interest, and a photographic image of the point of interest being visibly displayed within a crosshair display overlay;

obtaining raw observables from a Global Navigation Satellite System (GNSS) chipset that is an integral part of the cellular device;

determining a position fix based on the raw observables;

determining a three dimensional position of the cellular device based on a local gravity vector and the position fix;

capturing scale information; and

storing the image, the scale information, and the three dimensional position in hardware memory of the cellular device,

wherein the local gravity vector is local with respect to the cellular device,

wherein the cellular device is at the three dimensional position when the image is captured.

27. The non-transitory computer readable storage medium as recited by claim 26, wherein the method further comprises:

obtaining a tilt angle and a tilt direction of the cellular device; and

determining the three dimensional position associated with the cellular device based at least in part on the tilt angle and the tilt direction in relation to the local gravity vector.

28. The non-transitory computer readable storage medium as recited by claim 26, wherein the method further comprises:

determining the three dimensional position associated with the cellular device based at least in part on geometric information relating an antenna of the cellular device with an entrance pupil center of the cellular device.

29. The non-transitory computer readable storage medium as recited by claim 26, wherein the method further comprises:

displaying a bubble level overlay on a display of the cellular device;

displaying a graphical bubble in relation to the bubble level overlay; and

positioning the graphical bubble with respect to the bubble level overlay based on a degree of tilt of the cellular device.

30. The non-transitory computer readable storage medium as recited by claim 26, wherein the cellular device is not required to be perpendicular to the local gravity vector at a time of the capturing, the determining of the position fix, and the determining of the three dimensional position.