Scene documentation

• US 9,639,941 B2
• Filed: 10/13/2015
• Issued: 05/02/2017
• Est. Priority Date: 12/28/2012
• Status: Active Grant

• Current Assignee: Trimble Inc.
• Original Assignee: Trimble Inc.
• Inventors: Rudow, Richard, McFadden, Chad, Wold, Robert, Kasirajan, Venkateswaran, Talbot, Nicholas C., Loomis, Peter Van Wyck, Weisenburger, Shawn D., Janky, James M., McCusker, Michael V.
• Primary Examiner(s): Ton, Tri T
• Application Number:
  US14/882,243
• Publication Number:
  US 20160035096A1
• Time in Patent Office: 567 Days
• Field of Search: 356614-624
• CPC Class Codes: G01B 11/26 for measuring angles or tapers; for testing the alignment of axes
• CPC Class Codes: G01S 19/25 involving aiding data received from a cooperating element, e.g. assisted GPS
• CPC Class Codes: G01S 19/41 Differential correction, e.g. DGPS [differential GPS]
• CPC Class Codes: G01S 19/43 using carrier phase measurements, e.g. kinematic positioning; using long or short baseline interferometry
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04M 3/00 Automatic or semi-automatic exchanges
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/185 by embedding added-value information into content, e.g. geo-tagging
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

A plurality of images are captured by an image capturing device that is an integral part of the mobile data collection platform from at least two different perspectives that depict a point of interest in a scene. Coincident with capture of each of the plurality of images, orientation information is obtained via orientation sensors of the mobile data collection platform, a position fix of an antenna associated with the mobile data collection platform is determined, and a position of an entrance pupil of the image capturing device is calculated. Scale information associated with at least one of the images is captured. Scene data comprises the images, the orientation information and the entrance pupil positions. A three dimensional position of the point of interest at the scene is determined based on photogrammetric image processing of the scene data.

Claims

1. A method of scene documentation, the method comprising:

with a mobile data collection platform;

capturing, by an image capturing device that is an integral part of the mobile data collection platform, a plurality of images from at least two different perspectives that depict a point of interest in the scene, and coincident with capture of each of the plurality of images;

obtaining orientation information comprising a tilt angle and an azimuth angle via orientation sensors of the mobile data collection platform;

determining a position fix of an antenna associated with the mobile data collection platform, in three dimensions, based on raw Global Navigation Satellite System (GNSS) observables captured by the antenna; and

calculating a position of an entrance pupil of the image capturing device, in three dimensions, based on the orientation information and a known offset between the entrance pupil and the antenna, wherein each of the images is associated with one of a plurality of entrance pupil positions, wherein the determining, and the calculating are performed by one or more hardware processors that are part of the mobile data collection platform and outside of an internal GNSS chipset of the mobile data collection platform; and

capturing scale information associated with at least one of the images; and

with a server located remote from mobile data collection platform;

receiving from the mobile data collection platform, via a communication network, scene data comprising;

the images, the orientation information, and the entrance pupil positions; and

determining by one or more hardware processors of the server, based on photogrammetric image processing of the scene data, a three dimensional position of the point of interest at the scene.

2. The method as recited in claim 1, further comprising:

determining by the server, based on photogrammetric image processing of the scene data, a second three dimensional position of an additional point of interest at the scene.

3. The method as recited in claim 2, wherein the point of interest is a first point of interest and wherein further comprising:

calculating, by the server, a vector between the first point of interest and the additional point of interest.

4. The method as recited in claim 3, further comprising:

annotating, by the server, information associated with the vector, the point of interest, and the additional point of interest onto at least one of the images to create an annotated image.

5. The method as recited in claim 4, further comprising:

outputting, by the server, the annotated image.

6. The method as recited in claim 2, further comprising:

receiving, by the server, an identification of the additional point of interest.

7. The method as recited in claim 1, wherein the method further comprises:

with a depth camera array coupled with the mobile data collection platform, capturing at least one set of images; and

generating a depth-based scale factor based on the at least one set of images.

8. A system comprising:

a mobile data collection platform configured to;

capture, by an image capturing device that is an integral part of the mobile data collection platform, a plurality of images from at least two different perspectives that depict a point of interest in an scene, and coincident with capture of each of the plurality of images;

obtain orientation information comprising a tilt angle and an azimuth angle via orientation sensors of the mobile data collection platform;

determine a position fix of an antenna associated with the mobile data collection platform, in three dimensions, based on raw Global Navigation Satellite System (GNSS) observables captured by the antenna; and

calculate a position of an entrance pupil of the image capturing device, in three dimensions, based on the orientation information and a known offset between the entrance pupil and the antenna, wherein each of the images is associated with one of a plurality of entrance pupil positions, wherein one or more hardware processors that are part of the mobile data collection platform and outside of an internal GNSS chipset of the mobile data collection platform determine the position fix of the antenna, and calculate the entrance pupil position; and

capture scale information associated with at least one of the plurality of images; and

a server located remote from mobile data collection platform and configured to;

receive from the mobile data collection platform, via a communication network, scene data comprising;

the images, the orientation information, and the entrance pupil positions; and

determine by one or more hardware processors of the server, based on photogrammetric image processing of the scene data, a three dimensional position of the point of interest at the scene.

9. The system as recited in claim 8, wherein the server is further configured to:

determine, based on photogrammetric image processing of the scene data, a second three dimensional position of an additional point of interest at the scene.

10. The system as recited in claim 9, wherein the point of interest is a first point of interest and wherein the server is further configured to:

calculate a vector between the first point of interest and the additional point of interest.

11. The system as recited in claim 10, wherein the server is further configured to:

annotate information associated with the vector, the point of interest, and the additional point of interest onto at least one of the images to create an annotated image.

12. The system as recited in claim 11, wherein the server is further configured to:

output the annotated image.

13. The system as recited in claim 9, wherein the server is further configured to:

receive an identification of the additional point of interest.

14. The system as recited in claim 8, further comprising a depth camera array coupled with the mobile collection platform, wherein the depth camera array is configured to capture at least one set of images;

and

wherein the server is further configured to generate a depth-based scale factor based on the at least one set of images.

15. A mobile data collection platform comprising:

an image capturing device that is an integral part of the mobile data collection platform and is configured to capture a plurality of images from at least two different perspectives that depict a point of interest in an scene,

one or more hardware processors configured to;

obtain, coincident with capture of each of the plurality of images, orientation information comprising a tilt angle and an azimuth angle via orientation sensors of the mobile data collection platform;

determine, coincident with capture of each of the plurality of images, a position fix of an antenna associated with the mobile data collection platform, in three dimensions, based on raw Global Navigation Satellite System (GNSS) observables captured by the antenna; and

calculate, coincident with capture of each of the plurality of images, a position of an entrance pupil of the image capturing device, in three dimensions, based on the orientation information and a known offset between the entrance pupil and the antenna, wherein each of the images is associated with one of a plurality of entrance pupil positions,

wherein the one or more hardware processors are part of the mobile data collection platform and outside of an internal GNSS chipset of the mobile data collection platform; and

capture scale information associated with at least one of the plurality of images; and

a transceiver configured to;

send scene data, via a communication network, to a server located remote from mobile data collection platform, the scene data comprising;

the images, the orientation information, and the entrance pupil positions; and

receive a three dimensional position of the point of interest at the scene which has been determined by one or more hardware processors of the server based on photogrammetric image processing of the scene data.

16. A method of scene documentation, the method comprising:

with a mobile data collection platform;

capturing, by an image capturing device that is an integral part of the mobile data collection platform, a plurality of images from at least two different perspectives that depict a point of interest in the scene, and coincident with capture of each of the plurality of images;

obtaining orientation information comprising a tilt angle and an azimuth angle provided by orientation sensors of the mobile data collection platform;

determining a position fix of an antenna associated with the mobile data collection platform, in three dimensions, based on raw Global Navigation Satellite System (GNSS) observables captured by the antenna;

calculating a position of an entrance pupil of the image capturing device, in three dimensions, based on the orientation information and a known offset between the entrance pupil and the antenna, wherein each of the images is associated with one of a plurality of entrance pupil positions, and wherein scene data comprises the images, the orientation information, and the entrance pupil positions; and

capturing scale information associated with at least one of the plurality of images; and

determining based on photogrammetric image processing of the scene data, a three dimensional position of the point of interest at the scene,

wherein one or more hardware processors, that are part of the mobile data collection platform and outside of an internal GNSS chipset of the mobile data collection platform, perform the determining of the position fix of the antenna, the calculating of the entrance pupil position, the capturing of the scale information, and the determining of the three dimensional position.

17. The method as recited in claim 16, further comprising:

determining by the one or more hardware processors, based on photogrammetric image processing of the scene data, a second three dimensional position of an additional point of interest at the scene.

18. The method as recited in claim 17, wherein the point of interest is a first point of interest, the method further comprising:

calculating, by the one or more hardware processors, a vector between the first point of interest and the additional point of interest.

19. The method as recited in claim 18, further comprising:

annotating, by the one or more hardware processors, information associated with the vector, the point of interest, and the additional point of interest onto at least one of the images to create an annotated image.

20. The method as recited in claim 19, further comprising:

outputting, by the one or more hardware processors, the annotated image.

21. The method as recited in claim 17, further comprising:

receiving, by the one or more hardware processors, an identification of the additional point of interest.

22. The method as recited in claim 16, wherein the method further comprises:

with a depth camera array coupled with the mobile data collection platform, capturing at least one set of images; and

generating a depth-based scale factor based on the at least one set of images.

23. A mobile data collection platform comprising:

an image capturing device that is an integral part of the mobile data collection platform and is configured to capture a plurality of images from at least two different perspectives that depict a point of interest in an scene,

one or more hardware processors configured to;

obtain, coincident with capture of each of the plurality of images, orientation information comprising a tilt angle and an azimuth angle provided by orientation sensors of the mobile data collection platform;

determine, coincident with capture of each of the plurality of images, a position fix of an antenna associated with the mobile data collection platform, in three dimensions, based on raw Global Navigation Satellite System (GNSS) observables captured by the antenna;

calculate, coincident with capture of each of the plurality of images, a position of an entrance pupil of the image capturing device, in three dimensions, based on the orientation information and a known offset between the entrance pupil and the antenna, wherein each of the images is associated with one of a plurality of entrance pupil positions, and wherein scene data comprises the images, the orientation information, and the entrance pupil positions;

capture, coincident with capture of at least one of the plurality of images, scale information; and

determine based on photogrammetric image processing of the scene data, a three dimensional position of the point of interest at the scene,

wherein the one or more hardware processors are part of the mobile data collection platform and outside of an internal GNSS chipset of the mobile data collection platform.

24. A method of scene documentation, the method comprising:

with a mobile data collection platform;

capturing, by an image capturing device that is an integral part of the mobile data collection platform, an image from a perspective that depicts a point of interest in the scene, and coincident with capture of the image;

obtaining orientation information comprising a tilt angle and an azimuth angle provided by orientation sensors of the mobile data collection platform;

determining a position fix of an antenna associated with the mobile data collection platform, in three dimensions, based on raw Global Navigation Satellite System (GNSS) observables captured by the antenna; and

calculating a position of an entrance pupil of the image capturing device, in three dimensions, based on the orientation information and a known offset between the entrance pupil and the antenna, wherein the determining and the calculating are performed by one or more hardware processors that are part of the mobile data collection platform and outside of an internal GNSS chipset of the mobile data collection platform, and wherein scene data comprises;

the image, the orientation information, and the position of the entrance pupil;

capturing scale information depicted in the image; and

determining, based on photogrammetric image processing of the scene data, a three dimensional position of the point of interest at the scene.

25. A method as recited in claim 24, wherein

the one or more hardware processors perform:

the obtaining of the orientation information;

the determining of the position fix;

the calculating of the position of the entrance pupil;

the capturing of the scale information; and

the determining of the three dimensional position.

26. A method as recite in claim 24, wherein the method further comprises:

with a server located remotely from the mobile data collection platform;

performing the determining of the three dimensional position.

27. A method as recited in claim 24, wherein the capturing of the scale information further comprises:

receiving an electronic measured distance between a location where the image was captured and a point of interest captured in the image.

28. The method as recited in claim 24, wherein the method further comprises:

capturing a time stamp coincident with the capturing of the image.

29. The method as recited by claim 24, wherein the antenna is selected from a group consisting of an internal antenna that is part of the mobile data collection platform and an external antenna that is located outside of the mobile data collection platform, wherein the external antenna is maintained in a known spatial relationship with the entrance pupil.