Augmented Image Display Using a Camera and a Position and Orientation Sensor Unit
First Claim
1. A method for generating an augmented image, the method comprising the steps of:
- receiving combined visual image data representing a combined visual image captured with a digital camera, wherein the combined visual image data comprises;
site visual image data representing a visual image of a site; and
optical target visual image data representing a visual image of at least a portion of an optical target disposed on a reference surface of a position and orientation sensor unit;
identifying, in the combined visual image data, the optical target visual image data;
calculating, based at least in part on the optical target visual image data, a position and an orientation of the digital camera relative to the position and orientation sensor unit;
receiving measurements of a position and an orientation of the position and orientation sensor unit relative to a site reference frame;
calculating, based at least in part on the position and the orientation of the digital camera relative to the position and orientation sensor unit and based at least in part on the position and the orientation of the position and orientation sensor unit relative to the site reference frame, a position and an orientation of the digital camera relative to the site reference frame;
generating, based at least in part on the position and the orientation of the digital camera relative to the site reference frame and based at least in part on a digital model of the site, at least one graphical representation selected from the group consisting of;
a virtual image associated with the site;
information associated with the site; and
information associated with the virtual image; and
displaying on a display the at least one graphical representation superimposed on a displayed visual image selected from the group consisting of;
a displayed site visual image displaying at least a portion of the visual image of the site; and
a displayed combined visual image displaying at least a portion of the visual image of the site and displaying at least a portion of the visual image of at least a portion of the optical target.
1 Assignment
0 Petitions
Accused Products
Abstract
An augmented image is generated by capturing a visual image of a site with a digital camera, generating a virtual image or associated information from a digital model of the site, and superimposing the virtual image or associated information on the visual image. To register the digital model with the visual image, a sensor pole is introduced into the field of view, and a combined visual image of the site and an optical target on the sensor pole is captured. The position and orientation of the sensor pole with respect to the site reference frame are measured by sensors mounted on the sensor pole; the position and orientation of the digital camera with respect to the sensor pole are calculated from image analysis of the optical target on the sensor pole; and the position and orientation of the digital camera with respect to the site reference frame are calculated.
-
Citations
21 Claims
-
1. A method for generating an augmented image, the method comprising the steps of:
-
receiving combined visual image data representing a combined visual image captured with a digital camera, wherein the combined visual image data comprises; site visual image data representing a visual image of a site; and optical target visual image data representing a visual image of at least a portion of an optical target disposed on a reference surface of a position and orientation sensor unit; identifying, in the combined visual image data, the optical target visual image data; calculating, based at least in part on the optical target visual image data, a position and an orientation of the digital camera relative to the position and orientation sensor unit; receiving measurements of a position and an orientation of the position and orientation sensor unit relative to a site reference frame; calculating, based at least in part on the position and the orientation of the digital camera relative to the position and orientation sensor unit and based at least in part on the position and the orientation of the position and orientation sensor unit relative to the site reference frame, a position and an orientation of the digital camera relative to the site reference frame; generating, based at least in part on the position and the orientation of the digital camera relative to the site reference frame and based at least in part on a digital model of the site, at least one graphical representation selected from the group consisting of; a virtual image associated with the site; information associated with the site; and information associated with the virtual image; and displaying on a display the at least one graphical representation superimposed on a displayed visual image selected from the group consisting of; a displayed site visual image displaying at least a portion of the visual image of the site; and a displayed combined visual image displaying at least a portion of the visual image of the site and displaying at least a portion of the visual image of at least a portion of the optical target. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A system for generating an augmented image, the system comprising:
-
a digital camera; a sensor pole comprising; a sensor head comprising at least one sensor; a shaft coupled to the sensor head; and an optical target disposed on a reference surface of the shaft; at least one sensor measurement system operably coupled to the at least one sensor; a display; and a computational system operably coupled to the digital camera, the at least one sensor measurement system, and the display; wherein; when the digital camera and the sensor pole are disposed such that a site and at least a portion of the optical target are within a field of view of the digital camera, the digital camera is configured to; capture a combined visual image, wherein the combined visual image comprises; a visual image of the site; and a visual image of the at least a portion of the optical target; represent the combined visual image as combined visual image data, wherein the combined visual image data comprises; site visual image data representing the visual image of the site; and optical target visual image data representing the visual image of the at least a portion of the optical target; and transmit the combined visual image data; the at least one sensor and the at least one sensor measurement system are configured to; measure a position and an orientation of the sensor pole relative to a site reference frame; represent the position and the orientation of the sensor pole as measurement data; and transmit the measurement data; and the computational system is configured to; receive the combined visual image data; receive the measurement data; identify, in the combined visual image data, the optical target visual image data; calculate, based at least in part on the optical target visual image data, a position and an orientation of the digital camera relative to the position and orientation sensor unit; calculate, based at least in part on the position and the orientation of the digital camera relative to the position and orientation sensor unit and based at least in part on the position and the orientation of the position and orientation sensor unit relative to the site reference frame, a position and an orientation of the digital camera relative to the site reference frame; generate, based at least in part on the position and the orientation of the digital camera relative to the site reference frame and based at least in part on a digital model of the site, at least one graphical representation selected from the group consisting of; a virtual image associated with the site; information associated with the site; and information associated with the virtual image; and display the at least one graphical representation superimposed on a displayed visual image selected from the group consisting of; a displayed site visual image displaying at least a portion of the visual image of the site; and a displayed combined visual image displaying at least a portion of the visual image of the site and displaying at least a portion of the visual image of at least a portion of the optical target. - View Dependent Claims (9, 10, 11, 12, 13, 14, 15)
-
-
16. An apparatus for generating an augmented image, the apparatus comprising:
-
at least one processor unit; memory operably coupled to the at least one processor unit; a display operably coupled to the at least one processor unit; and at least one data storage device operably coupled to the at least one processor unit, wherein the at least one data storage device stores computer program instructions which, when executed by the at least one processor unit, causes the at least one processor unit to perform the method comprising the steps of; receiving combined visual image data representing a combined visual image captured with a digital camera, wherein the combined visual image data comprises; site visual image data representing a visual image of a site; and optical target visual image data representing a visual image of at least a portion of an optical target disposed on a reference surface of a position and orientation sensor unit; identifying, in the combined visual image data, the optical target visual image data; calculating, based at least in part on the optical target visual image data, a position and an orientation of the digital camera relative to the position and orientation sensor unit; receiving measurements of a position and an orientation of the position and orientation sensor unit relative to a site reference frame; calculating, based at least in part on the position and the orientation of the digital camera relative to the position and orientation sensor unit and based at least in part on the position and the orientation of the position and orientation sensor unit relative to the site reference frame, a position and an orientation of the digital camera relative to the site reference frame; generating, based at least in part on the position and the orientation of the digital camera relative to the site reference frame and based at least in part on a digital model of the site, at least one graphical representation selected from the group consisting of; a virtual image associated with the site; information associated with the site; and information associated with the virtual image; and displaying on the display the at least one graphical representation superimposed on a displayed visual image selected from the group consisting of; a displayed site visual image displaying at least a portion of the visual image of the site; and a displayed combined visual image displaying at least a portion of the visual image of the site and displaying at least a portion of the visual image of the at least a portion of the optical target. - View Dependent Claims (17, 18, 19, 20, 21)
-
Specification