Large-Scale Surface Reconstruction That Is Robust Against Tracking And Mapping Errors
First Claim
1. A method for reconstructing a physical space, comprising:
- obtaining a plurality of images of the physical space using a camera carried by a rig while the rig moves in the physical space through a plurality of poses;
determining an orientation and a coordinate system of each pose;
partitioning the physical space into a plurality of volumes;
for each volume of the plurality of volumes, obtaining a depth map of the physical space using a depth sensor carried by the rig;
providing a pose graph comprising a plurality of nodes and links between the nodes, each node being associated with one of the poses, one of the images associated with the one of the poses and one of the coordinate systems associated with the one of the poses, and the links defining correspondences between the images and transforms between the coordinate systems;
for each volume, anchoring the volume to one of the nodes;
updating a state of the pose graph as the plurality of images are obtained, the updating comprising updating one or more of the nodes and one or more of the links; and
reconstructing a surface in one or more of the volumes according to the state of the pose graph.
3 Assignments
0 Petitions
Accused Products
Abstract
Depth maps of a physical space are obtained using a depth sensor carried by a rig such as a robot or a head mounted display device worn by a user. Visible light images are also obtained. The images and orientation readings are used to create a pose graph which includes nodes connected by links. The nodes are associated with different poses of the rig and the corresponding images. Links between the nodes represent correspondences between the images, and transforms between coordinate systems of the nodes. As new images are captured, the pose graph is updated to reduce an accumulation of errors. Furthermore, surfaces in the physical space can be reconstructed at any time according to the current state of the pose graph. Volumes used in a surface reconstruction process are anchored to the nodes such that the positions of the volumes are adjusted as the pose graph is updated.
-
Citations
20 Claims
-
1. A method for reconstructing a physical space, comprising:
-
obtaining a plurality of images of the physical space using a camera carried by a rig while the rig moves in the physical space through a plurality of poses; determining an orientation and a coordinate system of each pose; partitioning the physical space into a plurality of volumes; for each volume of the plurality of volumes, obtaining a depth map of the physical space using a depth sensor carried by the rig; providing a pose graph comprising a plurality of nodes and links between the nodes, each node being associated with one of the poses, one of the images associated with the one of the poses and one of the coordinate systems associated with the one of the poses, and the links defining correspondences between the images and transforms between the coordinate systems; for each volume, anchoring the volume to one of the nodes; updating a state of the pose graph as the plurality of images are obtained, the updating comprising updating one or more of the nodes and one or more of the links; and reconstructing a surface in one or more of the volumes according to the state of the pose graph. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. An apparatus for reconstructing a physical space, comprising:
-
an orientation sensor; a depth sensor; a camera; and a processor in communication with the orientation sensor, the depth sensor and the camera, the processor; obtains a plurality of images of the physical space from the camera, associates a pose and a coordinate system with each image based on the orientation sensor, partitions the physical space into a plurality of volumes; for each volume of the plurality of volumes, obtains a depth map of the physical space from the depth sensor, provides a pose graph comprising a plurality of nodes and links between the nodes, each node being associated with one of the poses, one of the images associated with the one of the poses and one of the coordinate systems associated with the one of the poses, and the links define correspondences between the images and transforms between the coordinate systems; for each volume, anchor the volume to one of the nodes; update a state of the pose graph as the plurality of images are obtained; and reconstructs a surface in one or more of the volumes according to the state of the pose graph. - View Dependent Claims (13, 14, 15)
-
-
16. A processor-readable storage device having processor-readable software embodied thereon for programming a processor to perform a method for reconstructing a physical space, the method comprising:
-
obtaining a plurality of images of the physical space using a camera carried by a rig while the rig moves in the physical space; determining poses of the rig, each pose being associated with an orientation and a coordinate system; providing a pose graph based on the poses and the images, the pose graph comprising a plurality of nodes and links between the nodes; obtaining a depth map of a volume in the physical space using a depth sensor carried by the rig; anchoring the volume to one of the nodes; updating a state of the pose graph as the plurality of images are obtained, the updating comprises updating one of the coordinate systems which is associated with the one of the nodes; and reconstructing a surface in the volume according to the state of the pose graph. - View Dependent Claims (17, 18, 19, 20)
-
Specification