System for building a map and subsequent localization
First Claim
1. A simultaneous localization and mapping (SLAM) system comprising:
- a memory storing a plurality of layers of an artificial neural network trained using training data, the plurality of layers comprising a penultimate layer trained while the artificial neural network comprised the penultimate layer and a classification layer; and
at least one processor coupled to the memory and configured to;
receive sensor data descriptive of a plurality of spatial regions, the sensor data including image data;
train, using the image data, a new classification layer to identify spatial regions of the plurality of spatial regions using output from the penultimate layer;
implement a new artificial neural network using the plurality of layers and the new classification layer;
provide new sensor data descriptive of a spatial region of the plurality of spatial regions to the new artificial neural network;
receive, from the new artificial neural network, information identifying the spatial region of the plurality of spatial regions;
identify one or more keyframes associated with the spatial region; and
identify at least one camera pose using the one or more keyframes associated with the spatial region.
2 Assignments
0 Petitions
Accused Products
Abstract
SLAM systems are provided that utilize an artificial neural network to both map environments and locate positions within the environments. In some example embodiments, a sensor arrangement is used to map an environment. The sensor arrangement acquires sensor data from the various sensors and associates the sensor data, or data derived from the sensor data, with spatial regions in the environment. The sensor data may include image data and inertial measurement data that effectively describes the visual appearance of a spatial region at a particular location and orientation. This diverse sensor data may be fused into camera poses. The map of the environment includes camera poses organized by spatial region within the environment. Further, in these examples, an artificial neural network is adapted to the features of the environment by a transfer learning process using image data associated with camera poses.
14 Citations
25 Claims
-
1. A simultaneous localization and mapping (SLAM) system comprising:
-
a memory storing a plurality of layers of an artificial neural network trained using training data, the plurality of layers comprising a penultimate layer trained while the artificial neural network comprised the penultimate layer and a classification layer; and at least one processor coupled to the memory and configured to; receive sensor data descriptive of a plurality of spatial regions, the sensor data including image data; train, using the image data, a new classification layer to identify spatial regions of the plurality of spatial regions using output from the penultimate layer; implement a new artificial neural network using the plurality of layers and the new classification layer; provide new sensor data descriptive of a spatial region of the plurality of spatial regions to the new artificial neural network; receive, from the new artificial neural network, information identifying the spatial region of the plurality of spatial regions; identify one or more keyframes associated with the spatial region; and identify at least one camera pose using the one or more keyframes associated with the spatial region. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method of mapping and localizing within an environment using a computing device comprising a memory storing a plurality of layers of an artificial neural network trained using training data, the plurality of layers comprising a penultimate layer trained while the artificial neural network comprised the penultimate layer and a classification layer, the method comprising:
-
receiving sensor data descriptive of a plurality of spatial regions, the sensor data including image data; training, using the image data, a new classification layer to identify spatial regions of the plurality of spatial regions using output from the penultimate layer; implementing a new artificial neural network using the plurality of layers and the new classification layer; providing new image data descriptive of a spatial region of the plurality of spatial regions to the new artificial neural network; receiving, from the new artificial neural network, information identifying the spatial region of the plurality of spatial regions; identifying one or more keyframes associated with the spatial region; and identifying at least one camera pose using the one or more keyframes associated with the spatial region. - View Dependent Claims (12, 13, 14, 15, 16)
-
-
17. A non-transitory computer program product encoded with instructions that when executed by one or more processors cause a process for mapping and localizing within an environment using an artificial neural network to be carried out, the process comprising:
-
receiving sensor data descriptive of a plurality of spatial regions, the sensor data including image data; accessing a memory storing a plurality of layers of an artificial neural network trained using training data, the plurality of layers comprising a penultimate layer trained while the artificial neural network comprised the penultimate layer and a classification layer; training, using the image data, a new classification layer to identify spatial regions of the plurality of spatial regions using output from the penultimate layer; implementing a new artificial neural network using the plurality of layers and the new classification layer; providing new image data descriptive of a spatial region of the plurality of spatial regions to the new artificial neural network; receiving information identifying the spatial region of the plurality of spatial regions from the new artificial neural network; identifying one or more keyframes associated with the spatial region; and identifying at least one camera pose using the one or more keyframes associated with the spatial region. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24, 25)
-
Specification