Systems and methods for controlling a density of visual landmarks in a visual simultaneous localization and mapping system
First Claim
1. A method for efficiently using memory in an autonomous localization and mapping system for a mobile device, the method comprising:
- observing features in the environment with a visual sensor coupled to the mobile device;
receiving dead reckoning data from a sensor coupled to the mobile device;
maintaining a plurality of landmarks for one or more maps in a first memory, where a landmark is referenced in a map in a global reference frame;
copying data corresponding to the plurality of landmarks for the one or more maps from the first memory to a second memory, where the second memory operates faster than the first memory;
using the data stored in the second memory when searching for matching landmarks;
estimating a position for the mobile device within the global reference frame of the one or more maps at least partly in response to the observed features and the dead reckoning data; and
selectively maintaining only a first portion of the plurality of landmarks for the one or more maps in the second memory and releasing a second portion such that memory resources previously allocated to the second portion can be returned to the system, where the first portion corresponds to landmarks that are likely to be in the same vicinity of the global reference frame as the position estimate for the mobile device.
7 Assignments
0 Petitions
Accused Products
Abstract
The invention is related to methods and apparatus that use a visual sensor and dead reckoning sensors to process Simultaneous Localization and Mapping (SLAM). These techniques can be used in robot navigation. Advantageously, such visual techniques can be used to autonomously generate and update a map. Unlike with laser rangefinders, the visual techniques are economically practical in a wide range of applications and can be used in relatively dynamic environments, such as environments in which people move. One embodiment further advantageously uses multiple particles to maintain multiple hypotheses with respect to localization and mapping. Further advantageously, one embodiment maintains the particles in a relatively computationally-efficient manner, thereby permitting the SLAM processes to be performed in software using relatively inexpensive microprocessor-based computer systems.
115 Citations
51 Claims
-
1. A method for efficiently using memory in an autonomous localization and mapping system for a mobile device, the method comprising:
-
observing features in the environment with a visual sensor coupled to the mobile device;
receiving dead reckoning data from a sensor coupled to the mobile device;
maintaining a plurality of landmarks for one or more maps in a first memory, where a landmark is referenced in a map in a global reference frame;
copying data corresponding to the plurality of landmarks for the one or more maps from the first memory to a second memory, where the second memory operates faster than the first memory;
using the data stored in the second memory when searching for matching landmarks;
estimating a position for the mobile device within the global reference frame of the one or more maps at least partly in response to the observed features and the dead reckoning data; and
selectively maintaining only a first portion of the plurality of landmarks for the one or more maps in the second memory and releasing a second portion such that memory resources previously allocated to the second portion can be returned to the system, where the first portion corresponds to landmarks that are likely to be in the same vicinity of the global reference frame as the position estimate for the mobile device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A computer program embodied in a tangible medium for efficiently using memory in an autonomous localization and mapping system for a mobile device, the computer program comprising:
-
a module with instructions configured to observe features in the environment with a visual sensor coupled to the mobile device;
a module with instructions configured to receive dead reckoning data from a sensor coupled to the mobile device;
a module with instructions configured to maintain a plurality of landmarks for one or more maps in a first memory, where a landmark is referenced in a map in a global reference frame;
a module with instructions configured to copy data corresponding to the plurality of landmarks for the one or more maps from the first memory to a second memory, where the second memory operates faster than the first memory;
a module with instructions configured to use the data stored in the second memory when searching for matching landmarks;
a module with instructions configured to estimate a position for the mobile device within the global reference frame of the one or more maps at least partly in response to the observed features and the dead reckoning data; and
a module with instructions configured to selectively maintain only a first portion of the plurality of landmarks for the one or more maps in the second memory and releasing a second portion such that memory resources previously allocated to the second portion can be returned to the system, where the first portion corresponds to landmarks that are likely to be in the same vicinity of the global reference frame as the position estimate for the mobile device. - View Dependent Claims (13, 14)
-
-
15. A circuit for efficiently using memory in an autonomous localization and mapping system for a mobile device, the circuit comprising:
-
a circuit configured to observe features in the environment with a visual sensor coupled to the mobile device;
a circuit configured to receive dead reckoning data from a sensor coupled to the mobile device;
a circuit configured to maintain a plurality of landmarks for one or more maps in a first memory, where a landmark is referenced in a map in a global reference frame;
a circuit configured to copy data corresponding to the plurality of landmarks for the one or more maps from the first memory to a second memory, where the second memory operates faster than the first memory;
a circuit configured to use the data stored in the second memory when searching for matching landmarks;
a circuit configured to estimate a position for the mobile device within the global reference frame of the one or more maps at least partly in response to the observed features and the dead reckoning data; and
a circuit configured to selectively maintain only a first portion of the plurality of landmarks for the one or more maps in the second memory and releasing a second portion such that memory resources previously allocated to the second portion can be returned to the system, where the first portion corresponds to landmarks that are likely to be in the same vicinity of the global reference frame as the position estimate for the mobile device. - View Dependent Claims (16, 17, 18)
-
-
19. A method for managing content of a landmark database in a Visual Simultaneous Localization and Mapping System (VSLAM) for a mobile device, the method comprising:
-
retrieving an estimate of device pose for the mobile device;
determining which landmarks in a map are expected to be observed by the mobile device at least partly based on the estimate of device pose;
tracking over time whether the landmarks that are expected to be observed are observed; and
removing a landmark from the map when the landmark is repeatedly not observed where the landmark was expected to be observed. - View Dependent Claims (20, 21)
-
-
22. A computer program embodied in a tangible medium for managing content of a landmark database in a Visual Simultaneous Localization and Mapping System (VSLAM) for a mobile device, the computer program comprising:
-
a module with instructions configured to retrieve an estimate of device pose for the mobile device;
a module with instructions configured to determine which landmarks in a map are expected to be observed by the mobile device at least partly based on the estimate of device pose;
a module with instructions configured to track over time whether the landmarks that are expected to be observed are observed; and
a module with instructions configured to remove a landmark from the map when the landmark is repeatedly not observed where the landmark was expected to be observed. - View Dependent Claims (23)
-
-
24. A method for managing content of a landmark database in Visual Simultaneous Localization and Mapping System (VSLAM) for a mobile device, the method comprising:
-
visually obtaining a measurement of a landmark;
analyzing a quality of the measurement, where the quality of the measurement provides an indication of whether the measurement is an outlier;
storing an indication when the measurement has been identified as an outlier; and
removing a landmark from the map when the measurements of the landmark are repeatedly identified as outliers. - View Dependent Claims (25, 26, 27, 28, 29, 30, 31, 32)
-
-
33. A computer program embodied in a tangible medium for managing content of a landmark database in Visual Simultaneous Localization and Mapping System (VSLAM) for a mobile device, the computer program comprising:
-
a module with instructions configured to visually obtain a measurement of a landmark;
a module with instructions configured to analyze a quality of the measurement, where the quality of the measurement provides an indication of whether the measurement is an outlier;
a module with instructions configured to store an indication when the measurement has been identified as an outlier; and
a module with instructions configured to remove a landmark from the map when the measurements of the landmark are repeatedly identified as outliers. - View Dependent Claims (34, 35)
-
-
36. A method for efficiently managing content of a landmark database in an autonomous localization and mapping system, the method comprising:
-
determining a density of landmarks in one or more maps; and
discontinuing the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold. - View Dependent Claims (37, 38, 39, 40)
-
-
41. A circuit for managing content of a landmark database in an autonomous localization and mapping system, the circuit comprising:
-
a means for determining a density of landmarks in one or more maps; and
a means for discontinuing the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold. - View Dependent Claims (42, 43, 44)
-
-
45. A computer program embodied in a tangible medium for efficiently managing content of a landmark database in an autonomous localization and mapping system, the computer program comprising:
-
a module with instructions configured to determine a density of landmarks ill one or more maps; and
a module with instructions configured to discontinue the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold. - View Dependent Claims (46, 47)
-
-
48. A circuit adapted to efficiently manage content of a landmark database in an autonomous localization and mapping system, the circuit comprising:
-
a circuit configured to determine a density of landmarks in one or more maps; and
a circuit configured to discontinue the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold. - View Dependent Claims (49, 50, 51)
-
Specification