Systems and methods for controlling a density of visual landmarks in a visual simultaneous localization and mapping system

• US 7,573,403 B2
• Filed: 11/22/2006
• Issued: 08/11/2009
• Est. Priority Date: 12/17/2002
• Status: Expired due to Term

• Current Assignee: iRobot Corporation
• Original Assignee: Evolution Robotics Inc. (iRobot Corporation)
• Inventors: Di Bernardo, Enrico, Pirjanian, Paolo, Karlsson, L. Niklas, Goncalves, Luis Filipe Domingues
• Primary Examiner(s): Swarthout; Brent
• Application Number:
  US11/562,877
• Publication Number:
  US 20070262884A1
• Time in Patent Office: 993 Days
• Field of Search: 340/988, 340/995.1, 340/995.27, 340/995.24, 702150-160, 701 23- 28, 701207-217, 348119-142, 318/568.11, 318/568.12, 318/568.16, 318/587, 901/1, 700245-253, 382/48, 382/102, 180/168
• US Class Current: 340/995.1
• CPC Class Codes: G01C 21/12 executed aboard the object being navigated; Dead reckoning
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0234 using optical markers or beacons optical beacons per se G01S1/70
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/46 Sensing device
• CPC Class Codes: Y10S 901/47 Optical
• CPC Class Codes: Y10S 901/50 Miscellaneous

Abstract

The invention is related to methods and apparatus that use a visual sensor and dead reckoning sensors to process Simultaneous Localization and Mapping (SLAM). These techniques can be used in robot navigation. Advantageously, such visual techniques can be used to autonomously generate and update a map. Unlike with laser rangefinders, the visual techniques are economically practical in a wide range of applications and can be used in relatively dynamic environments, such as environments in which people move. One embodiment further advantageously uses multiple particles to maintain multiple hypotheses with respect to localization and mapping. Further advantageously, one embodiment maintains the particles in a relatively computationally-efficient manner, thereby permitting the SLAM processes to be performed in software using relatively inexpensive microprocessor-based computer systems.

Claims

1. A method for managing content of a landmark database in a Visual Simultaneous Localization and Mapping System (VSLAM) for a mobile device, the method comprising:

using a processor for retrieving an estimate of device pose from the VSLAM for the mobile device;

using the processor for determining which visual landmarks in a map for the VSLAM are expected to be observed by the mobile device at least partly based on the estimate of device pose;

using the processor for tracking over time whether the visual landmarks that are expected to be observed are observed; and

using the processor for removing a visual landmark from the map when the visual landmark is repeatedly not observed where the visual landmark was expected to be observed;

wherein the estimate of the device pose is based, at least in part, on the position of a plurality of visual landmarks in the map, wherein the landmark database is stored in a memory.

2. The method as defined in claim 1, wherein using the processor for removing the visual landmark further comprises using the processor for removing the visual landmark based on a combination of not observing the visual landmark and a minimum predetermined period of time for not observing the visual landmark.

3. The method as defined in claim 1, wherein repeatedly not observed further comprises not observing the visual landmark about 10 times.

4. A computer program embodied in a computer-readable medium for managing content of a landmark database in a Visual Simultaneous Localization and Mapping System (VSLAM) for a mobile device, the computer program comprising:

a module with instructions configured to retrieve an estimate of device pose from the VSLAM for the mobile device;

a module with instructions configured to determine which visual landmarks in a map for the VSLAM are expected to be observed by the mobile device at least partly based on the estimate of device pose;

a module with instructions configured to track over time whether the visual landmarks that are expected to be observed are observed; and

a module with instructions configured to remove a visual landmark from the map when the visual landmark is repeatedly not observed where the visual landmark was expected to be observed;

wherein the estimate of the device pose is based, at least in part, on the position of a plurality of visual landmarks in the map.

5. The computer program as defined in claim 4, wherein the module with instructions configured to remove the visual landmark further comprises instructions configured to remove the visual landmark based on a combination of not observing the visual landmark and a minimum predetermined period of time for not observing the visual landmark.

6. An apparatus for managing content of a landmark database in a Visual Simultaneous Localization and Mapping System (VSLAM) for a mobile device, the apparatus comprising:

means for retrieving an estimate of device pose from the VSLAM for the mobile device;

means for determining which visual landmarks in a map for the VSLAM are expected to be observed by the mobile device at least partly based on the estimate of device pose;

means for tracking over time whether the visual landmarks that are expected to be observed are observed; and

means for removing a visual landmark from the map when the visual landmark is repeatedly not observed where the visual landmark was expected to be observed;

wherein the estimate of the device pose is based, at least in part, on the position of a plurality of visual landmarks in the map.

7. The apparatus as defined in claim 6, wherein the removing means is configured to remove the visual landmark based on a combination of not observing the visual landmark and a minimum predetermined period of time for not observing the visual landmark.

8. The apparatus as defined in claim 6, wherein repeatedly not observed further comprises not observing the visual landmark about 10 times.