Systems and methods for using multiple hypotheses in a visual simultaneous localization and mapping system

• US 8,830,091 B2
• Filed: 08/12/2013
• Issued: 09/09/2014
• Est. Priority Date: 12/17/2002
• Status: Expired due to Fees

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Karlsson, L. Niklas, Pirjanian, Paolo, Goncalves, Luis Filipe Domingues, Di Bernardo, Enrico
• Primary Examiner(s): Swarthout, Brent
• Application Number:
  US13/964,979
• Publication Number:
  US 20130331987A1
• Time in Patent Office: 393 Days
• Field of Search: 340/995.1, 340/995.24, 340/995.25, 340/988, 318/568.11, 318/568.12, 318/568.16, 700245-253, 701/23, 701/28, 701/301, 701/200, 701/207, 701/217, 901/1, 901/50
• US Class Current: 340/995.1
• CPC Class Codes: G01C 21/12 executed aboard the object being navigated; Dead reckoning
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0234 using optical markers or beacons optical beacons per se G01S1/70
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/46 Sensing device
• CPC Class Codes: Y10S 901/47 Optical
• CPC Class Codes: Y10S 901/50 Miscellaneous

Abstract

The invention is related to methods and apparatus that use a visual sensor and dead reckoning sensors to process Simultaneous Localization and Mapping (SLAM). These techniques can be used in robot navigation. Advantageously, such visual techniques can be used to autonomously generate and update a map. Unlike with laser rangefinders, the visual techniques are economically practical in a wide range of applications and can be used in relatively dynamic environments, such as environments in which people move. One embodiment further advantageously uses multiple particles to maintain multiple hypotheses with respect to localization and mapping. Further advantageously, one embodiment maintains the particles in a relatively computationally-efficient manner, thereby permitting the SLAM processes to be performed in software using relatively inexpensive microprocessor-based computer systems.

Claims

1. An apparatus for autonomous localization and mapping, the apparatus comprising:

a visual sensor configured to visually observe an environment;

a processor configured to execute computer instructions;

a tangible, non-transitory computer readable medium configured to store a computer program, wherein the computer program comprises;

program instructions configured to maintain a map of landmarks in a data store, where the map of landmarks is based at least in part on visual observations of the environment;

program instructions configured to receive data from a dead reckoning sensor, where data from the dead reckoning sensor relates to movement of the visual sensor within the environment;

program instructions configured to use data from the dead reckoning sensor and a prior pose estimate to predict a device pose in a global reference frame at least partly in response to a determination that a known landmark has not been encountered; and

program instructions configured to use data from the visual sensor to predict a new device pose in the global reference frame at least partly in response to a determination that a known landmark has been recognized, where the new device pose estimate is based at least in part on a previous pose estimate associated with the known landmark and using the visual sensor data to update the map;

wherein the data from the visual sensor comprises 2D image coordinates of features associated with the recognized landmark.

2. The apparatus as defined in claim 1, wherein the apparatus comprises a mobile robot.

3. The apparatus as defined in claim 1, wherein the map comprises one or more maps.

4. The apparatus as defined in claim 1, wherein the visual sensor corresponds to one or more cameras.

5. The apparatus as defined in claim 1, wherein the dead reckoning sensor corresponds to at least one of an odometer or a pedometer.