GYROSCOPE ASSISTED SCALABLE VISUAL SIMULTANEOUS LOCALIZATION AND MAPPING

• US 20160209217A1
• Filed: 11/13/2015
• Published: 07/21/2016
• Est. Priority Date: 11/13/2014
• Status: Active Grant

• Current Assignee: Worcester Polytechnic Institute
• Original Assignee: Worcester Polytechnic Institute
• Inventors: Babu, Benzun Wisely, Duckworth, R. James, Cyganski, David
• Granted Patent: US 9,846,042 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/18 Stabilised platforms, e.g. by gyroscope
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/85 Stereo camera calibration

Abstract

An indoor localization system uses Visual Simultaneous Localization and Mapping (VSLAM) aided by gyroscope sensor information. Indoor environments pose several challenges which could cause a vision only system to fail due to tracking errors. Investigation revealed significant feature loss in a vision only system when traversing plain walls, windows and staircases. However, the addition of a gyroscope helps in handling such difficult conditions by providing additional rotational information. A portable system consisting of an Inertial Measurement Unit (IMU) and a stereo camera has been developed for indoor mapping. The images and gyroscope rates acquired by the system are stored and post-processed using Gyroscope Assisted Scalable Visual Simultaneous Localization and Mapping Algorithm.

Claims

1. A method of navigating an indoor environment

gathering visual information from sensors on a mobile device in the indoor environment;

retrieving angular information from an inertial measurement unit during movement of the mobile device; and

computing location information using the gathered visual information and the retrieved angular information.

3. The method of claim 1 wherein computing the location information is devoid of input from a GPS (Global Positioning System) device.

4. The method of claim 1 wherein the visual information is gathered from a Simultaneous Localization and Mapping (SLAM) system.

2. The method of claim wherein the inertial measurement unit (IMU) is a gyroscope adapted for retrieving inertial and angular momentum data.

5. A method of computing a localization path through an indoor environment, comprising:

retrieving visual information indicative of a location in a camera coordinate frame;

computing angular information indicative of a location in an inertial coordinate frame;

coalescing the visual information and the angular information to compute a location;

mapping the computed location to a world frame based on true locations in the indoor environment; and

rendering, based on the mapped location, a position in the indoor environment.

6. The method of claim 5 wherein the visual information includes a plurality of successive frames received from a camera, and

the angular information is indicative of a change in pose of the camera between the frames.

7. The method of claim 6 further comprising:

performing feature detection on the retrieved visual information to determine feature positions;

identifying features common to one or more of the plurality of frames; and

computing a location based on the angular information when the visual information fails to distinguish sufficient features to compute a location.

8. The method of claim 5 further comprising

receiving the visual information and angular information periodically based on an advancement of the camera coordinate frame and the inertial coordinate frame through the indoor environment;

computing the location based on relative movement of the camera coordinate frame and the inertial coordinate frame between the periodic receipt of the visual information and the angular information.

9. The method of claim 8 further comprising disposing a visual sensory device and an angular sensory device on a vehicle, and disposing the vehicle through the indoor environment for receiving the visual information and the angular information as the vehicle is disposed at different locations.

10. The method of claim 8 further comprising

initially calibrating the camera frame and the inertial frame at a common location, and

computing the angular information relative to the initial calibration positioning.

11. The method of claim 8 further comprising retrieving the visual information from a stereo camera arrangement including a right camera and a left camera.

12. The method of claim 9 wherein:

the visual sensory device is a camera and the visual information are camera frames; and

the angular sensory device is a gyroscope and the angular information is angular momentum.

13. A device for computing a localization path through an indoor environment, comprising:

visual sensors configured to retrieve visual information indicative of a location in a camera coordinate frame;

inertial sensors configured to compute angular information indicative of a location in an inertial coordinate frame;

a controller coupled to the visual sensors and the inertial sensors, the controller having memory and a processor for coalescing the visual information and the angular information to compute a location, the controller configured to map the computed location to a world frame based on true locations in the indoor environment; and

an output device for rendering, based on the mapped location, a position in the indoor environment.

14. The device of claim 13 wherein the visual information includes a plurality of successive frames received from a camera, and

the angular information is indicative of a change in pose of the camera between the frames, the controller including control logic for:

performing feature detection on the retrieved visual information to determine feature positions;

identifying features common to one or more of the plurality of frames; and

computing a location based on the angular information when the visual information fails to distinguish sufficient features to compute a location.

15. The device of claim 13 wherein the controller is further configured to

receive the visual information and angular information periodically based on an advancement of the camera coordinate frame and the inertial coordinate frame through the indoor environment;

and

compute the location based on relative movement of the camera coordinate frame and the inertial coordinate frame between the periodic receipt of the visual information and the angular information.

16. The device of claim 15 further comprising a visual sensory device and an angular sensory device disposed on a vehicle, and disposing the vehicle through the indoor environment for receiving the visual information and the angular information as the vehicle is disposed at different locations.

17. The device of claim 15 wherein the controller is responsive to an initially calibration of the camera frame and the inertial frame at a common location for computing the angular information relative to the initial calibration positioning.

18. The device of claim 16 wherein the visual sensory device further comprises a stereo camera arrangement including a right camera and a left camera.

19. The device of claim 16 wherein:

the visual sensory device is a camera and the visual information are camera frames; and

the angular sensory device is a gyroscope and the angular information is angular momentum, the controller further operable to

compute a location based on the angular information when there are no distinguishable features in a succession of the plurality of frames.

20. A computer program product on a non-transitory computer readable storage medium having instructions that, when executed by a processor, perform a method for computing a localization path through an indoor environment, the method comprising:

retrieving visual information indicative of a location in a camera coordinate frame;

computing angular information indicative of a location in an inertial coordinate frame;

coalescing the visual information and the angular information to compute a location;

mapping the computed location to a world frame based on true locations in the indoor environment; and

rendering, based on the mapped location, a position in the indoor environment.