Off-Target Tracking Using Feature Aiding in the Context of Inertial Navigation

• US 20150092048A1
• Filed: 09/25/2014
• Published: 04/02/2015
• Est. Priority Date: 09/27/2013
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Brunner, Christopher, Ramanandan, Arvind, Tyagi, Abhishek, Chari, Murali Ramaswamy, Ramachandran, Mahesh
• Application Number:
  US14/497,117
• Publication Number:
  US 20150092048A1
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G01C 21/1654 with electromagnetic compass
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 5/0264 at least one of the systems being a non-radio wave positioning system
• CPC Class Codes: G01S 5/0294 Trajectory determination or predictive filtering, e.g. target tracking or Kalman filtering
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings

Abstract

A Visual Inertial Tracker (VIT), such as a Simultaneous Localization And Mapping (SLAM) system based on an Extended Kalman Filter (EKF) framework (EKF-SLAM) can provide drift correction in calculations of a pose (translation and orientation) of a mobile device by obtaining location information regarding a target, obtaining an image of the target, estimating, from the image of the target, measurements relating to a pose of the mobile device based on the image and location information, and correcting a pose determination of the mobile device using an EKF, based, at least in part, on the measurements relating to the pose of the mobile device.

Claims

1. A method of correcting drift in a tracking system of a mobile device, the method comprising:

obtaining location information regarding a target;

obtaining an image of the target, the image captured by the mobile device;

estimating, from the image of the target, measurements relating to a pose of the mobile device based on the image and location information, wherein the pose comprises information indicative of a translation and orientation of the mobile device; and

correcting a pose determination of the mobile device using an Extended Kalman Filter (EKF), based, at least in part, on the measurements relating to the pose of the mobile device.

2. The method of claim 1, further comprising obtaining absolute coordinates of the target, wherein correcting the pose is further based, at least in part, on the absolute coordinates of the target.

3. The method of claim 1, further comprising processing the image of the target to determine that the target was captured in the image, wherein the processing includes comparing one or more keypoints of the image with one or more keypoints of each target in a plurality of known targets.

4. The method of claim 3, further comprising:

receiving one or more wireless signals from one or more access points;

determining a proximity of the one or more access points based on wireless signals; and

determining the plurality of known targets, based on the determined proximity of the one or more access points.

5. The method of claim 1, wherein the tracking system incorporates a Simultaneous Localization And Mapping (SLAM) system with the EKF.

6. The method of claim 1, wherein the pose determination is based, at least in part, on measurements from one or more of an accelerometer or a gyroscope of the mobile device.

7. The method of claim 6, further comprising determining a bias of one or more of the accelerometer or the gyroscope of the mobile device, based at least in part on the pose determination and the corrected pose.

8. A mobile device comprising:

a camera;

a memory; and

a processing unit operatively coupled with the camera and the memory and configured to;

obtain location information regarding a target;

obtain, an image of the target, the image captured by the camera of the mobile device;

estimate, from the image of the target, measurements relating to a pose of the mobile device based on the image and location information, wherein the pose comprises information indicative of a translation and orientation of the mobile device; and

correct a pose determination of the mobile device using an Extended Kalman Filter (EKF), based, at least in part, on the measurements relating to the pose of the mobile device.

9. The mobile device of claim 8, wherein the processing unit is configured to obtain absolute coordinates of the target, wherein the processing unit is further configured to correct the pose is based, at least in part, on the absolute coordinates of the target.

10. The mobile device of claim 8, wherein the processing unit is configured to process the image of the target to determine that the target was captured in the image, and wherein processing the image includes comparing one or more features of the image with one or more features of each target in a plurality of known targets.

11. The mobile device of claim 10, further comprising a wireless communication interface configured to receive one or more wireless signals from one or more access points, wherein the processing unit is further configured to:

determine a proximity of the one or more access points based on wireless signals; and

determine the plurality of known targets, based on the determined proximity of the one or more access points.

12. The mobile device of claim 8, wherein the processing unit is configured to incorporate a Simultaneous Localization And Mapping (SLAM) system with the EKF.

13. The mobile device of claim 8, further comprising one or more motion sensors, wherein the processing unit is further configured to determine the pose determination based, at least in part, on one or more measurements received from the one or more motion sensors.

14. The mobile device of claim 13, wherein the one or more motion sensors include one or more of an accelerometer or a gyroscope.

15. The mobile device of claim 13, wherein the processing unit is configured to determine a bias of the one or more motion sensors, based at least in part on the pose determination and the corrected pose.

16. An apparatus comprising:

means for obtaining location information regarding a target;

means for obtaining an image of the target, the image captured by a mobile device;

means for estimating, from the image of the target, measurements relating to a pose of the mobile device based on the image and location information, wherein the pose comprises information indicative of a translation and orientation of the mobile device; and

means for correcting a pose determination of the mobile device using an Extended Kalman Filter (EKF), based, at least in part, on the measurements relating to the pose of the mobile device.

17. The apparatus of claim 16, further comprising means for obtaining absolute coordinates of the target, wherein the means for correcting the pose is configured to base the corrected pose, at least in part, on the absolute coordinates of the target.

18. The apparatus of claim 16, further comprising means for processing the image of the target to determine that the target was captured in the image, wherein the means for processing the image include means for comparing one or more features of the image with one or more features of each target in a plurality of known targets.

19. The apparatus of claim 18, further comprising:

means for receiving one or more wireless signals from one or more access points;

means for determining a proximity of the one or more access points based on wireless signals; and

means for determining the plurality of known targets, based on the determined proximity of the one or more access points.

20. The apparatus of claim 16, further comprising means for incorporating a Simultaneous Localization And Mapping (SLAM) system with the EKF.

21. The apparatus of claim 16, further comprising means for basing the pose determination, at least in part, on measurements of one or more of an accelerometer or a gyroscope of the mobile device.

22. The apparatus of claim 21, further comprising means for determining a bias of one or more of the accelerometer or the gyroscope of the mobile device, based at least in part on the pose determination and the corrected pose.

23. A non-transitory machine-readable medium having instructions embedded thereon for correcting drift in a tracking system of a mobile device, the instructions including computer code for:

obtaining location information regarding a target;

obtaining an image of the target, the image captured by the mobile device;

estimating, from the image of the target, measurements relating to a pose of the mobile device based on the image and location information, wherein the pose comprises information indicative of a translation and orientation of the mobile device; and

correcting a pose determination of the mobile device using an Extended Kalman Filter (EKF), based, at least in part, on the measurements relating to the pose of the mobile device.

24. The non-transitory machine-readable medium of claim 23, the instructions further including computer code for obtaining absolute coordinates of the target, wherein the computer code is further configured to base correcting the pose, at least in part, on the absolute coordinates of the target.

25. The non-transitory machine-readable medium of claim 23, the instructions further including computer code for processing the image of the target to determine that the target was captured in the image, wherein the computer code for processing includes computer code for comparing one or more features of the image with one or more features of each target in a plurality of known targets.

26. The non-transitory machine-readable medium of claim 25, the instructions further including computer code for:

receiving one or more wireless signals from one or more access points;

determining a proximity of the one or more access points based on wireless signals; and

determining the plurality of known targets, based on the determined proximity of the one or more access points.

27. The non-transitory machine-readable medium of claim 23, the instructions further including computer code for incorporating a Simultaneous Localization And Mapping (SLAM) system with the EKF.

28. The non-transitory machine-readable medium of claim 23, wherein the computer code can be configured to base the pose determination, at least in part, on measurements from one or more of an accelerometer or a gyroscope of the mobile device.

29. The non-transitory machine-readable medium of claim 28, the instructions further including computer code for determining a bias of one or more of the accelerometer or the gyroscope of the mobile device, based at least in part on the pose determination and the corrected pose.