Localising portable apparatus

• US 10,436,878 B2
• Filed: 12/04/2015
• Issued: 10/08/2019
• Est. Priority Date: 12/15/2014
• Status: Active Grant

• Current Assignee: Chancellor Masters & Scholars of The University of Cambridge (University of Cambridge)
• Original Assignee: Chancellor Masters & Scholars of The University of Cambridge (University of Cambridge)
• Inventors: Linegar, Christopher James, Churchill, Winston Samuel, Newman, Paul Michael
• Primary Examiner(s): Weber, Tamara L
• Application Number:
  US15/536,216
• Publication Number:
  US 20170343643A1
• Time in Patent Office: 1,404 Days
• Field of Search: None
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method of localizing portable apparatus (200) obtains: a stored experience data set comprising a set of connected nodes; captured location representation data provided by at least one sensor associated with the portable apparatus, and a current pose estimate of the portable apparatus within the environment. The pose estimate is used to select a candidate set of the nodes that contain a potential match for the captured location representation data. The pose estimate is used to obtain a set of paths from path memory data, each said path comprising a set of said nodes previously traversed in the environment under similar environmental/visual conditions. The set of paths is used to refine the candidate set. The captured location representation data and the refined candidate set of nodes is compared in order to identify a current pose of the portable apparatus within the environment.

Claims

1. A method of localizing a portable apparatus in an environment, the method comprising:

obtaining a stored experience data set comprising a set of connected nodes, each said node comprising a representation of a location within the environment;

obtaining captured location representation data provided by at least one sensor associated with the portable apparatus;

obtaining a current pose estimate of the portable apparatus within the environment;

using the current pose estimate to select a candidate set of the nodes from the stored experience data set that contain a potential match for the captured location representation data;

using the current pose estimate to obtain a set of paths from path memory data, each said path comprising a set of said nodes previously traversed in the environment under similar environmental/visual conditions;

using the set of paths to refine the candidate set of nodes;

comparing the captured location representation data and the refined candidate set of nodes in order to identify a current pose of the portable apparatus within the environment; and

navigating the portable apparatus using at least the current pose of the portable apparatus.

2. The method according to claim 1, wherein using the set of paths to refine the candidate set of nodes comprises:

assessing if the current pose estimate corresponds to a said node in a said path in the set of paths; and

if a result of the assessing is positive then selecting the corresponding node in that path for inclusion in the refined set of candidate nodes.

3. The method according to claim 1, wherein using the set of paths to refine the candidate set of nodes comprises:

determining if the nodes traversed by the portable apparatus prior to the node corresponding to the current pose estimate correspond to said nodes in a said path in the set of paths; and

if a result of the determination is positive then selecting at least one neighboring said node in the path for inclusion in the refined candidate set.

4. The method according to claim 1, wherein using the current pose estimate to select the candidate set of nodes comprises selecting an initial set of candidate nodes from the stored experience data set that are neighbors of the current pose estimate.

5. The method according to claim 4, wherein the selecting of the initial set of nodes comprises ranking the neighboring nodes according to distance from the current pose estimate.

6. The method according to claim 1, wherein using the set of paths to refine the candidate set of nodes comprises:

counting a number of said paths in the set of paths that connect a node corresponding to the current pose estimate with each said node in the candidate set; and

selecting the nodes for retention in the refined candidate set based on a result of the counting.

7. The method according to claim 6, wherein the nodes that neighbor the node corresponding to the current pose estimate in the path(s) having highest a said count are retained in the refined candidate set.

8. The method according to claim 1, wherein the method further comprises:

obtaining previous localization data which indicates whether or not at least some of the nodes in the candidate set were processed in a previous localization attempt; and

using the previous localization data to produce the refined set of candidate nodes.

9. The method according to claim 8, wherein using the previous localization data to produce the refined set of candidate nodes comprises:

counting a number of nodes common between a said path in the path memory and the previous localization data; and

selecting the node(s) corresponding to a highest said count(s) for inclusion in the refined set of candidate nodes.

10. The method according to claim 9, wherein a probability distribution relating to the previous localization data is used to produce the refined set of candidate nodes.

11. The method according to claim 1, wherein the captured location representation data comprises at least one visual image.

12. A device adapted to localize a portable apparatus in an environment, the device comprising:

one or more components configured to

obtain a stored experience data set comprising a set of connected nodes, each said node comprising a representation of a location within the environment;

obtain captured location representation data provided by at least one sensor associated with the portable apparatus;

obtain a current pose estimate of the portable apparatus within the environment;

use the current pose estimate to select a candidate set of the nodes from the stored experience data set that contain a potential match for the captured location representation data;

use the current pose estimate to obtain a set of paths from path memory data, each said path comprising a set of said nodes previously traversed in the environment under similar environmental/visual conditions;

use the set of paths to refine the candidate set of nodes;

compare the captured location representation data and the refined candidate set of nodes in order to identify a current pose of the portable apparatus within the environment; and

navigate the portable apparatus using at least the current pose of the portable apparatus.

13. The device according to claim 12, wherein the one or more components configured to compare the captured location representation data comprises a plurality of processors configured to operate in parallel, each said processor configured to compare the captured location representation data with a particular one of the nodes in the refined candidate.

14. A vehicle including the device according to claim 12.

15. One or more non-transitory computer readable mediums storing a computer program that when executed by one or more processors cause a process to the carried out for localizing a portable apparatus in an environment, the process comprising:

obtaining a stored experience data set comprising a set of connected nodes, each said node comprising a representation of a location within the environment;

obtaining captured location representation data provided by at least one sensor associated with the portable apparatus;

obtaining a current pose estimate of the portable apparatus within the environment;

using the current pose estimate to select a candidate set of the nodes from the stored experience data set that contain a potential match for the captured location representation data;

using the current pose estimate to obtain a set of paths from path memory data, each said path comprising a set of said nodes previously traversed in the environment under similar environmental/visual conditions;

using the set of paths to refine the candidate set of nodes;

comparing the captured location representation data and the refined candidate set of nodes in order to identify a current pose of the portable apparatus within the environment; and

navigating the portable apparatus using at least the current pose of the portable apparatus.

16. The one or more non-transitory computer readable mediums according to claim 15, wherein using the set of paths to refine the candidate set of nodes comprises:

assessing if the current pose estimate corresponds to a said node in a said path in the set of paths; and

if a result of the assessing is positive then selecting the corresponding node in that path for inclusion in the refined set of candidate nodes.

17. The one or more non-transitory computer readable mediums according to claim 15, wherein using the set of paths to refine the candidate set of nodes comprises:

determining if the nodes traversed by the portable apparatus prior to the node corresponding to the current pose estimate correspond to said nodes in a said path in the set of paths; and

if a result of the determination is positive then selecting at least one neighboring said node in the path for inclusion in the refined candidate set.

18. The one or more non-transitory computer readable mediums according to claim 15, wherein using the current pose estimate to select the candidate set of nodes comprises selecting an initial set of candidate nodes from the stored experience data set that are neighbors of the current pose estimate.

19. The one or more non-transitory computer readable mediums according to claim 15, wherein using the set of paths to refine the candidate set of nodes comprises:

counting a number of said paths in the set of paths that connect a node corresponding to the current pose estimate with each said node in the candidate set; and

selecting the nodes for retention in the refined candidate set based on a result of the counting.

20. The one or more non-transitory computer readable mediums according to claim 15, the process further comprising:

obtaining previous localization data which indicates whether or not at least some of the nodes in the candidate set were processed in a previous localization attempt; and

using the previous localization data to produce the refined set of candidate nodes.