Method of locating a sensor and related apparatus

• US 9,576,206 B2
• Filed: 02/08/2013
• Issued: 02/21/2017
• Est. Priority Date: 02/10/2012
• Status: Active Grant

• Current Assignee: Oxford University Innovation Limited (University of Oxford)
• Original Assignee: Oxford University Innovation Limited (University of Oxford)
• Inventors: Churchill, Winston, Newman, Paul Michael
• Primary Examiner(s): Mistry, Oneal R
• Application Number:
  US14/375,407
• Publication Number:
  US 20140376777A1
• Time in Patent Office: 1,474 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G01C 21/3837 Data obtained from a single source
• CPC Class Codes: G01C 21/3867 Geometry of map features, e.g. shape points, polygons or for simplified maps
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A method of localizing a vehicle hosting a sensor comprises capturing data from the sensor providing a sensed scene around the vehicle at a current time, processing the sensed scene to extract a set of features from the sensed scene and determine a position of the vehicle from the sensed scene in relation to a previous position of the sensor, comparing the extracted set of features to one or more stored experiences, wherein each stored experience comprises a plurality of sets of features, where each set of features has been determined from a previously sensed scene, to ascertain whether the sensed scene can be recognized within any of the stored experiences, and if the extracted set of features of the sensed scene are recognized then that stored experience is used to produce an estimate of the position of the vehicle at the current time with respect to the stored experience.

Claims

1. A method of localizing a vehicle hosting a vehicle sensor comprising the steps of:

a) capturing data from the vehicle sensor providing a single representation of an environment around the vehicle at a current time;

b) processing the single representation to extract a set of features from the single representation and to determine, using the extracted features, a position of the vehicle from the single representation in relation to an earlier time, the determination made by;

comparing the extracted set of features from the single representation to a plurality of collections of single representations representing a same physical space of the environment, wherein each collection includes a plurality of single representations and each single representation within the collections provides a set of features, determined from a previously sensed scene, against which the extracted set of features is compared to ascertain whether the single representation can be recognised within any of the plurality of stored collections of single representations; and

recognizing a node within the single representation of the environment around the vehicle at the current time; and

c) if the extracted set of features of the single representation are recognised within any one of the stored collections of single representations then that stored collection of single representations is used to produce an estimate of the position of the vehicle via linking the node at the current time with respect to a node in the stored collection of single representations.

2. A method according to claim 1 in which the method is arranged such that if the single representation of the environment is not recognised within more than a predetermined number of the stored collections of single representations then the method stores the single representation of the environment such that the single representation of the environment becomes a part of a new collection of single representations for future comparisons.

3. A method according to claim 2 in which the method continues to try and recognise the single representation of the environment within the stored collections of single representations as the method stores the single representation of the environment sensed.

4. A method according to claim 1 in which the stored collections of single representations are used by the vehicle/sensor that generated them.

5. A method according to claim 1 in which the stored collections of single representations are used by vehicles and/or sensors in addition to the vehicle/sensor that generated them.

6. A method according to claim 1 which analyses the extracted set of features and links features within the single representation at the current time, with features identified to be the same feature in a subsequent single representation of the environment at a second, later, time using a transformation to specify movement between the current time and the second time.

7. A method according to claim 6 which is arranged to generate a trajectory of the vehicle using the transformation that links features identified to be the same within single representation of the environment.

8. A method according to claim 1 which uses coarse positioning of the vehicle to determine roughly a locality of the vehicle.

9. A method according to claim 8 which uses the locality of the vehicle to determine which stored collections of single representations should be considered to determine whether a single representation of the environment can be recognised therewithin.

10. A method according to claim 1 which uses an image matching technique to determine stored collections of single representations which appear similar to the single representation of the environment and subsequently use single representation of the environment which appear similar to determine whether a single representation of the environment can be recognised therewith.

11. A method according to claim 1 in which the single representation of the environment is recognised within a predetermined number of stored collections of single representations before it is determined that the vehicle has been localised.

12. A method according to claim 1 which determines whether stored collections of single representations relate to a similar environments as one another and, should it be determined that stored collections of single representations relate to substantially the same environment, link such stored experiences.

13. A method according to claim 1 in which each stored collections of single representations exists in its own frame of reference.

14. A method according to claim 1 wherein each stored collections of single representations represents at least a portion of an environment through which the vehicle moves, and wherein each stored collections of single representations relates to

a different time and/or date of capture.

15. A method according to claim 1 in which at least some of the collections of single representations are captured by the vehicle sensor.

16. A method according to claim 1 in which at least some of the collections of single representations are captured by a sensor other than the vehicle sensor.

17. A method according to claim 1 in which each collection of single representations is independent from the others.

18. A method according to claim 1 in which the plurality of collections of single representations are downloaded to the vehicle.

19. A method according to claim 1, wherein the comparing further comprises:

comparing the extracted set of features from the single representation to the plurality of collections of single representations in parallel at a same time.

20. A method according to claim 1, wherein the comparing further comprises:

determining whether the node within the single representation of the environment around the vehicle at the current time is recognized within any one of the single representations within the stored collections of single representations.

21. A system arranged to localise a vehicle, the system comprising a vehicle sensor arranged to generate data representing a single representation of an environment around the vehicle at a current time, processing circuitry arranged to process the data wherein the processing is arranged to:

extract a set of features from the single representation to determine a position of the vehicle from the data relative to an earlier time, the determination made by comparing the extracted set of features to a plurality of stored collections of single representations representing a same physical space of the environment held within a storage device, wherein each collection includes a plurality of single representations and each single representation within the collections provides a set of features, to ascertain whether the single representation can be recognised within any of the stored collections of single representations;

recognize a node within the single representation of the environment around the vehicle at the current time; and

should the data allow the single representation to be recognised within any stored collections of single representations then use that stored collection of single representations to estimate the position of the vehicle via linking the node at the current time with a node in the stored collection of single representations.

22. A system according to claim 21 in which at least a portion of the storage device is mounted off the vehicle and arranged to accessible from the vehicle.

23. A system according to claim 21 which comprises a server arranged to communicate with the processing circuitry on the vehicle.

24. A system according to claim 21 in which the processing circuitry is arranged to compare the data against a sub-set of the available stored collections of single representations.

25. A system according to claim 24 in which the processing circuitry is arranged to obtain, from a remote storage device, the sub-set of available stored collections of single representations.

26. A vehicle having a vehicle sensor mounted thereon, wherein the vehicle sensor is arranged to generate data representing a single representation of an environment around the vehicle at the current time, and a processing circuitry arranged to process the data wherein the processing circuitry is arranged to:

extract a set of features from the single representation to determine a position of the vehicle from the data relative to an earlier time, the determination made by comparing the extracted set of features to a plurality of stored collections of single representations representing a same physical space of the environment held within a storage device, wherein each collection includes a plurality of single representations and each single representation within the collections provides a set of features, to ascertain whether the single representation can be recognised within any of the stored collections of single representations;

recognize a node within the single representation of the environment around the vehicle at the current time; and

should the data allow the single representation to be recognised within any stored collections of single representations then use that stored collection of single representations to estimate the position of the vehicle via linking the node at the current time with a node in the stored collection of single representations.