Autonomous Human-Centric Place Recognition

• US 20170249504A1
• Filed: 02/29/2016
• Published: 08/31/2017
• Est. Priority Date: 02/29/2016
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Martinson, Eric, Kim, David, Nakano, Yusuke
• Granted Patent: US 10,049,267 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

The novel technology described in this disclosure includes an example method comprising capturing sensor data using one or more sensors describing a particular environment; processing the sensor data using one or more computing devices coupled to the one or more sensors to detect a participant within the environment; determining a location of the participant within the environment; querying a feature database populated with a multiplicity of features extracted from the environment using the location of the participant for one or more features being located proximate the location of the participant; and selecting, using the one or more computing devices, a scene type from among a plurality of predetermined scene types based on association likelihood values describing probabilities of each feature of the one or more features being located within the scene types.

Claims

1. A computer-implemented method comprising:

capturing sensor data using one or more sensors describing a particular environment;

processing the sensor data using one or more computing devices coupled to the one or more sensors to detect a participant within the environment;

determining a location of the participant within the environment;

querying a feature database populated with a multiplicity of features extracted from the environment using the location of the participant for one or more features being located proximate the location of the participant; and

selecting, using the one or more computing devices, a scene type from among a plurality of predetermined scene types based on association likelihood values describing probabilities of each feature of the one or more features being located within the scene types.

2. The computer-implemented method of claim 1, further comprising:

executing one or more autonomous routines based on the selected scene type.

3. The computer-implemented method of claim 1, further comprising:

generating the association likelihood values based on times of day.

4. The computer-implemented method of claim 1, wherein the association likelihood values are classification scores respectively describing the probabilities of the features belonging to the scene types, prior probabilities of classifying the scene types correctly using the features, N-dimensional locations with respect to an arbitrary reference point, and sizes.

5. The computer-implemented method of claim 3, wherein the association likelihood values are probabilities computed using

6. The computer-implemented method of claim 1, further comprising:

prior to querying the feature database, scanning the environment using the one or more sensors;

extracting the multiplicity of features of the environment using sensor data provided by the one or more sensors responsive to scanning the environment; and

populating the feature database with the multiplicity of features.

7. The computer-implemented method of claim 1, wherein selecting the scene type includes generating a gradient for an area in a vicinity of the participant, determining a directionality based on the gradient, and selecting the scene type further based on the directionality of the gradient.

8. The computer-implemented method of claim 1, wherein at least one of the one or more sensors is an RGB-D camera.

9. The computer-implemented method of claim 1, wherein the participant is a human.

10. An autonomous computing system comprising:

one or more processors;

one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising;

capturing sensor data using one or more sensors describing a particular environment,

processing the sensor data using one or more computing devices coupled to the one or more sensors to detect a participant within the environment,

determining a location of the participant within the environment,

querying a feature database populated with a multiplicity of features extracted from the environment using the location of the participant for one or more features being located proximate the location of the participant, and

selecting, using the one or more computing devices, a scene type from among a plurality of predetermined scene types based on association likelihood values describing probabilities of each feature of the one or more features being located within the scene types.

11. The system of claim 10, wherein the instructions, when executed by the one or more processors, further cause the system to perform operations comprising:

executing one or more autonomous routines based on the selected scene type.

12. The system of claim 10, wherein the instructions, when executed by the one or more processors, further cause the system to perform operations comprising:

generating the association likelihood values.

13. The system of claim 10, wherein the association likelihood values are classification scores respectively describing the probabilities of the features belonging to the scene types, prior probabilities of classifying the scene types correctly using the features, N-dimensional locations with respect to an arbitrary reference point, and sizes.

14. The system of claim 10, wherein the association likelihood values are probabilities computed using

15. The system of claim 10, wherein the instructions, when executed by the one or more processors, further cause the system to perform operations comprising:

prior to querying the feature database, scanning the environment using the one or more sensors,

extracting the multiplicity of features of the environment using sensor data provided by the one or more sensors responsive to scanning the environment, and

populating the feature database with the multiplicity of features.

16. The system of claim 10, wherein selecting the scene type includes generating a gradient for an area in a vicinity of the participant, determining a directionality based on the gradient, and selecting the scene type further based on the directionality of the gradient.

17. The system of claim 10, wherein at least one of the one or more sensors is an RGB-D camera.

18. The system of claim 10, wherein the participant is a human.