Travel data collection and publication

• US 10,317,240 B1
• Filed: 03/30/2017
• Issued: 06/11/2019
• Est. Priority Date: 03/30/2017
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Di Pietro, Adriano, Sibley, Gabriel Thurston, Philbin, James William Vaisey
• Primary Examiner(s): Weber, Tamara L
• Application Number:
  US15/473,996
• Time in Patent Office: 803 Days
• Field of Search: None
• CPC Class Codes: G01C 21/3679 Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities G01C21/3611 takes precedence
• CPC Class Codes: G01C 21/3694 Output thereof on a road map
• CPC Class Codes: G01C 21/3811 Point data, e.g. Point of Interest [POI]
• CPC Class Codes: G01C 21/3841 Data obtained from two or more sources, e.g. probe vehicles
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Travel data may be gathered and published on a global map for use by various entities. The travel data may be captured by one or more vehicle computing devices, such as those associated with the operation and/or control of a vehicle. The vehicle computing devices may receive input from one or more sensors, and may identify and/or classify an event and/or areas of interest based on the input. The vehicle computing devices may then send travel data to the global map server for publication. Additionally or alternatively, the travel data may be captured by one or more user devices. The user devices may be configured to receive travel data, such as via a user interface, and to send the travel data to the global map server for publication. The global map server may consolidate and publish the travel data on a global map.

Claims

1. A computer implemented method comprising:

receiving data from one or more sensors, the data associated with at least one of an event or an area of interest;

determining, from a plurality of classifications and based at least in part on a semantic scene parsing, a classification for the at least one of the event or the area of interest, wherein the plurality of classifications comprises;

a first classification associated with an accident;

a second classification associated with a construction site; and

a third classification associated with a fire;

determining, based at least in part on the classification, that the data is relevant data;

incorporating, at a global map server, the relevant data and the classification into a global map;

receiving a request for derived data;

determining, based at least in part on the global map, the derived data; and

causing, based at least in part on the derived data, an autonomous vehicle to be deployed to a location associated with the at least one of the event or the area of interest in order to gather additional data associated with the at least one of the event or the area of interest.

2. The method of claim 1, wherein the method further comprises determining a location of the at least one of the event or the area of interest associated with the data, wherein determining the location comprises:

receiving a vehicle location from a location sensor of a vehicle;

receiving a distance and a bearing from the vehicle to the event or the area of interest from a ranging sensor of the one or more sensors; and

calculating the location of the at least one of the event or the area of interest based on the vehicle location, the distance, and the bearing from the vehicle to the at least one of the event or the area of interest,

and further wherein the classification is based, at least in part, on one or more of;

object recognition on an image of the data, or

speech recognition on an audio signal of the data.

3. The method of claim 1, wherein the receiving the data from the one or more sensors comprises:

receiving the data from one or more of;

a camera;

a LIDAR sensor;

a RADAR sensor;

an ultrasonic transducer;

a GPS receiver;

an accelerometer;

a magnetometer;

or

a gyroscope.

4. The method of claim 1, wherein:

determining the classification comprises;

inputting at least a portion of the data into one or more machine learning algorithms;

executing the one or more machine learning algorithms; and

receiving, as output from the one or more machine learning algorithms, the classification.

5. The method of claim 1, further comprising:

determining, based at least in part on recognizing an object in an image of the data, a confidence level of the classification of the data.

6. A method comprising:

receiving, at a global map server, travel data from a vehicle, wherein the travel data is associated with an event located in an area of a global map;

determining, from a plurality of classifications and based at least in part on a semantic scene parsing, a classification for the event;

incorporating, by the global map server, travel data and the classification into the global map;

determining derived information from the global map, the derived information being associated with the area;

publishing the derived information; and

causing, based at least in part on the derived information, an autonomous vehicle to be deployed to the area.

7. The method of claim 6, wherein:

the travel data further comprises at least one of;

a text description of the event,

an image of the event, or

an audio signal related to the event;

the determining the derived information comprises selecting at least a portion of the text description, the image, or the audio signal; and

the plurality of classifications comprises at least one of;

a first classification associated with an accident,

a second classification associated with a construction site, or

a third classification associated with a fire.

8. The method of claim 6, further comprising:

identifying a subscriber to a global map service; and

determining a subscription level of the subscriber,

wherein publishing the derived information is based at least in part on the subscription level.

9. The method of claim 6, further comprising assigning a confidence level to the travel data, wherein the incorporating the event, the location, and the classification into the global map is based at least in part on the confidence level being at or above a threshold confidence level.

10. The method of claim 6, wherein the vehicle comprises a first vehicle, and the method further comprises:

assigning a confidence level to the classification;

receiving, at the global map server, second travel data from a second vehicle, wherein the second travel data corresponds to the event;

associating the second travel data with the event; and

increasing the confidence level based at least in part on the second travel data.

11. The method of claim 6, wherein the travel data further comprises text describing the event, and the method further comprises:

performing text recognition on the text;

removing sensitive data from the text;

associating the text with the global map; and

publishing the text.

12. The method of claim 6, wherein the travel data further comprises an image of the event and the method further comprises removing sensitive data from the image prior to publication of the global map, wherein the sensitive data comprises at least one of:

a license plate number,

a license plate issuing authority,

a face of a person in the image, or

a human form.

13. The method of claim 6, wherein the vehicle comprises a first vehicle, and the method further comprises:

receiving, at the global map server and from a second vehicle, an image corresponding to the event; and

incorporating the image into the global map,

wherein determining the derived information comprises selecting the image.

14. The method of claim 6, wherein the travel data comprises first travel data received from a first vehicle, and the method further comprises:

receiving, at the global map server, second travel data from a second vehicle, wherein the second travel data corresponds to the event; and

associating the second travel data with the event,

wherein determining the derived information comprises selecting at least a portion of the second travel data.

15. A system comprising:

one or more processors; and

one or more non-transitory computer readable media storing computer-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising;

receiving, at a global map server, travel data from a vehicle, wherein the travel data comprises a location of an event;

determining, from a plurality of classifications and based at least in part on a semantic scene parsing, a classification for the event;

determining a confidence level of the travel data;

incorporating the travel data and the classification into a global map based at least in part on the location of the event and the confidence level;

determining derived information from the global map;

publishing the derived information; and

causing, based at least in part on the derived information, the vehicle to be deployed to an area proximate the location of the event to gather additional data associated with the event.

16. The system of claim 15, wherein the system further comprises a first vehicle, and the operations further comprise:

receiving, at the global map server, an audio signal corresponding to a description of the event from a second vehicle; and

incorporating the audio signal into the global map by associating the audio signal with the travel data,

wherein determining the derived information comprises selecting at least a portion of the audio signal.

17. The system of claim 15, wherein the travel data comprises first travel data received from a first vehicle, and the operations further comprise:

receiving, at the global map server, second travel data from a second vehicle, wherein the second travel data corresponds to the event; and

associating the second travel data with the event,

wherein determining the derived information comprises selecting at least a portion of the second travel data.

18. The system of claim 15, wherein the travel data comprises first travel data received from a first vehicle, and the operations further comprise:

receiving, at the global map server, second travel data from a second vehicle, wherein the second travel data corresponds to the event;

associating the second travel data with the event;

determining whether a first time associated with the first travel data is more recent than a second time associated with the second travel data,

wherein determining the derived information comprises selecting the second travel data and not selecting the first travel data based on the second time being more recent than the first time.

19. The system of claim 15, wherein the travel data comprises first travel data received from a first vehicle, and the operations further comprise:

receiving, at the global map server, second travel data from a second vehicle, wherein the second travel data corresponds to the event;

associating the second travel data with the event;

determining that the first travel data comprises a first type of data and the second travel data comprises a second type of data,

wherein determining the derived information comprises selecting at least a portion of one or more of the first travel data or the second travel data.

20. The system of claim 15, wherein the travel data comprises first travel data received from a first vehicle, and the operations further comprise:

receiving, at the global map server, second travel data from a second vehicle, wherein the second travel data corresponds to the event;

associating the second travel data with the event; and

altering the confidence level associated with the event, wherein altering the confidence level comprises one or more of;

lowering a confidence level if the first travel data and the second travel data disagree, or

raising the confidence level if the first travel data and the second travel data agree.