System and method for traffic data collection using unmanned aerial vehicles (UAVs)

• US 9,952,594 B1
• Filed: 04/07/2017
• Issued: 04/24/2018
• Est. Priority Date: 04/07/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: Tusimple
• Inventors: Zhao, Yufei, Hou, Xiaodi
• Primary Examiner(s): EL CHANTI, HUSSEIN A
• Application Number:
  US15/482,038
• Time in Patent Office: 382 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/012 from other sources than vehicle or roadside beacons, e.g. mobile networks
• CPC Class Codes: G08G 1/0129 for creating historical data or processing based on historical data

Abstract

A system and method for traffic data collection using unmanned aerial vehicles (UAVs) are disclosed. A particular embodiment is configured to: deploy an unmanned aerial vehicle (UAV), equipped with a camera, to an elevated position at a monitored location or to track a specific target vehicle; capture video data of the monitored location or the target vehicle for a pre-determined period of time using the UAV camera; transfer the captured video data to a processing system; at the processing system, process the captured video data on a frame basis to identify vehicles or objects of interest for analysis; group the video data from multiple frames related to a particular vehicle or object of interest into a data group associated with the particular vehicle or object; create a data group for each of the vehicles or objects of interest; and provide the data groups corresponding to each of the vehicles or objects of interest as output data used to configure or train a human driver model for prediction or simulation of human driver behavior.

Claims

1. A system comprising:

an unmanned aerial vehicle (UAV), equipped with a camera, positioned at an elevated position at a monitored location or to track a specific target vehicle, the UAV configured to capture video data of the monitored location or the target vehicle for a pre-determined period of time using the UAV camera;

a data processor; and

a human driver model module, executable by the data processor, the human driver model module being configured to;

receive the captured video data from the UAV;

process the captured video data on a frame basis to identify vehicles or objects of interest for analysis;

group the video data from multiple frames related to a particular vehicle or object of interest into a data group associated with the particular vehicle or object;

create a data group for each of the vehicles or objects of interest; and

provide the data groups corresponding to each of the vehicles or objects of interest as output data used to configure or train a human driver model for prediction or simulation of human driver behavior.

2. The system of claim 1 wherein the human driver model includes machine learnable components.

3. The system of claim 1 wherein the UAV is configured to hover or cruise with the target vehicle.

4. The system of claim 1 wherein the human driver model is configured to retain information from the output data corresponding to human driver behaviors as mathematical or data representations.

5. The system of claim 1 wherein the human driver model is configured to predict or simulate human driver behaviors.

6. The system of claim 1 wherein the human driver model is indirectly used to configure a control system for an autonomous vehicle.

7. The system of claim 1 wherein the captured video data is received by the human driver model module in a wireless data transmission from the UAV.

8. The system of claim 1 wherein the captured video data is received by the human driver model module as a data transfer from a data storage device of the UAV.

9. A method comprising:

deploy an unmanned aerial vehicle (UAV), equipped with a camera, to an elevated position at a monitored location or to track a specific target vehicle;

capture video data of the monitored location or the target vehicle for a pre-determined period of time using the UAV camera;

transfer the captured video data to a processing system;

at the processing system, process the captured video data on a frame basis to identify vehicles or objects of interest for analysis;

group the video data from multiple frames related to a particular vehicle or object of interest into a data group associated with the particular vehicle or object;

create a data group for each of the vehicles or objects of interest; and

provide the data groups corresponding to each of the vehicles or objects of interest as output data used to configure or train a human driver model for prediction or simulation of human driver behavior.

10. The method of claim 9 wherein the processing system includes machine learnable components.

11. The method of claim 9 wherein the UAV is configured to hover or cruise with the target vehicle.

12. The method of claim 9 wherein the human driver model is configured to retain information from the output data corresponding to human driver behaviors as mathematical or data representations.

13. The method of claim 9 wherein the human driver model is configured to predict or simulate human driver behaviors.

14. The method of claim 9 wherein the human driver model is used to configure a control system for an autonomous vehicle.

15. The method of claim 9 wherein the captured video data is received by the processing system in a wireless data transmission from the UAV.

16. The method of claim 9 wherein the captured video data is received by the processing system as a data transfer from a data storage device of the UAV.

17. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive captured video data from an unmanned aerial vehicle (UAV), equipped with a camera, positioned at an elevated position at a monitored location or to track a specific target vehicle, the UAV configured to capture video data of the monitored location or the target vehicle for a pre-determined period of time using the UAV camera;

process the captured video data on a frame basis to identify vehicles or objects of interest for analysis;

group the video data from multiple frames related to a particular vehicle or object of interest into a data group associated with the particular vehicle or object;

create a data group for each of the vehicles or objects of interest; and

provide the data groups corresponding to each of the vehicles or objects of interest as output data used to configure or train a human driver model for prediction or simulation of human driver behavior.

18. The non-transitory machine-useable storage medium of claim 17 wherein the instructions are further configured to enable machine learning.

19. The non-transitory machine-useable storage medium of claim 17 wherein the instructions are further configured to retain information from the output data corresponding to human driver behaviors as mathematical or data representations.

20. The non-transitory machine-useable storage medium of claim 17 wherein the instructions are further configured to predict or simulate human driver behaviors.