Coordinated control of self-driving vehicles under emergency situations

• US 10,466,691 B2
• Filed: 11/06/2017
• Issued: 11/05/2019
• Est. Priority Date: 11/06/2017
• Status: Active Grant

• Current Assignee: Pony.ai, Inc.
• Original Assignee: Pony.ai, Inc.
• Inventors: Zhang, Zhuo, Xiao, Sinan, Yu, Xiang, Song, Hao, Li, Tianyi, Xiao, Bo, Hou, Jie, Liu, Yiming, Lou, Tiancheng, Peng, Jun
• Primary Examiner(s): Marc-Coleman, Marthe Y
• Application Number:
  US15/804,187
• Publication Number:
  US 20190137996A1
• Time in Patent Office: 729 Days
• Field of Search: 701 27
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3415 Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G08G 1/087 Override of traffic control, e.g. by signal transmitted by an emergency vehicle
• CPC Class Codes: G08G 1/0965 responding to signals from another vehicle, e.g. emergency vehicle
• CPC Class Codes: G08G 1/205 Indicating the location of the monitored vehicles as destination, e.g. accidents, stolen, rental

Abstract

A system included and a computer-implemented method performed in one of a plurality of self-driving vehicles that are connected through a network are described. The system performs: processing image data of one or more scene images received by said one of the plurality of self-driving vehicles, to detect one or more objects included in the one or more scene images; determining a target object from the one or more detected objects at least based on the processed image data; predicting movement of the target object at least based on a current position and a current movement state of the target object; and performing a self-driving operation to drive said one of the plurality of self-driving vehicles based on the predicted movement of the target object.

Claims

1. A system included in a self-driving vehicle of a plurality of self-driving vehicles that are connected through a network, comprising:

one or more processors; and

a memory storing instructions that, when executed by the one or more processors, cause the system to perform;

processing image data of one or more scene images received by the self-driving vehicle, to detect one or more objects included in the one or more scene images;

determining a target object from the one or more detected objects at least based on the processed image data;

predicting movement of the target object at least based on a current position and a current movement state of the target object;

performing a self-driving operation to drive the self-driving vehicle based on the predicted movement of the target object; and

determining whether to sacrifice the self-driving vehicle based on a comparison of a danger to others if the target object continues to run and a danger to others if the target object is forcibly stopped.

2. The system of claim 1, wherein said determining the target object comprises:

receiving a feature of the target object from an external source that is external to said one of the plurality of self-driving vehicles; and

comparing the received feature of the target object with a corresponding feature of one or more of the detected objects that are obtained from the processed image data.

3. The system of claim 1, wherein said determining the target object comprises:

determining whether or not movement of a selected one of the detected objects does not match a predetermined standard; and

upon determining that the movement of the selected one of the detected objects does not match the predetermined standard, determining the selected one of the detected objects as the target object.

4. The system of claim 1, wherein the current movement state of the target object includes a direction of movement of the target object, an orientation of the target object, and a speed of the movement of the target object.

5. The system of claim 1, wherein the system is caused to further perform communication with another one of the plurality of self-driving vehicles through the network, based on the predicted movement of the target object.

6. The system of claim 5, wherein the communication includes sending an instruction to take a designated route to said another one of the plurality of self-driving vehicles.

7. The system of claim 1, the system is caused to further perform communication with a traffic signal system to cause one or more traffic signals to be at a designated state.

8. The system of claim 7, wherein, the performing communication with the traffic signal system comprises, in response to the system sending a command to the traffic signal system that contradicts a command from another source, determining the designated state of the traffic signal on a first-come-first-serve basis.

9. The system of claim 7, wherein the system is caused to further perform:

determining whether the target object recognizes the system; and

predicting a position of the target object in response to determining that the target object recognizes the system.

10. The system of claim 7, wherein the performing the self-driving operation comprises proceeding ahead of a non-self-driving emergency vehicle system while pursuing the target object.

11. The system of claim 1, the system is caused to further perform generating an on-site notification from said one of the plurality of self-driving vehicles, based on the predicted movement of the target object.

12. The system of claim 1, the system is caused to further perform updating the predicted movement of the target object, based on the self-driving operation that has been performed.

13. The system of claim 1, wherein the system is further caused to, in response to determining to sacrifice the self-driving vehicle, select a time and a place of the self-driving operation that minimizes an injury to a passenger in the target object.

14. The system of claim 1, wherein the determining whether to sacrifice the self-driving vehicle is based on a probability of capturing the target object, and a probability of capturing the target object by another of the self-driving vehicles.

15. The system of claim 1, wherein the system is caused to further receive, from a server, information of a crime history of the target object, whether or not the target object is carrying a weapon, and whether or not the target object is with a hostage.

16. A computer-implemented method performed in one of a plurality of self-driving vehicles that are connected through a network, the method comprising:

processing image data of one or more scene images received by said one of the plurality of self-driving vehicles, to detect one or more objects included in the one or more scene images;

determining a target object from the one or more detected objects at least based on the processed image data;

predicting movement of the target object at least based on a current position and a current movement state of the target object; and

performing a self-driving operation to drive said one of the plurality of self-driving vehicles based on the predicted movement of the target object; and

determining whether to sacrifice the self-driving vehicle based on a comparison of a danger to others if the target object continues to run and a danger to others if the target object is forcibly stopped.

17. The computer-implemented method of claim 16, wherein said determining the target object comprises:

receiving a feature of the target object from an external source that is external to said one of the plurality of self-driving vehicles; and

comparing the received feature of the target object with a corresponding feature of one or more of the detected objects that are obtained from the processed image data.

18. The computer-implemented method of claim 16, further comprising performing communication with another one of the plurality of self-driving vehicles through the network, based on the predicted movement of the target object.

19. The computer-implemented method of claim 16, further comprising communicating with a traffic signal system to cause one or more traffic signals to be at a designated state.

20. The computer-implemented method of claim 16, further comprising updating the predicted movement of the target object, based on the self-driving operation that has been performed.