Evaluating driving control systems for elegant driving

• US 10,696,306 B1
• Filed: 09/25/2019
• Issued: 06/30/2020
• Est. Priority Date: 09/25/2019
• Status: Active Grant

• Current Assignee: LYFT Incorporated
• Original Assignee: LYFT Incorporated
• Inventors: Benisch, Michael Jared, Danna, Emilie Jeanne Anne, Qureshi, Sameer
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Huynh, Luat T
• Application Number:
  US16/583,109
• Time in Patent Office: 279 Days
• Field of Search: 701 1
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2520/10 Longitudinal speed
• CPC Class Codes: B60W 2520/105 Longitudinal acceleration
• CPC Class Codes: B60W 2520/14 Yaw
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/05 Big data
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 40/10 related to vehicle motion
• CPC Class Codes: B60W 60/001 Planning or execution of driving tasks
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations

Abstract

In one embodiment, a method includes accessing a first set of driving metrics of a vehicle navigating a segment using a driving control system; computing, based on the first set of driving metrics, a driving characteristic for the vehicle. The driving characteristic of the vehicle being represented by the first set of driving metrics. The method also includes comparing the driving characteristic of the vehicle to a driving characteristic of a number of manually controlled vehicles represented by a second set of driving metrics collected while navigating the segment; and computing a score indicating an elegance of the driving control system in navigating the segment based on the comparison of the driving characteristic of the vehicle to the driving characteristic of the number of manually controlled vehicles.

Claims

1. A method comprising, by a computing system:

determining a first plurality of driving metrics of at least one vehicle navigating a road segment using a first driving control system, wherein the first plurality of driving metrics include one or more measurements associated with movement of the at least one vehicle while navigating the road segment, and wherein the first driving control system controls the movement of the at least one vehicle while navigating the road segment;

determining, based on the first plurality of driving metrics, a first driving characteristic for the first driving control system;

comparing the first driving characteristic of the first driving control system to a second driving characteristic of a plurality of manually controlled vehicles represented by a second plurality of driving metrics collected while each of the plurality of manually controlled vehicles are navigating the road segment;

computing a first score indicating an elegance of the first driving control system used in navigating the road segment based on the comparison of the first driving characteristic to the second driving characteristic;

determining a second score indicating an elegance of at least one vehicle navigating the road segment using a second driving control system;

evaluating the first driving control system relative to the second driving control system based on a comparison of the first score and the second score; and

selecting, based on the evaluation, the second driving control system for a vehicle, wherein the selected second driving control system is provided to the vehicle and wherein the second driving control system controls the movement of the vehicle.

2. The method of claim 1, wherein comparing the first driving characteristic to the second driving characteristic comprises calculating one or more differences between the first plurality of driving metrics of the at least one vehicle and the second plurality of driving metrics of the plurality of manually controlled vehicles.

3. The method of claim 2, wherein computing the first score further comprises

applying one or more weights to the one or more differences.

4. The method of claim 1, further comprising:

computing a plurality of scores for a plurality of road segments, respectively, wherein each of the plurality of scores indicates an elegance of the first driving control system used in navigating the corresponding road segment associated with that score; and

calculating a route score for a route based on the plurality of scores for the plurality of road segments, wherein the route comprises the plurality of road segments.

5. The method of claim 4, further comprising identifying one or more road segments in the plurality of road segments in which the first driving control system underperformed, wherein each of the identified one or more road segments is identified by comparing the associated score with a pre-determined threshold value.

6. The method of claim 5, further comprising identifying one or more scenarios of underperformance associated with the identified one or more road segments.

7. The method of claim 1, wherein the first plurality of driving metrics comprises one or more of an acceleration, a deceleration, a distance from a center of a driving lane, a distance from nearby objects, an amount of centripetal force, a steering angle, a change of acceleration, or an average speed over the segment.

8. The method of claim 1, wherein the elegance of the first driving control system indicates a level of driving similarity between the first driving control system and the plurality of manually controlled vehicles.

9. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to perform operations comprising:

determine a first plurality of driving metrics of at least one vehicle navigating a road segment using a first driving control system, wherein the first plurality of driving metrics include one or more measurements associated with movement of the at least one vehicle while navigating the road segment, and wherein the first driving control system controls the movement of the at least one vehicle while navigating the road segment;

determine, based on the first plurality of driving metrics, a first driving characteristic for the first driving control system;

compare the first driving characteristic of the first driving control system to a second driving characteristic of a plurality of manually controlled vehicles represented by a second plurality of driving metrics collected while each of the plurality of manually controlled vehicles are navigating the road segment;

compute a first score indicating an elegance of the first driving control system used in navigating the road segment based on the comparison of the first driving characteristic to the second driving characteristic;

determine a second score indicating an elegance of at least one vehicle navigating the road segment using a second driving control system;

evaluate the first driving control system relative to the second driving control system based on a comparison of the first score and the second score; and

select, based on the evaluation, the second driving control system for a vehicle, wherein the selected second driving control system is provided to the vehicle and wherein the second driving control system controls the movement of the vehicle.

10. The media of claim 9, wherein the software is further operable when executed to calculate one or more differences between the first plurality of driving metrics of the at least one vehicle and the second plurality of driving metrics of the plurality of manually controlled vehicles.

11. The media of claim 10, wherein the software is further operable when executed to

apply one or more weights to the one or more differences.

12. The media of claim 9, wherein the elegance of the first driving control system indicates a level of driving similarity between the first driving control system and the plurality of manually controlled vehicles.

13. The media of claim 9, wherein the software is further operable when executed to:

compute a plurality of scores for a plurality of road segments, respectively, wherein each of the plurality of scores indicates an elegance of the first driving control system used in navigating the corresponding road segment associated with that score; and

calculate a route score for a route based on the plurality of scores for the plurality of road segments, wherein the route comprises the plurality of road segments.

14. The media of claim 13, wherein the software is further operable when executed to identify one or more road segments in the plurality of road segments in which the first driving control system underperformed, wherein each of the identified one or more road segments is identified by comparing the associated score with a pre-determined threshold value.

15. A computing system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to the one or more processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by the one or more processors to cause the computing system to perform operations comprising;

determine a first plurality of driving metrics of at least one vehicle navigating a road segment using a first driving control system, wherein the first plurality of driving metrics include one or more measurements associated with movement of the at least one vehicle while navigating the road segment, and wherein the first driving control system controls the movement of the at least one vehicle while navigating the road segment;

determine, based on the first plurality of driving metrics, a first driving characteristic for the first driving control system;

compare the first driving characteristic of the first driving control system to a second driving characteristic of a plurality of manually controlled vehicles represented by a second plurality of driving metrics collected while each of the plurality of manually controlled vehicles are navigating the road segment;

compute a first score indicating an elegance of the first driving control system used in navigating the road segment based on the comparison of the first driving characteristic to the second driving characteristic;

determine a second score indicating an elegance of at least one vehicle navigating the road segment using a second driving control system;

evaluate the first driving control system relative to the second driving control system based on a comparison of the first score and the second score; and

select, based on the evaluation, the second driving control system for a vehicle, wherein the selected second driving control system is provided to the vehicle and wherein the second driving control system controls the movement of the vehicle.

16. The system of claim 15, wherein the one or more processors are further operable to calculate one or more differences between the first plurality of driving metrics of the at least one vehicle and the second plurality of driving metrics of the plurality of manually controlled vehicles.

17. The system of claim 16, wherein the one or more processors are further operable to

apply one or more weights to the one or more differences.

18. The system of claim 17, wherein the elegance of the first driving control system indicates a level of driving similarity between the first driving control system and the plurality of manually controlled vehicles.

19. The system of claim 15, wherein the one or more processors are further operable to:

compute a plurality of scores for a plurality of road segments, respectively, wherein each of the plurality of scores indicates an elegance of the first driving control system used in navigating the corresponding road segment associated with that score; and

calculate a route score for a route based on the plurality of scores for the plurality of road segments, wherein the route comprises the plurality of road segments.

20. The system of claim 19, wherein the one or more processors are further operable to identify one or more road segments in the plurality of road segments in which the first driving control system underperformed, wherein each of the identified one or more road segments is identified by comparing the associated score with a pre-determined threshold value.