Supervisory control of vehicles

• US 10,303,166 B2
• Filed: 05/23/2016
• Issued: 05/28/2019
• Est. Priority Date: 05/23/2016
• Status: Active Grant

• Current Assignee: Motional AD, LLC (Aptiv PLC)
• Original Assignee: nuTonomy, Inc. (BorgWarner Incorporated)
• Inventors: Iagnemma, Karl
• Primary Examiner(s): Nguyen, Cuong H
• Application Number:
  US15/161,996
• Publication Number:
  US 20170336788A1
• Time in Patent Office: 1,100 Days
• Field of Search: 701 2, 701 22, 701 23, 701 25, 701123, 340901, 340905, 340435, 3409921
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2540/229 Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/4045 Intention, e.g. lane change or imminent movement
• CPC Class Codes: B60W 2554/60 Traversable objects, e.g. speed bumps or curbs
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 2710/00 Output or target parameters relating to a particular sub-units
• CPC Class Codes: B60W 2756/10 Involving external transmission of data to or from the vehicle
• CPC Class Codes: B60W 30/143 Speed control B60W30/16 takes precedence
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: B60W 60/00253 Taxi operations
• CPC Class Codes: G01C 21/3658 Lane guidance
• CPC Class Codes: G05D 1/0038 : by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0055 : with safety arrangements
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means

Abstract

Among other things, a command is received expressing an objective for operation of a vehicle within a denominated travel segment of a planned travel route. The objective spans a time series of (for example, is expressed at a higher or more abstract level than) control inputs that are to be delivered to one or more of the brake, accelerator, steering, or other operational actuator of the vehicle. The command is expressed to cause operation of the vehicle along a selected man-made travel structure of the denominated travel segment. A feasible manner of operation of the vehicle is determined to effect the command. A succession of control inputs is generated to one or more of the brake, accelerator, steering or other operational actuator of the vehicle in accordance with the determined feasible manner of operation.

Claims

1. A method comprising:

receiving, by an autonomous vehicle, a command from a source that is remote from the autonomous vehicle, the command expressing an objective for operation of the autonomous vehicle within a travel segment of a travel route, the travel segment comprising a highway, road, or street, the objective spanning a time series of control inputs that are to be delivered by the remote source to the autonomous vehicle;

determining an operation of the autonomous vehicle to effect the command, the operation comprising a candidate travel speed and a candidate travel lane of the travel segment associated with the time series of control inputs;

generating the time series of control inputs for one or more actuators of the autonomous vehicle in accordance with the determined operation; and

causing the autonomous vehicle to perform the determined operation in accordance with the time series of control inputs.

2. The method of claim 1 in which the source comprises a teleoperation facility.

3. The method of claim 1 in which the source is located in another vehicle.

4. The method of claim 1 in which the command is received from a human being or a process or a combination of a human being and a process at the remote source.

5. The method of claim 1 in which a display of the autonomous vehicle and its environment is provided at the remote source to a human operator.

30. The method of claim 1 in which the denominated travel segment comprises a named or numbered highway, road, or street.

6. A method comprising:

receiving, by an autonomous vehicle, a command from a human operator at the autonomous vehicle in response to a display of available options for alternative objectives, the command expressing an objective for operation of the autonomous vehicle within a travel segment of a travel route, the objective spanning a time series of control inputs that are to be delivered to the autonomous vehicle;

determining an operation of the autonomous vehicle to effect the command, the operation comprising a candidate travel speed and a candidate travel lane of the travel segment associated with the time series of control inputs;

generating the time series of control inputs for one or more actuators of the autonomous vehicle in accordance with the determined operation; and

causing the autonomous vehicle to perform in accordance with the time series of control inputs.

7. The method of claim 6 in which the display comprises a video display within the autonomous vehicle.

8. The method of claim 7 in which the available options are displayed as icons.

9. The method of claim 6 in which the available options are displayed on a display that is part of a steering wheel, a center console, or a back of a front seat.

10. The method of claim 6 in which the available options are displayed on a head up display.

11. The method of claim 6 in which the available options are displayed together with a representation of at least one man-made travel structure towards which the autonomous vehicle is traveling.

12. The method of claim 6 in which the objective comprises a travel lane.

13. The method of claim 6 in which the objective comprises other than a travel lane.

14. A method comprising:

receiving, by an autonomous vehicle, a command expressing an objective for operation of autonomous vehicle within a travel segment of a travel route, the objective spanning a time series of control inputs that are to be delivered to the autonomous vehicle;

determining an operation of the autonomous vehicle to effect the command, the operation comprising a candidate travel speed and a candidate travel lane of the travel segment associated with the time series of control inputs;

generating the time series of control inputs for one or more actuators of the autonomous vehicle in accordance with the determined operation; and

causing the autonomous vehicle to perform the determined operation in accordance with the time series of control inputs.

15. The method of claim 14 in which the maneuver comprises two or more lane changes.

16. The method of claim 14 in which the maneuver comprises changing speed.

17. The method of claim 14 in which the maneuver comprises bringing the autonomous vehicle to a stop at a place where a stop is permissible.

18. The method of claim 14 in which the maneuver comprises entering a shoulder of the road.

19. The method of claim 14 in which the maneuver comprises proceeding on a ramp.

20. The method of claim 14 in which the maneuver comprises a U-turn.

21. The method of claim 14 in which the maneuver comprises an emergency stop.

22. The method of claim 14 in which the determining of the operation of the autonomous vehicle to effect the command comprises determining that the manner of operation will not violate a rule of operation of the autonomous vehicle.

37. The method of claim 14 in which the command is received from a human operator in the autonomous vehicle.

23. A method comprising:

receiving, by an autonomous vehicle, a command expressing an objective for operation of autonomous vehicle within a travel segment of a travel route, the objective spanning a time series of control inputs that are to be delivered to the autonomous vehicle;

determining an operation of the autonomous vehicle to effect the command, the operation comprising a candidate travel speed and a candidate travel lane of the travel segment associated with the time series of control inputs;

confirming that the operation will not violate a rule of operation of the autonomous vehicle;

generating the time series of control inputs for one or more actuators of the autonomous vehicle in accordance with the determined operation; and

causing the autonomous vehicle to perform the determined operation in accordance with the time series of control inputs.

24. The method of claim 23 comprising, if the operation will violate a rule of operation of the autonomous vehicle, minimizing the extent of the violation of the rule or ruling out the violation entirely or applying a compromise strategy between enforcing the rules of operation or ignoring them.

25. The method of claim 23 in which the objective comprises a lane change.

26. A method comprising:

receiving, by an autonomous vehicle, a command in response to an operator activating a button located in the autonomous vehicle, the command expressing an objective for operation of autonomous vehicle within a travel segment of a travel route, the objective spanning a time series of control inputs that are to be delivered by a remote source to the autonomous vehicle;

determining an operation of the autonomous vehicle to effect the command, the operation comprising a candidate travel speed and a candidate travel lane of the travel segment associated with the time series of control inputs;

generating the time series of control inputs for one or more actuators of the autonomous vehicle in accordance with the determined operation; and

causing the autonomous vehicle to perform the determined operation in accordance with the time series of control inputs to achieve the objective.

27. A method comprising:

receiving, by an autonomous vehicle, a command from a computer process expressing an objective for operation of autonomous vehicle within a travel segment of a travel route, the objective spanning a time series of control inputs that are to be delivered to the autonomous vehicle;

determining an operation of the vehicle to effect the command, the operation comprising a candidate travel speed and a candidate travel lane of the travel segment associated with the time series of control inputs;

generating the time series of control inputs for one or more actuators of the autonomous vehicle in accordance with the determined operation; and

causing the autonomous vehicle to perform the determined operation in accordance with the time series of control inputs to achieve the objective.

28. The method of claim 27 in which the computer process is running in a location that is remote from the autonomous vehicle.

29. The method of claim 27 in which the computer process is running at the autonomous vehicle.

31. A method comprising

displaying to a human operator or other passenger of an autonomous vehicle a representation of one or more optional objectives for operation of the autonomous vehicle within a travel segment of a travel route, each of the objectives spanning a time series of control inputs that are to be delivered to one or more actuators of the autonomous vehicle;

and

receiving from the human operator or other passenger a selection of one or more of the objectives;

forming one or more commands based on the selection, the commands being expressed to cause operation of the autonomous vehicle along the travel segment;

determining an operation of the autonomous vehicle to effect the one or more commands, the operation comprising a candidate travel speed and a candidate travel lane of the travel segment associated with the time series of control inputs;

generating a time series of control inputs for one or more actuators of the autonomous vehicle in accordance with the determined operation, and

causing the autonomous vehicle to perform the determined operation in accordance with the time series of control inputs to achieve the objective.

32. The method of claim 31 in which the displayed representation comprises icons each representing one or more of the optional objectives.

33. The method of claim 31 in which the displayed representation comprises visual representations of the one or more optional objectives overlaid on a representation of at least a portion of the denominated travel segment.

34. The method of claim 31 in which the optional objectives are displayed on a head up display.

35. The method of claim 31 in which the optional objectives are displayed on a video display.

36. The method of claim 31 in which the optional objectives are displayed on a steering wheel.