Collaborative vehicle control using both human operator and automated controller input

• US 9,464,410 B2
• Filed: 05/19/2011
• Issued: 10/11/2016
• Est. Priority Date: 05/19/2011
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Johnson, David August, Sanchez, Julian, Anderson, Eric R., Sahlin, Mark P.
• Primary Examiner(s): Phan, Hai
• Assistant Examiner(s): Bousono, Orlando
• Application Number:
  US13/111,500
• Publication Number:
  US 20120293316A1
• Time in Patent Office: 1,972 Days
• Field of Search: 340/3.1, 340/3.3, 340/3.42, 340/3.44, 340/4.36, 340/5.1, 340/438, 340/439, 700159-195, 701 291- 292, 701/50, 702182-183
• US Class Current: 1/1
• CPC Class Codes: B60W 2050/0071 Controller overrides driver automatically
• CPC Class Codes: B60W 2050/0073 Driver overrides controller
• CPC Class Codes: B60Y 2200/412 Excavators
• CPC Class Codes: E02F 3/434 providing automatic sequences of movements, e.g. automatic dumping or loading, automatic return-to-dig
• CPC Class Codes: E02F 9/2012 Setting the functions of the control levers, e.g. changing assigned functions among operations levers, setting functions dependent on the operator or seat orientation
• CPC Class Codes: E02F 9/2029 Controlling the position of implements in function of its load, e.g. modifying the attitude of implements in accordance to vehicle speed control for hydraulic or pneumatic drives E02F9/2203, E02F9/2221 and E02F9/2253 take precedence
• CPC Class Codes: E02F 9/2037 Coordinating the movements of the implement and of the frame
• CPC Class Codes: E02F 9/2041 Automatic repositioning of implements, i.e. memorising determined positions of the implement for dipper-arms or bucket-arms E02F3/434, E02F3/437, E02F3/438, E02F3/439 take precedence
• CPC Class Codes: E02F 9/2079 Control of mechanical transmission for hydrostatic transmission or hydraulic torque converter E02F9/2253
• CPC Class Codes: E02F 9/2083 Control of vehicle braking systems
• CPC Class Codes: E02F 9/2087 Control of vehicle steering for steering with hydraulic or pneumatic drives E02F9/225
• CPC Class Codes: E02F 9/265 with follow-up actions (e.g. control signals sent to actuate the work tool)
• CPC Class Codes: G09B 19/167 Control of land vehicles
• CPC Class Codes: G09B 9/04 for teaching control of land vehicles

Abstract

Control of a vehicle to perform a task employs first control input provided by a human operator and second control input provided by an automated controller. A relationship between the first control input and the second control input is determined. Either the first control input or the second control input is used to control the vehicle to perform the task based on the determined relationship. An indicator may be used to indicate to the operator when the control input provided by the human operator is being used to control the vehicle.

Claims

1. A method of controlling a vehicle, comprising:

obtaining, by a processor unit, a first control input, wherein the first control input is relative to a manual input provided by a human operator;

obtaining, by the processor unit, a number of sensor inputs, wherein the number of sensor inputs indicate a state of the vehicle;

generating, by the processor unit, a second control input using the number of sensor inputs;

determining, by the processor unit, a relationship between the first control input and the second control input;

selecting, by the processor unit, one of the first control input and the second control input based on the determined relationship between the first control input and the second control input;

responsive to selecting the first control input, using the manual input provided by the human operator to control the vehicle; and

responsive to selecting the second control input, using the second control input to control the vehicle.

2. The method of claim 1, wherein:

determining the relationship between the first control input and the second control input comprises determining which one of the first control input and the second control input has a lower value; and

selecting one of the first control input and the second control input comprises selecting the first control input responsive to a determination that the first control input has the lower value and selecting the second control input responsive to a determination that the second control input has the lower value.

3. The method of claim 1, wherein:

determining the relationship between the first control input and the second control input comprises determining whether the first control input is within a range from the second control input; and

selecting one of the first control input and the second control input comprises selecting the first control input responsive to a determination that the first control input is within the range from the second control input and selecting the second control input responsive to a determination that the first control input is not within the range from the second control input.

4. The method of claim 3, wherein the range extends from less than the second control input to greater than the second control input.

5. The method of claim 1 further comprising:

responsive to selecting the first control input, controlling, by the processor unit, an indicator to indicate to the human operator that the manual input provided by the human operator is being used to control the vehicle.

6. The method of claim 5, wherein the indicator is a visual indicator and wherein controlling the indicator comprises turning the visual indicator on to indicate to the human operator that the manual input provided by the human operator is being used to control the vehicle.

7. The method of claim 1, wherein:

the vehicle is a loader comprising a boom and a bucket attached to the boom;

using the manual input provided by the human operator to control the vehicle comprises using the manual input provided by the human operator to control movement of the boom and the bucket to load a material into the bucket; and

using the second control input to control the vehicle comprises using the second control input to control movement of the boom and the bucket to load the material into the bucket.

8. The method of claim 1, wherein the number of sensor inputs comprises an input from a sensor configured to detect a level of torque applied to a wheel of the vehicle.

9. An apparatus, comprising:

a human operator controller configured to receive manual input from a human operator to control a vehicle, wherein the human operator controller is configured to provide a first control input relative to the manual input;

a number of sensors mounted on the vehicle, wherein the number of sensors are configured to generate a number of sensor inputs and wherein the number of sensor inputs indicate a state of the vehicle; and

a processor unit coupled to the human operator controller and to the number of sensors and configured to;

generate a second control input using the number of sensor inputs,

determine a relationship between the first control input and the second control input,

select one of the first control input and the second control input based on the determined relationship between the first control input and the second control input,

generate first control signals to control the vehicle using the manual input provided by the human operator responsive to selecting the first control input, and

generate the first control signals to control the vehicle using the second control input responsive to selecting the second control input.

10. The apparatus of claim 9, wherein the processor unit is configured to:

determine the relationship between the first control input and the second control input by determining which one of the first control input and the second control input has a lower value; and

select one of the first control input and the second control input by selecting the first control input responsive to a determination that the first control input has the lower value and selecting the second control input responsive to a determination that the second control input has the lower value.

11. The apparatus of claim 9, wherein the processor unit is configured to:

determine the relationship between the first control input and the second control input by determining whether the first control input is within a range from the second control input; and

select one of the first control input and the second control input by selecting the first control input responsive to a determination that the first control input is within the range from the second control input and selecting the second control input responsive to a determination that the first control input is not within the range from the second control input.

12. The apparatus of claim 11, wherein the range extends from less than the second control input to greater than the second control input.

13. The apparatus of claim 9 further comprising an indicator and wherein the processor unit is coupled to the indicator and the processor unit is further configured to generate second control signals responsive to selecting the first control input, wherein the second control signals control the indicator to indicate to the human operator that the manual input provided by the human operator is being used to control the vehicle.

14. The apparatus of claim 13, wherein the indicator is a visual indicator and wherein the second control signals comprise control signals configured to turn the visual indicator on to indicate to the human operator that the manual input provided by the human operator is being used to control the vehicle.

15. The apparatus of claim 9, wherein:

the vehicle is a loader comprising a boom and a bucket attached to the boom; and

the first control signals are configured to control movement of the boom and the bucket to load a material into the bucket.

16. The apparatus of claim 9, wherein the number of sensors comprises a sensor configured to detect a level of torque applied to a wheel of the vehicle.

17. The apparatus of claim 9, wherein the human operator controller comprises a joystick.

18. A computer program product for controlling a vehicle, comprising:

a non-transitory computer readable storage medium;

first program instructions to obtain a first control input, wherein the first control input is relative to a manual input provided by a human operator;

second program instructions to obtain a number of sensor inputs, wherein the number of sensor inputs indicate a state of the vehicle;

third program instructions to generate a second control input using the number of sensor inputs;

fourth program instructions to determine a relationship between the first control input and the second control input;

fifth program instructions to select one of the first control input and the second control input based on the determined relationship between the first control input and the second control input;

sixth program instructions to generate first control signals to control the vehicle using the manual input provided by the human operator responsive to selecting the first control input;

seventh program instructions to generate the first control signals to control the vehicle using the second control input responsive to selecting the second control input; and

wherein the first, second, third, fourth, fifth, sixth, and seventh program instructions are stored on the computer readable storage medium.

19. The computer program product of claim 18, wherein:

the fourth program instructions comprise program instructions to determine which one of the first control input and the second control input has a lower value; and

the fifth program instructions comprise program instructions to select the first control input responsive to a determination that the first control input has the lower value and to select the second control input responsive to a determination that the second control input has the lower value.

20. The computer program product of claim 18, wherein:

the fourth program instructions comprise program instructions to determine whether the first control input is within a range from the second control input; and

the fifth program instructions comprise program instructions to select the first control input responsive to a determination that the first control input is within the range from the second control input and to select the second control input responsive to a determination that the first control input is not within the range from the second control input.

21. The computer program product of claim 20, wherein the range extends from less than the second control input to greater than the second control input.

22. The computer program product of claim 18 further comprising:

eighth program instructions to generate second control signals to control an indicator to indicate to the human operator that the manual input provided by the human operator is being used to control the vehicle responsive to selecting the first control input; and

wherein the eighth program instructions are stored on the computer readable storage medium.

23. The computer program product of claim 22, wherein the indicator is a visual indicator and wherein the second control signals comprise control signals configured to turn the visual indicator on to indicate to the human operator that the manual input provided by the human operator is being used to control the vehicle.

24. The computer program product of claim 18, wherein:

the vehicle is a loader comprising a boom and a bucket attached to the boom; and

wherein the first control signals are configured to control movement of the boom and the bucket to load a material into the bucket.

25. The computer program product of claim 18, wherein the number of sensor inputs comprises an input from a sensor configured to detect a level of torque applied to a wheel of the vehicle.