Dynamic in-cabin autonomous vehicle control systems

• US 10,372,126 B2
• Filed: 05/27/2016
• Issued: 08/06/2019
• Est. Priority Date: 05/27/2016
• Status: Active Grant

• Current Assignee: Autonomous Solutions, Inc., CNH Industrial America LLC (CNH Industrial N.V.)
• Original Assignee: CNH Industrial America LLC (CNH Industrial N.V.)
• Inventors: Foster, Christopher A., Posselius, John H., Turpin, Bret T., Morwood, Daniel J.
• Primary Examiner(s): McPherson, James M
• Application Number:
  US15/166,734
• Publication Number:
  US 20170344004A1
• Time in Patent Office: 1,166 Days
• Field of Search: None
• CPC Class Codes: A01B 69/00 Steering of agricultural machines or implements; Guiding agricultural machines or implements on a desired track
• CPC Class Codes: B62D 49/06 adapted for multi-purpose use
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0295 by at least one leading vehicle of the fleet

Abstract

One embodiment describes a control system in an automation system including a first portion located at a first vehicle, which includes a first autonomous module that autonomously controls operation of the first vehicle to perform operations in a first area based at least in part on a first target operation result while the first portion is in an autonomous mode; and a second portion located at a second vehicle, in which the second portion includes a second autonomous module that autonomously controls operation of the second vehicle to perform operations in a second area based at least in part on a second target operation result while the second portion is in the autonomous mode and a first command module that determines the first target operation result and the second target operation result based at least in part on a global plan that indicates a total target operation result.

Claims

1. A control system configured to facilitate controlling operation of an agricultural automation system, comprising:

a first portion located at a first autonomous agricultural vehicle, wherein the first portion of the control system comprises;

a first autonomous module configured to autonomously control operation of the first autonomous agricultural vehicle to perform first agricultural operations in a first area of a field based at least in part on a first target operation result while the first portion of the control system is in an autonomous mode; and

a first command module; and

a second portion located at a second autonomous agricultural vehicle, wherein the second portion is communicatively coupled to the first portion of the control system via a network and comprises;

a second autonomous module configured to autonomously control operation of the second autonomous agricultural vehicle to perform second agricultural operations in a second area of the field based at least in part on a second target operation result while the second portion of the control system is in the autonomous mode; and

a second command module;

wherein the control system is configured to;

deactivate the first command module located at the first autonomous agricultural vehicle and activate the second command module located at the second autonomous agricultural vehicle when an operator is located at the second autonomous agricultural vehicle to enable the operator to centrally monitor and control operation of the agricultural automation system from the second autonomous agricultural vehicle; and

activate the first command module located at the first autonomous agricultural vehicle and deactivate the second command module located at the second autonomous agricultural vehicle when the operator is located at the first autonomous agricultural vehicle to enable the operator to centrally monitor and control operation of the agricultural automation system from the first autonomous agricultural vehicle;

wherein an activated command module in the control system is configured to determine the first target operation result and the second target operation result based at least in part on a global plan that indicates a total target operation result of the agricultural automation system.

2. The control system of claim 1, wherein the second portion of the control system is configured to:

receive user inputs from the operator via one or more input devices within the second autonomous agricultural vehicle;

instruct the second autonomous agricultural vehicle to adjust operation based at least in part on the user inputs when the second portion of the control system is in a manual mode to enable the operator to manually control operation of the second autonomous agricultural vehicle; and

when a disturbance condition affecting operation of the first autonomous agricultural vehicle is detected;

instruct the first portion of the control system to switch to the manual mode;

communicate the user inputs to first portion of the control system via the network; and

instruct the first portion of the control system to adjust operation of the first autonomous agricultural vehicle based at least in part on the user inputs to enable the operator to manually control operation of the first autonomous agricultural vehicle from the second autonomous agricultural vehicle.

3. The control system of claim 1, wherein:

the first target operation result is configured to indicate the first area in the field, first target locations in the first area at which to perform the first agricultural operations, density with which to perform the first agricultural operations in the first area, spacing with which to perform the first agricultural operations in the first area, or any combination thereof;

the second target operation result is configured to indicate the second area in the field, second target locations in the second area at which to perform the second agricultural operations, density with which to perform the second agricultural operations in the second area, spacing with which to perform the second agricultural operations in the second area, or any combination thereof; and

the global plan comprises the first target operation result and the second target operation result.

4. The control system of claim 1, wherein:

when the first command module located at the first autonomous agricultural vehicle is activated, the second command module is configured to;

receive sensor data from one or more sensors; and

instruct a first display in a first cabin of the first autonomous agricultural vehicle to present a visual representation of the sensor data to enable the operator to control and monitor operation of the agricultural automation system based at least in part on the sensor data and information gathered by the operator while located in the first cabin of the first autonomous agricultural vehicle; and

when the second command module located at the second autonomous agricultural vehicle is activated, the second command module is configured to;

receive the sensor data from the one or more sensors; and

instruct a second display in a second cabin of the second autonomous agricultural vehicle to present the visual representation of the sensor data to enable the operator to control and monitor operation of the agricultural automation system based at least in part on the sensor data and information gathered by the operator while located in the second cabin of the second autonomous agricultural vehicle.

5. The control system of claim 1, wherein:

the first autonomous module located at the first autonomous agricultural vehicle is configured to;

determine first operational parameters to achieve the first target operation result of the first autonomous agricultural vehicle; and

instruct the first autonomous agricultural vehicle to implement the first operational parameters; and

the second autonomous module located at the second autonomous agricultural vehicle is configured to;

determine second operational parameters to achieve the second target operation result of the second autonomous agricultural vehicle; and

instruct the second autonomous agricultural vehicle to implement the second operational parameters.

6. The control system of claim 1, wherein:

the first portion of the control system is configured to instruct the first autonomous agricultural vehicle to tow a first agricultural attachment to facilitate performing the first agricultural operations at first target locations in the first portion of the field indicated by the first target operation result; and

the second portion of the control system is configured to instruct the second autonomous agricultural vehicle to tow a second agricultural attachment to facilitate performing the second agricultural operations at second target locations in the second portion of the field indicated by the second target operation result.

7. The control system of claim 1, wherein the first autonomous agricultural vehicle and the second autonomous agricultural vehicle each comprises a tractor, an off-road vehicle, or a work vehicle.

8. An agricultural automation system comprising:

a first autonomous agricultural vehicle, wherein the first autonomous agricultural vehicle comprises a first autonomous module configured to autonomously control operation of the first autonomous agricultural vehicle to perform first agricultural operations in a first area of a field based at least in part on a first target operation result while the first autonomous agricultural vehicle is in an autonomous mode; and

a second autonomous agricultural vehicle, wherein the second autonomous agricultural vehicle comprises;

a second autonomous module configured to autonomously control operation of the second autonomous agricultural vehicle to perform second agricultural operations in a second area of the field based at least in part on a second target operation result while the second autonomous agricultural vehicle in the autonomous mode;

a command module is configured to determine the first target operation result and the second target operation result based at least in part on a global plan that indicates a total target operation result of the agricultural automation system when activated; and

one or more input devices configured to receive user inputs at the second autonomous agricultural vehicle, wherein the second autonomous agricultural vehicle is configured to communicate a user input received via the one or more input devices at the second autonomous agricultural vehicle to the first autonomous agricultural vehicle to enable an operator to manually control operation of the first autonomous agricultural vehicle from the second autonomous agricultural vehicle when the first autonomous agricultural vehicle is in a manual mode.

9. The agricultural automation system of claim 8, wherein:

the one or more input devices comprise a steering wheel;

the user input comprises actuation of the steering wheel; and

when the first autonomous agricultural vehicle is in the manual mode;

the second autonomous agricultural vehicle is configured to communicate the actuation of the steering wheel at the second autonomous agricultural vehicle to the first autonomous agricultural vehicle; and

the first autonomous agricultural vehicle is configured to adjust travel direction based at least in part on the actuation of the steering wheel at the second autonomous agricultural vehicle.

10. The agricultural automation system of claim 8, wherein:

the one or more input devices comprise a pedal, wherein the pedal comprises a gas pedal or a brake pedal;

the user input comprises depression of the pedal; and

when the first autonomous agricultural vehicle is in the manual mode;

the second autonomous agricultural vehicle is configured to communicate the depression of the pedal at the second autonomous agricultural vehicle to the first autonomous agricultural vehicle; and

the first autonomous agricultural vehicle is configured to adjust travel speed based at least in part on the depression of the pedal at the second autonomous agricultural vehicle.

11. The agricultural automation system of claim 8, wherein the second autonomous agricultural vehicle is configured to adjust travel direction, travel speed, or both based at least in part on the user inputs when the second autonomous agricultural vehicle is in the manual mode.

12. The agricultural automation system of claim 8, wherein the first autonomous agricultural vehicle comprises another command module, wherein:

the other command module is configured to be deactivated while the command module in the second autonomous agricultural vehicle is activated;

the other command module is configured to determine the first target operation result and the second target operation result based at least in part on the global plan that indicates the total target operation result of the agricultural automation system when activated; and

the command module in the second autonomous agricultural vehicle is configured to be deactivated while the other command module in the first autonomous agricultural vehicle is activated.

13. The autonomous agricultural system of claim 12, wherein:

the command module in the second autonomous agricultural vehicle is configured to be activated when the operator is located at the second autonomous agricultural vehicle; and

the other command module in the first autonomous agricultural vehicle is configured to be activated when the operator is located at the first autonomous agricultural vehicle.

14. A control system configured to facilitate controlling operation of an agricultural automation system, comprising:

a first portion located at a first autonomous agricultural vehicle, wherein the first portion of the control system comprises;

a first autonomous module configured to autonomously control operation of the first autonomous agricultural vehicle to perform first agricultural operations in a first area of a field based at least in part on a first target operation result while the first portion of the control system is in an autonomous mode; and

a first command module, wherein the control system is configured to selectively activate the first command module based at least in part on location of an operator; and

a second portion located at a second autonomous agricultural vehicle, wherein the second portion is communicatively coupled to the first portion of the control system via a network and comprises;

a second autonomous module configured to autonomously control operation of the second autonomous agricultural vehicle to perform second agricultural operations in a second area of the field based at least in part on a second target operation result while the second portion of the control system is in the autonomous mode; and

a second command module, wherein the control system is configured to selectively activate the second command module based at least in part on the location of the operator;

wherein an activated command module in the control system is configured to determine the first target operation result and the second target operation result based at least in part on a global plan that indicates a total target operation result of the agricultural automation system;

when the first command module located at the first autonomous agricultural vehicle is activated, the first command module is configured to;

wirelessly communicate the second target operation result of the second autonomous agricultural vehicle to the second autonomous module located at the second autonomous agricultural vehicle via the network; and

communicate the first target operation result of the first autonomous agricultural vehicle to the first autonomous module located at the first autonomous agricultural vehicle via a first internal bus; and

when the second command module located at the second autonomous agricultural vehicle is activated, the second command module is configured to;

wirelessly communicate the first target operation result of the first autonomous agricultural vehicle to the first autonomous module of the first autonomous agricultural vehicle via the network; and

communicate the second target operation result of the second autonomous agricultural vehicle to the second autonomous module located at the second autonomous agricultural vehicle via a second internal bus.

15. The control system of claim 14, wherein the control system is configured to:

deactivate the first command module located at the first autonomous agricultural vehicle and activate the second command module located at the second autonomous agricultural vehicle when the operator is located at the second autonomous agricultural vehicle to enable the operator to centrally monitor and control operation of the agricultural automation system from the second autonomous agricultural vehicle; and

activate the first command module located at the first autonomous agricultural vehicle and deactivate the second command module located at the second autonomous agricultural vehicle when the operator is located at the first autonomous agricultural vehicle to enable the operator to centrally monitor and control operation of the agricultural automation system from the first autonomous agricultural vehicle.

16. The control system of claim 14, wherein the second portion of the control system is configured to:

receive user inputs from the operator via one or more input devices within the second autonomous agricultural vehicle;

instruct the second autonomous agricultural vehicle to adjust operation based at least in part on the user inputs when the second portion of the control system is in a manual mode to enable the operator to manually control operation of the second autonomous agricultural vehicle; and

when a disturbance condition affecting operation of the first autonomous agricultural vehicle is detected;

instruct the first portion of the control system to switch to the manual mode;

communicate the user inputs to first portion of the control system via the network; and

instruct the first portion of the control system to adjust operation of the first autonomous agricultural vehicle based at least in part on the user inputs to enable the operator to manually control operation of the first autonomous agricultural vehicle from the second autonomous agricultural vehicle.

17. The control system of claim 14, wherein:

the first target operation result is configured to indicate the first area in the field, first target locations in the first area at which to perform the first agricultural operations, density with which to perform the first agricultural operations in the first area, spacing with which to perform the first agricultural operations in the first area, or any combination thereof;

the second target operation result is configured to indicate the second area in the field, second target locations in the second area at which to perform the second agricultural operations, density with which to perform the second agricultural operations in the second area, spacing with which to perform the second agricultural operations in the second area, or any combination thereof; and

the global plan comprises the first target operation result and the second target operation result.

18. The control system of claim 14, wherein:

the first autonomous module located at the first autonomous agricultural vehicle is configured to;

determine first operational parameters to achieve the first target operation result of the first autonomous agricultural vehicle; and

instruct the first autonomous agricultural vehicle to implement the first operational parameters; and

the second autonomous module located at the second autonomous agricultural vehicle is configured to;

determine second operational parameters to achieve the second target operation result of the second autonomous agricultural vehicle; and

instruct the second autonomous agricultural vehicle to implement the second operational parameters.

19. The control system of claim 14, wherein:

the first portion of the control system is configured to instruct the first autonomous agricultural vehicle to tow a first agricultural attachment to facilitate performing the first agricultural operations at first target locations in the first portion of the field indicated by the first target operation result; and

the second portion of the control system is configured to instruct the second autonomous agricultural vehicle to tow a second agricultural attachment to facilitate performing the second agricultural operations at second target locations in the second portion of the field indicated by the second target operation result.

20. The control system of claim 14, wherein the first autonomous agricultural vehicle and the second autonomous agricultural vehicle each comprises a tractor, an off-road vehicle, or a work vehicle.