Neurointerface for human control of complex machinery

• US 20010044789A1
• Filed: 02/13/2001
• Published: 11/22/2001
• Est. Priority Date: 02/17/2000
• Status: Abandoned Application

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Lamego, Marcelo M., Widrow, Bernard
• Application Number:
  US09/782,898
• Publication Number:
  US 20010044789A1
• Time in Patent Office: Days
• US Class Current: 706/14
• CPC Class Codes: G06N 3/061 using biological neurons, e.g. biological neurons connected to an integrated circuit

Abstract

A Neurointerface is a trainable filter based on neural networks that serves as a coupler between a human operator and a nonlinear system or plant that is to be controlled or directed. The purpose of the coupler is to ease the task of the human controller.

The Neurointerface can be adapted to be an inverse or an approximate inverse of the plant. The Neurointerface can be adapted so that when it is cascaded with the plant, the overall plant response closely approximates the human command input. In this way, it is easy for the human operator to direct the response of the plant.

A Neurointerface and a plant disturbance canceller have been applied to the steering system of a truck and trailer(s). The Neurointerface is used only while the truck is backing. Backing a truck with two or more trailers is essentially impossible for a professional truck driver, but is easily done by an unskilled driver when using the Neurointerface.

Neurointerface designs are presented for human control of construction cranes and multi-jointed robot arms. The same principles can be applied to ease human control of other complex machines such as aircraft, helicopters, heavy earth moving equipment, and so forth. Obstacle avoidance can also be done with Neurointerface control.

Claims

1. A Neurointerface for receiving input signals from an operator and controlling a complex system or machine comprising:

an adaptive filter with adjustable weights connected to signal delaying elements, said adjustable weights serving as variable multipliers of their respective signals,

summation means for combining said respective signals, nonlinear devices for processing the summed signals, and

means for combining the nonlinearly processed summed signals to generate the output signals of the Neurointerface, said output signals being used to control or direct a complex system or machine.

2. The Neurointerface of

claim 1

wherein said signal delaying elements receive inputs from both the input signals and the output signals of said Neurointerface.

3. The Neurointerface of

claim 1

wherein said input signals from a human operator consist of a plurality of individual input signals and wherein said output signals consist of a plurality of individual output signals.

4. The Neurointerface of

claim 2

wherein said input signals consist of a plurality of individual input signals and wherein said output signals consist of a plurality of individual output signals.

5. The Neurointerface of

claim 1

including a cascade of Neurointerfaces wherein said adjustable weights are determined by an automatic optimization process so that the overall response of the cascade of said Neurointerface and the system to be controlled closely approximates the human command input to said Neurointerface.

6. The Neurointerface of

claim 1

including a cascade of Neurotransmitters wherein said adjustable weights are determined by an automatic optimization process so that the overall response to the human command input to the cascade of said Neurointerface and the system to be controlled closely approximates the response of a reference model to said human command input.

8. A cascade of a Neurointerface as in

claim 1

and a disturbed system to be controlled, wherein said system is connected to an adaptive disturbance canceller.

9. The cascade of Neurointerface as in

claim 8

wherein said adaptive disturbance canceller includes weights which are determined by an automatic optimization process to minimize disturbance of said disturbed system.

10. A Neurointerface as in

claim 1

for steering, while backing only, the front wheels of a truck with a single trailer, wherein the input signal to said Neurointerface is a humanly generated command input that determines the radius of curvature of the trajectory of the truck and trailer.

11. A Neurointerface as in

claim 1

for steering, while backing only, the front wheels of a truck with a plurality of trailers, wherein the input signal to said Neurointerface is a humanly generated command input that determines the radius of curvature of the trajectory of the truck and trailers.

12. The Neurointerface and truck with trailer of

claim 10

, wherein state variable feedback is used to stabilize the backing truck and trailer, the state variable being the angle θ

₂ between the truck and the trailer.

13. The Neurointerface and truck with trailers of

claim 11

, wherein state variable feedback is used to stabilize the backing truck and trailers, the state variables being the angle θ

₂ between the truck and the first trailer, the angle θ

₃ between the first and second trailer, θ

₄ the angle between the second and third trailer, and so forth.

14. A Neurointerface as in

claim 5

for human control of a backing truck and trailers, wherein said Neurointerface is automatically optimized to minimize the mean square error between the humanly applied command input as filtered by a reference model and the angle between the last and the next to last trailers.

15. A truck with a single trailer steered while backing by a human providing command inputs to a Neurointerface as in

claim 1

, wherein said truck with a single trailer is connected to an adaptive disturbance canceller, said adaptive disturbance canceller optimized to minimize disturbance effects in said truck with a single trailer.

16. A truck with a plurality of trailers steered while backing by a human providing command inputs to a Neurointerface as in

claim 1

, wherein said truck with a plurality of trailers is connected to an adaptive disturbance canceller, said adaptive disturbance canceller optimized to minimize disturbance effects in said truck with a plurality of trailers.

17. A Neurointerface as in

claim 2

for controlling boom angle, position of trolley on said boom, and length of support cable of a construction crane, wherein the input signals to said Neurointerface are command inputs of a human operator provided to determine the position in three-dimensional space of the load supported by said support cable.

18. The Neurointerface of

claim 17

, wherein said humanly-generated command inputs are applied to said Neurointerface by means of a three-dimensional joystick.

19. The Neurointerface and construction crane of

claim 17

, wherein state variable feedback is used to stabilize said construction crane, the state variables being θ

₂, the angle between the load support cable and the boom, and its time derivative

20. A Neurointerface as in

claim 1

for human control of a construction crane, wherein said Neurointerface is optimized to minimize the mean-square error between the three-dimensional operator-applied command input as filtered by a reference model and the corresponding three components of velocity of the load.

21. A construction crane with a cable-supported load controlled by a human operator providing command inputs to a Neurointerface as in

claim 1

, wherein said construction crane is connected to an adaptive disturbance canceller, said adaptive disturbance canceller optimized to minimize disturbance effects in said construction crane.

22. A Neurointerface as in

claim 1

for controlling the joint torques of a robot arm, wherein the input signals to said Neurointerface are humanly-generated command inputs that determine the position of the effector of said robot arm in three-dimensional space.

24. A Neurointerface as in

claim 1

for human control of a robot arm, wherein said Neurointerface is optimized to minimize the mean square error between the three-dimensional humanly-applied command inputs as filtered by a reference model and the corresponding three components of position of the effector of said robot arm.

25. A robot arm controlled by a human operator providing command inputs to a Neurointerface as in

claim 1

, wherein said robot arm is connected to an adaptive disturbance canceller, said adaptive disturbance canceller optimized to minimize disturbance effects in said robot arm.

31. The Neurointerface of

claim 3

, wherein said Neurointerface is connected to an obstacle avoidance system, said avoidance system comprised of proximity sensors, a sensor signal processor, a summer that combines the output of said sensor signal processor with said input signals from a human operator.

32. The Neurointerface of

claim 10

, wherein said Neurointerface is connected to an obstacle avoidance system, said avoidance system comprised of proximity sensors attached to the truck and trailer, a sensor signal processor, a summer that combines the output of said sensor signal processor with said humanly generated command input.

33. The Neurointerface of

claim 11

, wherein said Neurointerface is connected to an obstacle avoidance system, said avoidance system comprised of proximity sensors attached to the truck and trailers, a sensor signal processor, a summer that combines the output of said sensor signal processor with said humanly generated command input.

34. The Neurointerface of

claim 22

, wherein said Neurointerface is connected to an obstacle avoidance system, said avoidance system comprised of proximity sensors attached to the links of the robot arm and to the effector, a sensor signal processor, a summer that combines the outputs of said sensor signal processor with said humanly-generated command inputs.

35. The Neurointerface of

claim 2

, wherein said Neurointerface serves as a coupling between a human operator and a multi-link robot arm, said Neurointerface receiving a humanly-generated command input signal, the outputs of said Neurointerface controlling torques applied to the joints of the robot arm, said robot arm having links with attached proximity sensors, said sensors providing inputs to an obstacle avoidance system for the robot arm.

36. The Neurointerface of

claim 2

, wherein said Neurointerface serves as a coupling between a human operator and a multi-link robot arm, said Neurointerface receiving a humanly-generated command input signal, the outputs of said Neurointerface controlling torques applied to the joints of the robot arm, said robot arm having links with attached pressure sensors, said sensors providing inputs to a system that minimizes the maximum pressure on the robot arm.

7. A cascade of a Neurointerface and an unstable system to be controlled, wherein said system is stabilized by conventional feedback.

23. The Neurointerface and robot arm of

claim 23

, wherein state variable feedback is used to stabilize said robot arm, the state variables being the joint angles of said robot arm and their respective time derivatives.

26. A Neurointerface containing a neural network, said Neurointerface receiving command input signals from a human operator and serving as a coupling between the human operator and the plant, system, or complex machine to be directed or controlled, said Neurointerface also receiving input signals which are the state variable signals of said plant to be controlled, said command input signals and said state variable signals providing inputs to said neural network, said inputs applied to adjustable weights which serve as variable multipliers for said inputs, summation means for combining the weighted signals, nonlinear devices for processing the summed signals, means for combining the nonlinearly processed summed signals to generate the output signals of the Neurointerface, said output signals being used to control or direct said plant or said complex machine.

27. The Neurointerface of

claim 26

, wherein said nonlinearly processed summed signals are further weighted with variable weights, summed, nonlinearly processed, and finally combined to generate the Neurointerface outputs.

28. The Neurointerface of

claim 27

, wherein said Neurointerface is trained to provide input signals to the plant or complex machine to be controlled so that the differences between the plant output signals and the corresponding output signals of a reference model are minimized in the mean square sense, when the command input is applied to both said Neurointerface and said reference model.

29. The Neurointerface of

claim 28

, wherein said Neurointerface is also trained to stabilize said plant.

30. The Neurointerface of

claim 29

, wherein said Neurointerface is also trained to provide disturbance canceling for said plant.