Operation assistance system and method

• US 7,386,371 B2
• Filed: 12/16/2004
• Issued: 06/10/2008
• Est. Priority Date: 12/16/2003
• Status: Expired due to Fees

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Kuge, Nobuyuki, Yamamura, Tomohiro
• Primary Examiner(s): ARTHUR JEANGLAUD, GERTRUDE
• Application Number:
  US11/012,208
• Publication Number:
  US 20050131590A1
• Time in Patent Office: 1,272 Days
• Field of Search: 701/1, 701/23, 701/28, 701/41, 701/42, 701/88, 701/93, 701/117, 701/29, 180/167, 180/168, 180/169
• US Class Current: 701/1
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal

Abstract

A method and system for providing automatic assistance in operating a machine. A current operation performed by an operator of the machine is detected, and data related to an estimated intention of the operator is generated based on the detected operation. A state of the estimated intention of the operator is then determined. The state indicates, for example, the reliability or strength of the estimated intention. The operation of the machine is altered based on the estimated intention and the state of the estimated intention.

Claims

1. An operation assistance method comprising the steps of:

detecting an operation performed by a real operator of a machine, wherein the operation may correspond to multiple possible intentions;

generating data related to one or more imaginary operators based on the detected operation of the real operator; and

determining an estimated intention of the real operator based on the generated data related to the one or more imaginary operators.

2. The method of claim 1 further comprising the steps of:

generating data related to an estimated period of time that the real operator retains the estimated intention; and

generating a state of the estimated intention of the real operator based on the estimated period of time, wherein the state of the estimated intention indicates a degree of reliability of the estimated intention of the real operator.

3. The method of claim 2, wherein the step of generating data related to an estimated period of time comprises the steps of:

generating data related to an estimated intention of the real operator at a specific point in time and at least one past estimated intention of the real operator before the specific point in time; and

generating data related to the estimated period of time based on the data related to the estimated intention of the real operator at the specific point in time and the at least one past estimated intention of the real operator before the specific point in time.

4. The method of claim 1, wherein:

the machine is a vehicle; and

the real operator is a driver of the vehicle.

5. The method of claim 1, wherein:

the detecting step detects an operation of the real operator at each one of different points in time; and

the step of generating data related to the one or more imaginary operators comprises the steps of;

providing data related to a plurality of imaginary operators, each of the plurality of imaginary operators associated with a sequence of intentions corresponding to the different points in time, wherein each of the sequence of intentions is associated with an operation;

for each imaginary operator, calculating a likelihood value based on partial likelihood values of each imaginary operator corresponding to the different points in time, wherein each of the partial likelihood values is respectively associated with each of the sequence of intentions and the respective operation at each one of the different points in time, and is generated based on the respective detected operation of the real operator at each one of the different points in time and the respective operation of the respective one of the plurality of imaginary operators at each one of the different points in time;

selecting one of the plurality of imaginary operators based on the likelihood value of each one of the imaginary operators; and

generating the estimated intention of the real operator based on a chosen intention of the selected one of the imaginary operators.

6. The method of claim 5, wherein the step of generating data related to one or more imaginary drivers comprises the steps of:

determining the point in time corresponding to the chosen intention;

for the selected one of the plurality of imaginary operators, determining the point in time corresponding to the last intention in the sequence of intentions that is different from the chosen intention; and

determining a period of time that the chosen intention is retained by the selected one of the plurality of imaginary operators, based on the point in time corresponding to the chosen intention and the point in time corresponding to the last intention in the sequence of intentions that is different from the chosen intention; and

generating the state of the estimated intention based on the determined period of time.

7. The method of claim 4 further comprising the steps of:

generating a state of the estimated intention based on the estimated period of time, wherein the state of the estimated intention indicates a degree of reliability of the estimated intention;

calculating a risk potential associated with the vehicle;

calculating a reaction force based on the calculated risk potential, the estimated intention and the state of the estimated intention; and

applying the reaction force to a vehicle control device of the vehicle.

8. The method of claim 7, wherein the vehicle control device is an acceleration pedal or a steering wheel.

9. The method of claim 7, wherein the step of calculating a reaction force comprises the steps of:

modifying the risk potential based on the state of the estimated intention; and

calculating the reaction force based on the modified risk potential and the estimated intention.

10. The method of claim 7, wherein the step of calculating a reaction force comprises the steps of:

calculating the reaction force based on the calculated risk potential and the estimated intention; and

modifying the calculated reaction force based on the state of the estimated intention.

11. The method of claim 1 further comprising the steps of:

generating a state of the estimated intention based on the estimated period of time, wherein the state of the estimated intention indicates a degree of reliability of the estimated intention; and

modifying the operation of the machine based on the state of the estimated intention of the real operator.

12. An operation assistance system comprising:

a first device configured to detect an operation performed by a real operator of a machine, wherein the operation may correspond to multiple possible intentions;

a second device configured to generate data related to one or more imaginary operators based on the detected operation of the real operator; and

a third device configured to determine an estimated intention of the real operator based on the generated data related to the one or more imaginary operators.

13. The method of claim 12, wherein the third device is configured to generate data related to an estimated period of time that the real operator retains the estimated intention, and to generate a state of the estimated intention based on the estimated period of time.

14. The method of claim 13, wherein the third device is configured to generate data related to an estimated intention of the real operator at a specific point in time and at least one past estimated intention of the real operator before the specific point in time, and to generate data related to the estimated period of time based on the data related to the estimated intention of the real operator at the specific point in time and the at least one past estimated intention of the real operator before the specific point in time.

15. The system of claim 12, wherein:

the machine is a vehicle; and

the real operator is a driver of the vehicle.

16. The system of claim 12, wherein:

the first device is configured to detect an operation of the real operator at each one of different points in time; and

the second device is configured to perform the steps of;

providing data related to a plurality of imaginary operators, each of the plurality of imaginary operators associated with a sequence of intentions corresponding to the different points in time, wherein each of the sequence of intentions is associated with an operation;

for each imaginary operator, calculating a likelihood value based on partial likelihood values of each imaginary operator corresponding to the different points in time, wherein each of the partial likelihood values is respectively associated with each of the sequence of intentions and the respective operation at each one of the different points in time, and is generated based on the respective detected operation of the real operator at each one of the different points in time and the respective operation of the respective one of the plurality of imaginary operators at each one of the different points in time;

selecting one of the plurality of imaginary operators based on the likelihood value of each one of the imaginary operators; and

generating the estimated intention of the real operator based on a chosen intention of the selected one of the imaginary operators.

17. The system of claim 16, wherein second device is configured to perform the steps of:

determining the point in time corresponding to the chosen intention;

for the selected one of the plurality of imaginary operators, determining the point in time corresponding to the last intention in the sequence of intentions that is different from the chosen intention; and

determining a period of time that the chosen intention is retained by the selected one of the plurality of imaginary operators, based on the point in time corresponding to the chosen intention and the point in time corresponding to the last intention in the sequence of intentions that is different from the chosen intention;

generating a state of the estimated intention based on the determined period of time.

18. The system of claim 15 further comprising:

a device configured to determine a state of the estimated intention based on the data related to one or more imaginary operators;

a risk calculation device configured to calculate a risk potential associated with the vehicle;

a reaction force calculation device configured to calculate a reaction force based on the calculated risk potential, the estimated intention and the state of the estimated intention; and

applying the reaction force to a vehicle control device of the vehicle.

19. The system of claim 18, wherein the vehicle control device is an acceleration pedal or a steering wheel.

20. An operation assistance system comprising:

means for detecting an operation performed by a real operator of a;

machine, wherein the operation may correspond to multiple possible intentions;

means for generating data related to one or more imaginary operators based on the detected operation; and

means for determining an estimated intention of the real operator based on the generated data related to one or more imaginary operators.

21. A machine-readable medium bearing instructions for assisting operation of a machine, the instructions, upon execution by a data processing system, causing the data processing system to perform the steps of:

receiving data related to an operation performed by a real operator of the machine, wherein the operation may correspond to multiple possible intentions;

generating data related to one or more imaginary operators based on the detected operation; and

determining an estimated intention of the real operator based on the generated data.

22. A vehicle comprising:

a first device configured to detect an operation performed by a driver of the vehicle, wherein the operation may correspond to multiple possible intentions;

a second device configured to generate data related to one or more imaginary drivers based on the detected operation, wherein each of the imaginary drivers is associated with a pre-assigned operation; and

a third device configured to determine an estimated intention of the driver based on the generated data related to the one or more imaginary drivers.

23. The vehicle of claim 22 further comprising a control device to determine a state of the estimated intention of the driver and control the operation of the vehicle based on the identified state of the estimated intention of the driver.

24. An operation assistance method comprising the steps of:

detecting an operation performed by an operator of a machine, wherein the operation may correspond to multiple possible intentions;

estimating an intention of the operator based on the detected operation;

generating data related to the estimated intention of the operator based on the detected operation; and

identifying a state of the estimated intention of the operator based on the generated data.

25. An operation assistance system comprising:

a first device configured to detect an operation performed by an operator of a machine, wherein the operation may correspond to multiple possible intentions;

a second device configured to estimate an intention of the operator based on the detected operation;

a third device configured to generate data related to the estimated intention of the operator based on the detected operation; and

a fourth device configured to identify a state of the estimated intention of the operator based on the generated data.

26. An operation assistance system comprising:

means for detecting an operation performed by an operator of a machine, wherein the operation may correspond to multiple possible intentions;

means for estimating an intention of the operator based on the detected operation;

means for generating data related to the estimated intention of the operator based on the detected operation; and

means for identifying a state of the estimated intention of the operator based on the generated data.

27. A computer readable storage medium having stored therein data representing instructions executable by a computer to assist operation of a machine, comprising:

instructions to detect an operation performed by an operator of a machine, wherein the operation may correspond to multiple possible intentions;

instructions to estimate an intention of the operator based on the detected operation;

instructions to generate data related to the estimated intention of the operator based on the detected operation; and

instructions to identify a state of the estimated intention of the operator based on the generated data.

28. A vehicle comprising:

a first device configured to detect an operation performed by an operator of the vehicle, wherein the operation may correspond to multiple possible intentions;

a second device configured to estimate an intention of the operator based on the detected operation;

a third device configured to generate data related to the estimated intention of the operator based on the detected operation; and

a fourth device configured to identify a state of the estimated intention of the operator based on the generated data.