Bias of physical controllers in a system

• US 10,065,658 B2
• Filed: 05/23/2016
• Issued: 09/04/2018
• Est. Priority Date: 05/23/2016
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: International Business Machines Corporation
• Inventors: Deligianni, Hariklia, Pickover, Clifford A., Sailer, Anca
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/161,649
• Publication Number:
  US 20170334456A1
• Time in Patent Office: 834 Days
• Field of Search: 701 36- 49
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/14517 for sweat
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4088 Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/7475 User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 2040/0863 due to erroneous selection or response of the driver
• CPC Class Codes: B60W 2050/0071 Controller overrides driver automatically
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/26 Incapacity
• CPC Class Codes: B60W 2756/10 Involving external transmission of data to or from the vehicle
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 50/087 where the control system corrects or modifies a request from the driver
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G05D 1/0061 : for transition from automatic pilot to manual pilot and vice versa

Abstract

An embodiment of the invention provides a method to control a mechanical system based on the cognitive state of a user, where a first action is performed at an input device that is associated with the user. The cognitive state of the user is detected at the input device; and, a change to the first action is determined based on the cognitive state of the user. A controlled action is performed based on the recommended change. A system can include an input device associated with user, where a first action is performed at the input device. A processor connected to the input device detects the cognitive state of the user at the input device and determines a change to the first action based on the cognitive state of the user. A controller connected to the processor performs a controlled action based on the recommended change.

Claims

1. A method to control a mechanical system based on the cognitive state of at least one user, said method comprising:

performing a first action at an input device, wherein the input device is associated with at least one user;

detecting the cognitive state of the at least one user at the input device by a first processing unit;

determining a change to the first action based on the cognitive state of the at least one user by a second processing unit; and

performing a controlled action by a controller based on the recommended change, the controlled action selected from the group consisting of delaying a command from the at least one user, sending an alert to a third party indicating the detected cognitive state of the at least one user, providing a recommendation to the at least one user, and activation of a detent in the physical controller.

2. The method according to claim 1, wherein the cognitive state of the at least one user is selected from the group consisting of above a threshold level of anxiety, above a threshold level of level of elation, above a threshold level hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a disease cognitive state.

3. The method according to claim 1, wherein the cognitive state of the at least one user includes above a threshold level of anxiety, above a threshold level of level of elation, above a threshold level hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a disease cognitive state.

4. The method according to claim 1, wherein said controlled action includes activating an autopilot system in the mechanical system when the processor determines that the cognitive state of the at least one user is selected from the group consisting of above a threshold level of anxiety, above a threshold level of elation, above a threshold level of hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a neurological disease cognitive state.

5. The method according to claim 1, wherein said controlled action is performed when the processor determines that the cognitive state of the at least one user is selected from the group consisting of above a threshold level of anxiety, above a threshold level of elation, above a threshold level of hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a neurological disease cognitive state.

6. The method according to claim 1, wherein said controlled action is performed when the processor determines that the cognitive state of the at least one user is above a threshold level of anxiety, above a threshold level of elation, above a threshold level of hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a neurological disease cognitive state.

7. The method according to claim 1, wherein said controlled action includes increasing a sensitivity of a physical controller of the mechanical system when the processor determines that the cognitive state of the at least one user is selected from the group consisting of above a threshold level of anxiety, above a threshold level of elation, above a threshold level of hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a neurological disease cognitive state.

8. The method according to claim 1, wherein said controlled action includes decreasing a sensitivity of a physical controller of the mechanical system when the processor determines that the cognitive state of the at least one user is selected from the group consisting of above a threshold level of anxiety, above a threshold level of elation, above a threshold level of hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a neurological disease cognitive state.

9. The method according to claim 1, wherein said controlled includes changing a type of vibration feedback in the physical controller of the mechanical system when the processor determines that the cognitive state of the at least one user is selected from the group consisting of above a threshold level of anxiety, above a threshold level of elation, above a threshold level of hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a neurological disease cognitive state.

10. The method according to claim 1, wherein said controlled action is selected from the group consisting of decreasing an amount of vibration feedback and changing a type of vibration feedback in the physical controller of the mechanical system when the processor determines that the cognitive state of the at least one user is selected from the group consisting of above a threshold level of anxiety, above a threshold level of elation, above a threshold level of hesitation, above a threshold level of distraction, above a threshold level of fatigue, above a threshold level of frustration, above a threshold level of nervousness, above a threshold level of panic, above a threshold level of anger, below a threshold level of expertise, and a neurological disease cognitive state.

11. The method according to claim 1, wherein said detecting of the cognitive state of the at least one user includes detecting a level of fatigue of the at least one user based on a factor selected from the group consisting of a calendar of the at least one user, a facial expression of the at least one user, and head nods detected by the input device.

12. The method according to claim 1, wherein the mechanical system includes an automobile, the input device includes a camera on the automobile, and wherein said detecting of the cognitive state of the at least one user includes detecting that the at least one user is at least one of distracted and fatigued when lane deviation is detected in the input.

13. The method according to claim 1, wherein said detecting of the cognitive state of the at least one user includes detecting a level of hesitation of the at least one user by a timing analysis, said timing analysis including:

determining an average response time for a task by the at least one user;

determining a current response time for the task by the at least one user;

determining that the at least one user has a high level of hesitation when the current response time for the task by the at least one user exceeds the average response time for a task by the at least one user by a threshold amount of time.

14. The method according to claim 1, wherein said detecting of the cognitive state of the at least one user includes detecting a level of anxiety of the at least one user by detecting hand tremors of the at least one user with a sensor in a physical controller of the mechanical system.

15. The method according to claim 1, wherein said detecting of the cognitive state of the at least one user includes detecting the cognitive state of the at least one user from information from a calendar of the at least one user and a profile of the at least one user, the profile of the at least one user including a level of expertise of the at least one user, a disease cognitive state of the at least one user, and a list of medications of the at least one user.

16. A system to control a mechanical system based on the cognitive state of a user, said system comprising:

an input device associated with at least one user, wherein a first action is performed at said input device;

a processor connected to said input device, said processor detects the cognitive state of the at least one user at said input device and determines a change to a first action based on the cognitive state of the user; and

a controller connected to said processor, said controller performs a controlled action based on the recommended change, the controlled action selected from the group consisting of delaying a command from the at least one user, sending an alert to a third party indicating the detected cognitive state of the at least one user, providing a recommendation to the at least one user, and activation of a detent in the input device.

17. The system according to claim 16, wherein said processor detects the cognitive state of the at least one user from a factor selected from the group consisting of a facial expression of the at least one user, information from a calendar of the at least one user, a profile of the at least one user, and analysis of audible input, the analysis of audible input including detected keywords, tone of spoken words, frequency of the spoken words, and pitch of the spoken words.

18. An input device associated with at least one user, said input device comprising:

a user interface, wherein a first action is performed at said user interface;

a first processing unit connected to said user interface, said first processing unit detects the cognitive state of the at least one user at said user interface; and

a second processing unit connected to said first processing unit, said second processing unit determines a change to a first action based on the cognitive state of the user, the change to the first action selected from the group consisting of delaying a command from the at least one user, sending an alert to a third party indicating the detected cognitive state of the at least one user, providing a recommendation to the at least one user, and activation of a detent in the user interface.