Coordinating System Responses Based on an Operator's Cognitive Response to a Relevant Stimulus and to the Position of the Stimulus in the Operator's Field of View

• US 20100185113A1
• Filed: 01/21/2009
• Published: 07/22/2010
• Est. Priority Date: 01/21/2009
• Status: Abandoned Application

• Current Assignee: Teledyne Scientific & Imaging, LLC (Teledyne Technologies Incorporated)
• Original Assignee: Teledyne Scientific & Imaging, LLC (Teledyne Technologies Incorporated)
• Inventors: Peot, Mark A., Aguilar, Mario
• Application Number:
  US12/356,681
• Publication Number:
  US 20100185113A1
• Time in Patent Office: Days
• US Class Current: 600/544
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/378 Visual stimuli
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Neurophysiological responses such as EEG signals of brainwave activity, dilation signals of pupillary response, dwell time signals of eye movement and imaging signals of vascular response are monitored as a correlate of the operator'"'"'s cognitive response. These responses are processed to determine if there is a significant cognitive response to a stimulus (visual or non-visual) to generate a positive cue. The operator'"'"'s eye movement is monitored to determine when the operator fixates on the stimulus and the position of the stimulus in the operator'"'"'s field-of view (FOV). The positive cue and position of the stimulus, and typically the time-stamp of the cue are output triggering a system response. The temporal sequence of cues and stimulus position may be processed to reinforce or reject the cue or refine the stimulus position.

Claims

1. A method of cueing a system response, comprising:

monitoring neurophysiological responses of an operator subjected to stimuli;

processing the neurophysiological responses to determine if the operator had a significant cognitive response to a stimulus to generate a positive cue;

monitoring the operator'"'"'s eye movement to determine when the operator fixates on the stimulus;

determining the position of the stimulus in the operator'"'"'s field-of view (FOV);

outputting the positive cue and the position of the stimulus; and

triggering a system response to the positive cue and the position of the stimulus.

2. The method of claim 1, wherein determining the stimulus position comprises:

determining the orientation of the stimulus with respect to the operator; and

determining the range of the stimulus with respect to the operator.

3. The method of claim 2, further comprising:

determining the geo-location of the operator; and

determining a geo-location of the stimulus from the geo-location of the operator and the position of the stimulus.

4. The method of claim 1, wherein the stimulus comprises a non-visual stimulus.

5. The method of claim 1, wherein the stimulus comprises a visual stimulus.

6. The method of claim 5, wherein the operator fixates on the visual stimulus prior to the operator'"'"'s cognitive response to the stimulus, further comprising:

synchronizing the neurophysiological responses to the operator'"'"'s fixation on the stimulus for processing; and

processing the synchronized neurophysiological responses.

7. The method of claim 6, wherein at least one window is positioned relative to the onset of fixation and the neurophysiological responses within the window processed.

8. The method of claim 7, wherein one said window is positioned to capture neurophysiological responses both before and after the onset of fixation.

9. The method of claim 7, wherein the neurophysiological responses within said at least one window are processed by:

extracting features from the neurophysiological responses, said features based on the assumption that the at least one window has a specified position relative to the onset of fixation; and

presenting the extracted features to a computer-implemented classifier trained to detect patterns of the extracted features and to generate the positive cue indicative of the occurrence a significant brain response.

10. The method of claim 1, wherein the monitored neurophysiological responses include at least one of EEG signals of brainwave activity, dilation signals of pupillary response, dwell time signals of eye movement and imaging signals of vascular response.

11. The method of claim 10, wherein at least two of the EEG, dilation, dwell time and imaging signals are monitored and processed.

12. The method of claim 1, wherein the response is triggered in real-time with respect to the operator'"'"'s cognitive response to the stimulus.

13. The method of claim 1, further comprising:

wireless transmitting the output of the positive cue and position of the stimulus from the operator to a remote location where the system response is triggered.

14. The method of claim 13, wherein the system response is to aim a weapon at the position of the stimulus.

15. The method of claim 13, where the system response is to command multiple operators to turn toward the position of the stimulus in a show of force.

16. The method of claim 13, wherein a control system at the remote locations receives cues and position data from multiple operators and synthesizes the data to trigger the system response.

17. The method of claim 1, wherein the positive cue and position of the stimulus are provided as one of a plurality of inputs to classifier, the output of the classifier triggering the system response.

18. The method of claim 1, wherein the triggered system response is to record the positive cue and position of the stimulus and any action taken in response to that stimulus.

19. The method of claim 1, wherein a time sequence of cues and fixation measurements are output, further comprising processing the temporal cues and fixation measurements to reinforce or reject the positive cue and refine the stimulus position.

20. A system comprising:

a head-mounted system for an operator, said system comprising;

at least one sensor monitoring neurophysiological responses of an operator subjected to stimuli;

a classifier processing the neurophysiological responses to determine if the operator had a significant cognitive response to a stimulus to generate a positive cue;

an eye-tracking device monitoring the operator'"'"'s eye movement to determine when the operator fixates on a stimulus;

means for determining the position of the stimulus in the operator'"'"'s field-of view (FOV); and

a wireless data link transmitting the positive cue and the position of the stimulus; and

a remote system receiving the cue and position and triggering a system response to the positive cue and the position of the stimulus.

21. The system of claim 20, wherein said at least one sensor monitors neurophysiological responses including at least one of EEG signals of brainwave activity, dilation signals of pupillary response, dwell time signals of eye movement and imaging signals of vascular response.

22. The system of claim 20, wherein the remote system triggers a system response of aiming a weapon at the position of the stimulus or commanding multiple operators to turn toward the position of the stimulus.

23. The system of claim 22, wherein the remote system receives cues and position data from multiple operators and synthesizes the data to trigger the system response.

24. The system of claim 20, wherein the operator fixates on the visual stimulus prior to the operator'"'"'s cognitive response to the stimulus, further comprising a data pre-processing system that windows the monitored neurophysiological responses into at least one time window synchronized to the operator'"'"'s fixation on the stimulus.