Real-time human collaboration monitoring and intervention

• US 7,751,878 B1
• Filed: 11/10/2005
• Issued: 07/06/2010
• Est. Priority Date: 11/10/2004
• Status: Expired due to Fees

• Current Assignee: National Technology & Engineering Solutions Of Sandia LLC (Honeywell International Inc.)
• Original Assignee: Sandia Corporation (Martin Marietta Corporation)
• Inventors: Warner, David J., Jones, Wendell B., Doser, Adele B., Yonas, Gerold, Merkle, Peter B., Johnson, Curtis M.
• Primary Examiner(s): Mallari; Patricia C
• Application Number:
  US11/271,214
• Time in Patent Office: 1,699 Days
• Field of Search: 600/545
• US Class Current: 600/545
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: A61B 5/726 using Wavelet transforms
• CPC Class Codes: A61B 5/7267 involving training the classification device

Abstract

A method of and apparatus for monitoring and intervening in, in real time, a collaboration between a plurality of subjects comprising measuring indicia of physiological and cognitive states of each of the plurality of subjects, communicating the indicia to a monitoring computer system, with the monitoring computer system, comparing the indicia with one or more models of previous collaborative performance of one or more of the plurality of subjects, and with the monitoring computer system, employing the results of the comparison to communicate commands or suggestions to one or more of the plurality of subjects.

Claims

1. A method of monitoring and intervening in, in real time, a collaboration between a plurality of subjects, the method comprising the steps of:

measuring one or more indications of physiological and cognitive states of each of the plurality of subjects;

communicating one or more of the indications to a monitoring computer system;

with the monitoring computer system, comparing the indications with one or more models of previous collaborative performance of one or more of the plurality of subjects, wherein one or more of the models in the comparing step comprises a self-organizing map of multiple subjects of the plurality of subjects; and

with the monitoring computer system, employing the results of the comparison to communicate commands or suggestions to one or more of the plurality of subjects.

2. The method of claim 1 wherein the measuring step comprises measuring using one or more of the group consisting of:

electroencephalography devices;

electro-oculography devices;

electrocardiography devices;

respiration rate devices;

galvanic skin response devices;

blood pulse volume devices;

electromechanical motion sensors; and

accelerometry devices.

3. The method of claim 1 wherein the comparing step comprises applying over time a sliding window Fourier transform to one or more of the indications.

4. The method of claim 1 wherein the self-organizing map is a self-organizing map of a single subject of the plurality of subjects.

5. The method of claim 1 wherein the self-organizing map is a self-organizing map of all of the plurality of subjects.

6. The method of claim 1 wherein the employing step comprises automatically communicating commands or suggestions.

7. The method of claim 1 wherein the employing step comprises displaying one or more of the indications or one or more results of the comparison to a monitoring human who initiates commands or suggestions.

8. The method of claim 7 wherein the communicating step comprises communicating visual images of one or more of the plurality of subjects and the method additionally comprises the step of displaying the visual images to the monitoring human.

9. An apparatus for monitoring and intervening in, in real time, a collaboration between a plurality of subjects, said apparatus comprising:

means for measuring one or more indications of physiological and cognitive states of each of the plurality of subjects;

means for communicating one or more of the indications to a monitoring computer system;

a monitoring computer system that compares the indications with one or more models of previous collaborative performance of one or more of the plurality of subjects and employs the results of the comparison to communicate commands or suggestions to one or more of the plurality of subjects, wherein one or more of the models comprises a self-organizing map of multiple subjects of the plurality of subjects.

10. The apparatus of claim 9 wherein said means for measuring comprises one or more of the group consisting of:

electroencephalography devices;

electro-oculography devices;

electrocardiography devices;

respiration rate devices;

galvanic skin response devices;

blood pulse volume devices;

electromechanical motion sensors; and

accelerometry devices.

11. The apparatus of claim 9 wherein said monitoring computer system compares by applying over time a sliding window Fourier transform to one or more of the indications.

12. The apparatus of claim 9 wherein the self-organizing map is a self-organizing map of a single subject of the plurality of subjects.

13. The apparatus of claim 9 wherein said monitoring computer system automatically communicates commands or suggestions.

14. The apparatus of claim 9 wherein said monitoring computer system displays one or more of the indications or one or more results of the comparison to a monitoring human.

15. The apparatus of claim 14 wherein said means for communicating communicates visual images of one or more of the plurality of subjects and the apparatus additionally comprises one or more displays for displaying the visual images to the monitoring human.

16. A method of monitoring and intervening in, in real time, a collaboration between a plurality of subjects, the method comprising the steps of:

measuring one or more indications of physiological and cognitive states of each of the plurality of subjects;

communicating one or more of the indications to a monitoring computer system;

with the monitoring computer system, comparing the indications with one or more models of previous collaborative performance of one or more of the plurality of subjects; and

with the monitoring computer system, employing the results of the comparison to communicate commands or suggestions to one or more of the plurality of subjects, by displaying one or more of the indications or one or more results of the comparison to a monitoring human who initiates commands or suggestions.

17. The method of claim 16 wherein the comparing step comprises applying over time a sliding window Fourier transform to one or more of the indications.

18. The method of claim 16 wherein the communicating step comprises communicating visual images of one or more of the plurality of subjects and the method additionally comprises the step of displaying the visual images to the monitoring human.

19. An apparatus for monitoring and intervening in, in real time, a collaboration between a plurality of subjects, said apparatus comprising:

means for measuring one or more indications of physiological and cognitive states of each of the plurality of subjects;

means for communicating one or more of the indications to a monitoring computer system, wherein said means for communicating communicates visual images of one or more of the plurality of subjects and the apparatus additionally comprises one or more displays for displaying the visual images to a monitoring human;

a monitoring computer system which displays one or more of the indications or one or more results of the comparison to the monitoring human and which monitoring computer system compares the indications with one or more models of previous collaborative performance of one or more of the plurality of subjects and employs the results of the comparison to communicate commands or suggestions to one or more of the plurality of subjects.

20. The apparatus of claim 19 wherein the self-organizing map is a self-organizing map of a single subject of the plurality of subjects.