Optimal rapid serial visual presentation (RSVP) spacing and fusion for electroencephalography (EEG)-based brain computer interface (BCI)

• US 9,489,596 B1
• Filed: 12/09/2014
• Issued: 11/08/2016
• Est. Priority Date: 12/21/2010
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Keegan, Matthew S., Khosla, Deepak, Martin, Kevin R.
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US14/565,094
• Time in Patent Office: 700 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/378 Visual stimuli
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06V 10/7784 based on feedback from supervisors
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Described is system for optimizing rapid serial visual presentation (RSVP) spacing and fusion. The system receives a sequence of a plurality of rapid serial visual presentation (RSVP) image chips. The plurality of RSVP image chips are generated from an image via a pre-processing step and have a high probability of containing a target of interest. The system alters the order of the sequence of the plurality of RSVP image chips to increase the probability of detection of a true target of interest when presented to a human subject.

Claims

1. A system for rapid serial visual presentation (RSVP) spacing and fusion, the system comprising:

one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of;

receiving a sequence of a plurality of rapid serial visual presentation (RSVP) image chips having an order, wherein the plurality of RSVP image chips are generated from an image via a pre-processing step and have a high probability of containing a target of interest; and

altering the order of the sequence of the plurality of RSVP image chips to increase the probability of detection of a true target of interest when presented to a human subject.

2. The system as set forth in claim 1, wherein if a posterior probability of a RSVP image chip containing a target of interest is known, then the one or more processors further perform an operation of removing the RSVP image chip from the sequence of the plurality of RSVP image chips if the posterior probability of the RSVP chip is equal to or above a predetermined threshold.

3. The system as set forth in claim 2, wherein the predetermined threshold is set according to a permissible false positive rate.

4. The system as set forth in claim 1, wherein the one or more processors further perform operations of:

performing N staggered raster scans of the image to generate N sequences of RSVP image chips; and

concatenating the N sequences of RSVP image chips.

5. The system as set forth in claim 1, wherein the one or more processors further perform operations of:

removing a RSVP image chip from the sequence of the plurality of RSVP image chips if a posterior probability of the RSVP chip is equal to or above a predetermined threshold, thereby generating a plurality of remaining RSVP image chips;

performing N staggered raster scans of the image to generate N sequences of RSVP image chips from the remaining RSVP image chips; and

concatenating the N sequences of RSVP image chips.

6. The system as set forth in claim 1, wherein if a likelihood of the presence of a target of interest in one or more RSVP image chips is above a predetermined percentile, then the one or more processors perform an operation of spacing the one or more RSVP image chips that are above the predetermined percentile between a plurality of RSVP image chips that are below the predetermined percentile.

7. A computer-implemented method for rapid serial visual presentation (RSVP) spacing and fusion, comprising:

an act of causing one or more processors to execute instructions stored on a non-transitory memory such that upon execution, the one or more processors perform operations of;

receiving a sequence of a plurality of rapid serial visual presentation (RSVP) image chips having an order, wherein the plurality of RSVP image chips are generated from an image via a pre-processing step and have a high probability of containing a target of interest; and

altering the order of the sequence of the plurality of RSVP image chips to increase the probability of detection of a true target of interest when presented to a human subject.

8. The method as set forth in claim 7, wherein if a posterior probability of a RSVP image chip containing a target of interest is known, then the one or more processors further perform an operation of removing the RSVP image chip from the sequence of the plurality of RSVP image chips if the posterior probability of the RSVP chip is equal to or above a predetermined threshold.

9. The method as set forth in claim 8, wherein the predetermined threshold is set according to a permissible false positive rate.

10. The method as set forth in claim 7, wherein the one or more processors further perform operations of:

performing N staggered raster scans of the image to generate N sequences of RSVP image chips; and

concatenating the N sequences of RSVP image chips.

11. The method as set forth in claim 7, wherein the one or more processors further perform operations of:

removing a RSVP image chip from the sequence of the plurality of RSVP image chips if a posterior probability of the RSVP chip is equal to or above a predetermined threshold, thereby generating a plurality of remaining RSVP image chips;

performing N staggered raster scans of the image to generate N sequences of RSVP image chips from the remaining RSVP image chips; and

concatenating the N sequences of RSVP image chips.

12. The method as set forth in claim 7, wherein if a likelihood of the presence of a target of interest in one or more RSVP image chips is above a predetermined percentile, then the one or more processors perform an operation of spacing the one or more RSVP image chips that are above the predetermined percentile between a plurality of RSVP image chips that are below the predetermined percentile.

13. A computer program product for rapid serial visual presentation (RSVP) spacing and fusion, the computer program product comprising computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

receiving a sequence of a plurality of rapid serial visual presentation (RSVP) image chips having an order, wherein the plurality of RSVP image chips are generated from an image via a pre-processing step and have a high probability of containing a target of interest; and

altering the order of the sequence of the plurality of RSVP image chips to increase the probability of detection of a true target of interest when presented to a human subject.

14. The computer program product as set forth in claim 13, wherein if a posterior probability of a RSVP image chip containing a target of interest is known, then the one or more processors further perform an operation of removing the RSVP image chip from the sequence of the plurality of RSVP image chips if the posterior probability of the RSVP chip is equal to or above a predetermined threshold.

15. The computer program product as set forth in claim 14, wherein the predetermined threshold is set according to a permissible false positive rate.

16. The computer program product as set forth in claim 13, further comprising instructions for causing the processor to perform operations of:

performing N staggered raster scans of the image to generate N sequences of RSVP image chips; and

concatenating the N sequences of RSVP image chips.

17. The computer program product as set forth in claim 13, further comprising instructions for causing the processor to perform operations of:

removing a RSVP image chip from the sequence of the plurality of RSVP image chips if a posterior probability of the RSVP chip is equal to or above a predetermined threshold, thereby generating a plurality of remaining RSVP image chips;

performing N staggered raster scans of the image to generate N sequences of RSVP image chips from the remaining RSVP image chips; and

concatenating the N sequences of RSVP image chips.

18. The computer program product as set forth in claim 13, wherein if a likelihood of the presence of a target of interest in one or more RSVP image chips is above a predetermined percentile, then the one or more processors perform an operation of spacing the one or more RSVP image chips that are above the predetermined percentile between a plurality of RSVP image chips that are below the predetermined percentile.

19. A rapid serial visual presentation (RSVP) system, the system comprising:

one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of;

receiving a sequence of a plurality of rapid serial visual presentation (RSVP) image chips having an order, wherein the plurality of RSVP image chips are generated from an image via a pre-processing step and have a high probability of containing a target of interest; and

altering the order of the sequence of the plurality of RSVP image chips to increase the probability of detection of a true target of interest when presented to a human subject;

a visual display for presenting the plurality of RSVP image chips to the human subject; and

at least one electoencephalography (EEG) sensor for measuring brain activity of the human subject.