CATEGORIZING PERCEPTUAL STIMULI BY DETECTING SUBCONCIOUS RESPONSES

• US 20090062679A1
• Filed: 08/27/2007
• Published: 03/05/2009
• Est. Priority Date: 08/27/2007
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Shenoy, Pradeep, Tan, Desney
• Application Number:
  US11/845,583
• Publication Number:
  US 20090062679A1
• Time in Patent Office: Days
• US Class Current: 600/544
• CPC Class Codes: A61B 5/316 Modalities, i.e. specific diagnostic methods
• CPC Class Codes: A61B 5/372 Analysis of electroencephalograms
• CPC Class Codes: A61B 5/378 Visual stimuli
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A perceptual stimulus categorization technique is presented which identifies the stimuli category of a perceptual stimulus that has been presented to a person whose brain activity is being monitored. This generally accomplished by first training a detection module to recognize the part of the brain activity generated in response to the presentation of a stimulus belonging to each of one or more stimuli categories using brain activity information. Once the detection module is trained, a subsequent instance of a stimulus belonging to a trained stimuli category being presented to the person is detected, and this detection is used to identify the trained stimuli category to which the presented stimulus belongs.

Claims

1. A computer-implemented process for identifying a stimuli category of a perceptual stimulus that has been presented to one or more people whose brain activity is being monitored, comprising using a computer to perform the following process actions for each person:

training a detection module to recognize the part of the brain activity generated in response to a presentation of a stimulus belonging to each of one or more categories of perceptual stimuli using the monitored brain activity;

once the detection module is trained, detecting a subsequent instance or instances of a stimulus belonging to a trained stimuli category being presented to the person based on the monitored brain activity;

identifying the trained stimuli category that the presented stimulus belongs to; and

designating that the presented stimulus belongs to the identified trained stimuli category.

2. A computer-implemented process for identifying a stimuli category of a perceptual stimulus that has been presented to one or more people whose brain activity is being monitored, comprising using a computer to perform the following process actions for each person:

prior to presenting the stimulus to the person,

presenting to the person, at least once, a training stimulus belonging to each of one or more stimuli categories of interest,

inputting signals from a brain activity sensing device captured during a time the training stimulus was presented and beyond, wherein the signals exhibit different distinguishing characteristics whenever a stimulus belonging to a different one of the one or more stimuli categories is presented to the person, and wherein the distinguishing characteristics are indicative of an involuntary, subconscious response of the brain of the person to the stimulus, and

employing the inputted signals to train a detection module to recognize the respective distinguishing characteristics associated with each of the stimuli categories of interest; and

once the detection module is trained,

presenting a stimulus belonging to a trained stimuli category to the person,

inputting signals from the brain activity sensing device captured during a time the stimulus was presented and beyond,

using the detection module to determine if distinguishing characteristics associated with a stimulus belonging to one of the trained stimuli categories are present in the inputted signal, and

outputting an indicator identifying the trained stimuli category that the stimulus belongs to, whenever it is determined distinguishing characteristics associated with a stimulus belonging to one of the trained stimuli categories are present in the inputted signal.

3. The process of claim 2, wherein it is known that the involuntary, subconscious response of the brain of the person to a stimulus belonging to a stimuli category will occur within a particular period of time after the stimulus is presented to the person, and wherein signals from the brain activity sensing device are captured for this particular period of time after the stimulus has been presented.

4. The process of claim 2, wherein the brain activity sensing device is an electroencephalograph (EEG) which produces multiple signals representing the difference in potential over time between pairs of sensors each of which is placed at a different location on the scalp of the person, and wherein the process actions of inputting signals from the brain activity sensing device comprises the actions of:

inputting the multiple signals from the EEG; and

pre-processing the signals prior to providing them to the detection module.

5. The process of claim 4, wherein the process action of pre-processing the signals, comprises the actions of:

converting each of the EEG signals associated with pairs of sensors of interest to a digital signal;

sampling each of the resulting digital signals at a prescribed rate to reduce the amount of processing necessary to analyze the signals;

transforming each sampled signal to the frequency domain using a prescribed transformation technique; and

bandpass filtering each of the transformed signals to eliminate frequencies above a prescribed upper limit and below a prescribed lower limit.

6. The process of claim 4, wherein it is known that the involuntary, subconscious response of the brain of the person to a stimulus belonging to a stimuli category will occur within a particular period of time after the stimulus is presented to the person, and wherein the process action of pre-processing the signals, further comprises an action of retaining only the portion of the signals occurring within the particular period of time after the stimulus is presented to the person.

7. The process of claim 2, wherein each time a stimulus is presented to the person, it is presented in such a way as the person is attentive to the stimulus.

8. The process of claim 2, wherein each time a stimulus is presented to the person, it is presented in such a way that it is in the person'"'"'s attentive periphery and so the person is not conscious of the stimulus.

9. The process of claim 2, wherein the process action of outputting an indicator identifying the trained stimuli category that the stimulus belongs to, comprises the actions of:

for each of the one or more stimuli categories, determining the degree to which distinguishing characteristics associated with the stimuli category under consideration are exhibited in the signals;

determining if the distinguishing characteristics associated with one of the trained stimuli categories are exhibited in the signals to at least a prescribed degree and to a degree that exceeds the other categories, if any; and

outputting the indicator identifying the stimuli category that the stimulus presented to the person belongs to, whenever it is determined the distinguishing characteristics associated with one of the trained stimuli categories are exhibited in the signals to at least the prescribed degree and to a degree that exceeds the other categories, if any.

10. The process of claim 2, wherein the process action of outputting an indicator identifying the trained stimuli category that the stimulus belongs to, comprises the actions of:

for each of the one or more stimuli categories, determining the degree to which distinguishing characteristics associated with the stimuli category under consideration are exhibited in the signals;

whenever distinguishing characteristics associated with a trained stimuli category are exhibited to at least a prescribed degree in the signals, establishing a weighted indicator whose weight represents the likelihood that the stimulus presented to the person belongs to that stimuli category;

whenever distinguishing characteristics associated with a trained stimuli category are not exhibited in the signals to at least the prescribed degree, establishing a zero weighted indicator; and

outputting the weighted indicators.

11. The process of claim 10, wherein the process action of outputting an indicator identifying the trained stimuli category that the stimulus belongs to, further comprises an action of designating that the presented stimulus belongs to the trained stimuli category associated with the largest weighted indicator.

12. The process of claim 2, wherein a stimulus has been presented to more than one person, and wherein the process further comprises the actions of:

for each person presented with a stimulus, identifying the trained stimuli category associated with the indicator that was output;

employing a voting scheme, wherein the output of an indicator identifying a particular trained stimuli category is a vote that the stimulus presented to the people belongs to that category; and

designating, based on the results of the voting scheme, which trained stimuli category that the stimulus presented to the people belongs to.

13. The process of claim 12, wherein the process action of designating which trained stimuli category that the stimulus presented to the people belongs to, comprises an action of designating that the stimulus presented to the people belongs to the trained stimuli category getting the most votes.

14. The process of claim 10, wherein a stimulus has been presented to more than one person, and wherein the process further comprises the actions of:

for each trained stimuli category, combining the weighted indicators associated with the category under consideration for all the people presented with the stimulus to produce an overall indicator for the category under consideration;

identifying which of the overall indicators has the greatest weight;

determining if the identified overall indicator exceeds a prescribed minimum weight; and

designating that the stimulus presented to the people was a stimulus belonging to the trained stimuli category associated with the identified overall indicator, whenever it is determined the identified overall indicator exceeds the prescribed minimum weight.

15. A computer-implemented process for identifying a stimuli category of a perceptual stimulus that has been presented to a person whose brain activity is being monitored, comprising using a computer to perform the following process actions:

in a training phase,

presenting to the person, at least once, a training stimulus belonging to each of one or more stimuli categories of interest,

inputting signals from a brain activity sensing device, wherein the signals exhibit different distinguishing characteristics whenever a stimulus belonging to a different one of the one or more stimuli categories is presented to the person, and wherein the distinguishing characteristics are indicative of an involuntary, subconscious response of the brain of the person to the stimulus, and

employing the inputted signals to train a detection module to recognize the respective distinguishing characteristics associated with each of the stimuli categories of interest; and

in a detection phase,

presenting a stimulus belonging to a trained stimuli category to the person a prescribed number of times,

inputting signals from the brain activity sensing device captured during a period encompassing the time the stimulus is repeatedly presented to the person and beyond,

using the detection module to identify each time distinguishing characteristics associated with a stimulus belonging to one of the trained stimuli categories are present in the inputted signals and capturing the results of the identifying,

determining which of the one or more trained stimuli categories the presented stimulus belongs to based on the captured results, and

outputting an indicator identifying the trained stimuli category that the presented stimulus belongs to.

16. The process of claim 15, wherein the process action of determining which of the one or more trained stimuli categories the presented stimulus belongs to, comprises the actions of:

for each time the stimulus was presented to the person, determining if distinguishing characteristics associated with a trained stimuli category are exhibited in the signals within a prescribed period of time following the presentation of the stimulus;

employing a voting scheme, wherein whenever it is determined that distinguishing characteristics associated with a trained stimuli category are exhibited in the signals within a prescribed period of time following the presentation of the stimulus, a vote is cast that the stimulus belongs to that stimuli category; and

designating, based on the results of the voting scheme, which of the trained stimuli categories that the stimulus presented to the person belongs to.

17. The process of claim 16, wherein the process action of designating which of the trained stimuli categories that the stimulus presented to the person belongs to, comprises an action of designating that the stimulus presented to the person belongs to the trained stimuli category getting the most votes.

18. The process of claim 15, wherein the process action of determining which of the one or more trained stimuli categories the presented stimulus belongs to, comprises the actions of:

for each time the stimulus was presented to the person,

determining the degree to which distinguishing characteristics associated with each trained stimuli category are exhibited in the signals within a prescribed period of time following the presentation of the stimulus,

whenever the distinguishing characteristics are exhibited to at least a prescribed minimum degree, establishing a weighted indicator for each trained stimuli category whose weight represents the likelihood that the stimulus presented to the person belongs to that stimuli category, and

whenever distinguishing characteristics associated with a trained stimuli category are not exhibited to at least the prescribed minimum degree, establishing a zero weighted indicator for that stimuli category;

for each trained stimuli category, combining the weighted indicators associated with the category under consideration for all the presentation instances to produce an overall indicator for the stimuli category under consideration;

identifying which of the overall indicators has the greatest weight;

determining if the identified overall indicator exceeds a prescribed minimum weight; and

designating that the stimulus presented to the person was a stimulus belonging to the trained stimuli category associated with the identified overall indicator, whenever it is determined the identified overall indicator exceeds the prescribed minimum weight.

19. The process of claim 15, wherein the process action of determining which of the one or more trained stimuli categories the presented stimulus belongs to, comprises the actions of:

for each time the stimulus was presented to the person,

determining the degree to which distinguishing characteristics associated with each trained stimuli category are exhibited in the signals within a prescribed period of time following the presentation of the stimulus,

whenever the distinguishing characteristics are exhibited to at least a prescribed minimum degree, establishing a weighted indicator for each trained stimuli category whose weight represents the likelihood that the stimulus presented to the person belongs to that stimuli category, and

whenever distinguishing characteristics associated with a trained stimuli category are not exhibited to at least the prescribed minimum degree, establishing a zero weighted indicator for that stimuli category;

for each trained stimuli category, combining the weighted indicators associated with the category under consideration for all the presentation instances to produce an overall indicator for the stimuli category under consideration;

identifying which of the overall indicators has the greatest weight;

determining if the identified overall indicator exceeds at least one of, a prescribed minimum weight, and the second largest indicator by more than a prescribed amount;

designating that the stimulus presented to the person was a stimulus belonging to the trained stimuli category associated with the identified overall indicator, whenever it is determined the identified overall indicator exceeds the prescribed minimum weight, or exceeds the second largest indicator by more than the prescribed amount, or both;

whenever it is determined the identified overall indicator does not exceed the prescribed minimum weight, or it is determined the identified overall indicator does not exceed the second largest indicator by more than a prescribed amount,

presenting the same stimulus to the person that was previously presented;

repeating the inputting, identifying and determining actions;

designating that the stimulus presented to the person was a stimulus belonging to the trained stimuli category associated with the identified overall indicator, whenever it is determined the identified overall indicator exceeds the prescribed minimum weight and exceeds the second largest indicator by more than the prescribed amount.

20. The process of claim 15, wherein the process action of presenting the stimulus belonging to the trained stimuli category to the person the prescribed number of times, comprises presenting the stimulus between one and ten times.