Neurophysiologically driven high speed image triage system and method

• US 7,835,581 B2
• Filed: 08/30/2006
• Issued: 11/16/2010
• Est. Priority Date: 08/30/2006
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Mathan, Santosh, Carciofini, James C., Ververs, Patricia M., Dorneich, Michael C., Whitlow, Stephen D.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Akhavannik; Hadi
• Application Number:
  US11/513,723
• Publication Number:
  US 20080056611A1
• Time in Patent Office: 1,539 Days
• Field of Search: 382/225
• US Class Current: 382/224
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

A system and method of efficiently and effectively triaging an image that may include one or more target entities. The image is divided into a plurality of individual image chips, and each image chip is successively displayed to a user for a presentation time period. Data are collected from the user at least while each image chip is being displayed. For each image chip, a probability that the image chip at least includes a target entity is assigned, based at least in part on the collected data. The image is then displayed with the assigned probabilities overlaid thereon.

Claims

1. A method of conducting image triage of an image that may include one or more target entities, comprising:

dividing the image into a plurality of individual image chips;

successively displaying each image chip to a user for a presentation time period;

collecting neurophysiological data and physical response data from the user at least while each image chip is being displayed;

for each image chip, assigning a first probability that the image chip at least includes a target entity, based on the collected physical response data;

for each image chip, assigning a second probability that the image chip at least includes a target entity, based on the collected neurophysiological data;

applying a weighting factor to the second probability;

for each image chip, determining and assigning an overall probability from a combination of the first probability and the weighted second probability; and

displaying the image with the assigned probabilities overlaid thereon.

2. The method of claim 1, further comprising:

generating a contour map that includes data representative of the probability assigned to each image chip.

3. The method of claim 2, wherein the step of displaying the image with the assigned probabilities overlaid thereon comprises displaying the image with the generated contour map overlaid thereon.

4. The method of claim 1, wherein the image chips are successively displayed to the user in accordance with a rapid serial visualization (RSVP) paradigm at a preselected RSVP rate.

5. The method of claim 4, wherein the preselected RSVP rate ranges from about 75 milliseconds per image chip to about 200 milliseconds per image chip.

6. The method of claim 1, wherein the step of successively displaying the image chips comprises:

dividing the image chips into image chip groups, each image chip group including a subset of the plurality of image chips;

successively displaying the image chips in a each image chip group to the user for the presentation time period; and

displaying a predetermined fixation image to the user for a predetermined relaxation time period between the successive display of the image chips in each image chip group.

7. The method of claim 1, further comprising:

collecting the neurophysiological data and the physical response data from the user from a predetermined time period before an image chip is displayed to a predetermined time period after the image chip is displayed.

8. The method of claim 7, wherein the predetermined time period before an image chip is displayed and the predetermined time period after an image chip is displayed are each about 1.0 second.

9. The method of claim 1, further comprising:

supplying the collected neurophysiological data and the collected physical response data to a first classifier and a second classifier, respectively, for assignment of the first and second probabilities, respectively.

10. The method of claim 9, wherein the first and second classifiers each comprise a support vector machine (SVM).

11. The method of claim 10, wherein the first and second classifiers each comprise trained SVMs associated with the user.

12. The method of claim 1, further comprising:

monitoring one or more states of the user; and

supplying one or more alerts to the user based on the one or more states of the user,

wherein the monitored states of the user include one or more of user attention lapses, eye activity, and head movements.

16. The system of claim 1, wherein the processor is further configured to successively display the image chips to the user in accordance with a rapid serial visualization (RSVP) paradigm at a preselected RSVP rate.

17. The system of claim 16, wherein the preselected RSVP rate ranges from about 75 milliseconds per image chip to about 200 milliseconds per image chip.

13. A system for conducting image triage of an image that may include one or more target entities, comprising:

a display device operable to receive display commands and, in response thereto, to display an image;

a data collector comprising a neurophysiological data collector configured to at least selectively collect neurophysiological data from a user and a user interface configured to receive input stimulus from the user and, in response thereto, to supply physical response data;

processor coupled to receive the collected data from the data collector, the processor further coupled to the display device and configured to;

selectively retrieve an image,

divide the image into a plurality of individual image chips,

successively command the display device to display each image chip to a user for a presentation time period,

implement a first classifier and a second classifier, the first classifier configured to assign a first probability to each image chip based on the collected physical response data, the second classifier configured to assign a second probability to each image chip based on the collected neurophysiological data

apply a weighting factor to the second probability,

determine and assign a probability to each displayed image chip from a combination of the first probability and the weighted second probability, each assigned probability representative of a likelihood that the image chip at least includes a target entity, and

supply commands to the display device that cause the display device to display the image with the assigned probabilities overlaid thereon.

14. The system of claim 13, wherein the processor is further configured to generate a contour map that includes data representative of the probability assigned to each image chip.

15. The system of claim 14, wherein the processor is configured to display the image with the generated contour map overlaid thereon.

18. The system of claim 13, wherein the processor is further configured to:

divide the image chips into image chip groups, each image chip group including a subset of the plurality of image chips;

supply display commands to the display device that cause the display device to successively display the image chips in a each image chip group to the user for the presentation time period; and

supply display commands to the display device that cause the display device to display a predetermined fixation image to the user for a predetermined relaxation time period between the successive display of the image chips in each image chip group.

19. The system of claim 13, wherein the neurological data collector is configured to collect the data from the user from a predetermined time period before an image chip is displayed to a predetermined time period after the image chip is displayed.

20. The system of claim 19, wherein the predetermined time period before an image chip is displayed and the predetermined time period after an image chip is displayed are each about 1.0 second.

21. The system of claim 13, wherein the processor is further configured to collect the physical response data from a predetermined time period before an image chip is displayed to a predetermined time period after the image chip is displayed.

22. The system of claim 13, wherein:

the collected neurophysiological data and the collected physical response data are supplied to the first classifier and the second classifier, respectively, for assignment of the first and second probabilities, respectively.

23. The system of claim 22, wherein the first and second classifiers each comprise a support vector machine (SVM).

24. The system of claim 23, wherein the first and second classifiers each comprise trained SVMs associated with the user.

25. The system of claim 13, further comprising:

one or more user state monitors configured to monitoring sensor one or more states of the user and supply user state data representative thereof,

wherein;

the processor is further configured to receive the user state data, and to determine the user is in a state that could adversely compromise probability assignment effectiveness, and selectively generate one or more alerts based on the one or more states of the user, and the states of the user include one or more of user attention lapses, eye activity, and head movements.