Neurophysiologically driven high speed image triage system and method

• US 20080056611A1
• Filed: 08/30/2006
• Published: 03/06/2008
• Est. Priority Date: 08/30/2006
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Mathan, Santosh, Carciofini, James C., Ververs, Patricia M., Dorneich, Michael C., Whitlow, Stephen D.
• Granted Patent: US 7,835,581 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

A system and method of efficiently and effectively triaging an image that may include one or more target entities. The image is divided into a plurality of individual image chips, and each image chip is successively displayed to a user for a presentation time period. Data are collected from the user at least while each image chip is being displayed. For each image chip, a probability that the image chip at least includes a target entity is assigned, based at least in part on the collected data. The image is then displayed with the assigned probabilities overlaid thereon.

Claims

1. A method of conducting image triage of an image that may include one or more target entities, comprising:

dividing the image into a plurality of individual image chips;

successively displaying each image chip to a user for a presentation time period;

collecting data from the user at least while each image chip is being displayed;

for each image chip, assigning a probability that the image chip at least includes a target entity, based at least in part on the collected data; and

displaying the image with the assigned probabilities overlaid thereon.

2. The method of claim 1, further comprising:

generating a contour map that includes data representative of the probability assigned to each image chip.

3. The method of claim 2, wherein the step of displaying the image with the assigned probabilities overlaid thereon comprises displaying the image with the generated contour map overlaid thereon.

4. The method of claim 1, wherein the image chips are successively displayed to the user in accordance with a rapid serial visualization (RSVP) paradigm at a preselected RSVP rate.

5. The method of claim 4, wherein the preselected RSVP rate ranges from about 75 milliseconds per image chip to about 200 milliseconds per image chip.

6. The method of claim 1, wherein the step of successively displaying the image chips comprises:

dividing the image chips into image chip groups, each image chip group including a subset of the plurality of image chips;

successively displaying the image chips in a each image chip group to the user for the presentation time period; and

displaying a predetermined fixation image to the user for a predetermined relaxation time period between the successive display of the image chips in each image chip group.

7. The method of claim 1, further comprising:

collecting the data from the user from a predetermined time period before an image chip is displayed to a predetermined time period after the image chip is displayed.

8. The method of claim 7, wherein the predetermined time period before an image chip is displayed and the predetermined time period after an image chip is displayed are each about 1.0 second.

9. The method of claim 1, further comprising:

supplying the collected data to a classifier for assignment of the probabilities.

10. The method of claim 9, wherein the classifier comprises a support vector machine (SVM).

11. The method of claim 10, wherein the SVM is a trained SVM associated with the user.

12. The method of claim 1, wherein the collected data are neurophysiological data.

13. The method of claim 1, wherein the collected data are physical response data.

14. The method of claim 1, wherein the collected data are neurophysiological data and physical response data, and wherein the method further comprises:

for each image chip, assigning the probability that the image chip at least includes a target entity, based on the collected neurophysiological data and the collected physical response data.

15. The method of claim 14, wherein the assigned probability is an overall probability, and wherein the method further comprises:

for each image chip, assigning a first probability that the image chip at least includes a target entity, based on the collected physical response data;

for each image chip, assigning a second probability that the image chip at least includes a target entity, based on the collected neurophysiological data;

applying a weighting factor to the second probability; and

determining the overall probability from a combination of the first probability and the weighted second probability.

16. The method of claim 14, further comprising:

collecting the physical response data from the user from a predetermined time period before an image chip is displayed to a predetermined time period after the image chip is displayed.

17. The method of claim 14, further comprising:

supplying the collected neurophysiological data and the collected physical response data to a first classifier and a second classifier, respectively, for assignment of the first and second probabilities, respectively.

18. The method of claim 17, wherein the first and second classifiers each comprise a support vector machine (SVM).

19. The method of claim 18, wherein the first and second classifiers each comprise trained SVMs associated with the user.

20. The method of claim 1, further comprising:

monitoring one or more states of the user; and

supplying one or more alerts to the user based on the one or more states of the user,

wherein the monitored states of the user include one or more of user attention lapses, eye activity, and head movements.

24. The system of claim 1, wherein the processor is further configured to successively display the image chips to the user in accordance with a rapid serial visualization (RSVP) paradigm at a preselected RSVP rate.

25. The system of claim 24, wherein the preselected RSVP rate ranges from about 75 milliseconds per image chip to about 200 milliseconds per image chip.

29. The system of claim 1, wherein:

the processor implements a classifier; and

the collected neurophysiological data are supplied to the classifier for assignment of the probabilities.

30. The system of claim 29, wherein the classifier comprises a support vector machine (SVM).

31. The system of claim 30, wherein the SVM is a trained SVM associated with the user.

21. A system for conducting image triage of an image that may include one or more target entities, comprising:

a display device operable to receive display commands and, in response thereto, to display an image;

a data collector configured to at least selectively collect data from a user;

processor coupled to receive the collected data from the data collector, the processor further coupled to the display device and configured to;

selectively retrieve an image,

divide the image into a plurality of individual image chips,

successively command the display device to display each image chip to a user for a presentation time period,

assign a probability to each displayed image chip based at least in part on the collected data, each assigned probability representative of a likelihood that the image chip at least includes a target entity, and

supply commands to the display device that cause the display device to display the image with the assigned probabilities overlaid thereon.

22. The system of claim 21, wherein the processor is further configured to generate a contour map that includes data representative of the probability assigned to each image chip.

23. The system of claim 22, wherein the processor is configured to display the image with the generated contour map overlaid thereon.

26. The system of claim 21, wherein the processor is further configured to:

divide the image chips into image chip groups, each image chip group including a subset of the plurality of image chips;

supply display commands to the display device that cause the display device to successively display the image chips in a each image chip group to the user for the presentation time period; and

supply display commands to the display device that cause the display device to display a predetermined fixation image to the user for a predetermined relaxation time period between the successive display of the image chips in each image chip group.

27. The system of claim 21, wherein the neurological data collector is configured to collect the data from the user from a predetermined time period before an image chip is displayed to a predetermined time period after the image chip is displayed.

28. The system of claim 27, wherein the predetermined time period before an image chip is displayed and the predetermined time period after an image chip is displayed are each about 1.0 second.

32. The system of claim 21, wherein the data collector comprises a neurophysiological data collector configured to at least selectively collect neurophysiological data from the user.

33. The system of claim 21, wherein the data collector comprises a user interface configured to receive input stimulus from the user and, in response thereto, to supply physical response data.

34. The system of claim 21, wherein:

the data collector comprises a neurophysiological data collector configured to at least selectively collect neurophysiological data from the user and a user interface configured to receive input stimulus from the user and, in response thereto, to supply physical response data; and

the processor is configured to assign the probability based on the collected neurophysiological data and the collected physical response data.

35. The system of claim 34, wherein:

the assigned probability is an overall probability;

the processor implements a first classifier and a second classifier, the first classifier configured to assign a first probability to each image chip based on the collected physical response data, the second classifier configured to assign a second probability to each image chip based on the collected neurophysiological data; and

the processor is further configured to (i) apply a weighting factor to the second probability and (ii) determine the overall probability from a combination of the first probability and the weighted second probability.

36. The system of claim 34, wherein the processor is further configured to collect the physical response data from a predetermined time period before an image chip is displayed to a predetermined time period after the image chip is displayed.

37. The system of claim 34, wherein:

the processor implements a first classifier and a classifier; and

the collected neurophysiological data and the collected physical response data are supplied to the first classifier and the second classifier, respectively, for assignment of the first and second probabilities, respectively.

38. The system of claim 37, wherein the first and second classifiers each comprise a support vector machine (SVM).

39. The system of claim 38, wherein the first and second classifiers each comprise trained SVMs associated with the user.

40. The system of claim 21, further comprising:

one or more user state monitors configured to monitoring sensor one or more states of the user and supply user state data representative thereof,

wherein;

the processor is further configured to receive the user state data, and to determine the user is in a state that could adversely compromise probability assignment effectiveness, and selectively generate one or more alerts based on the one or more states of the user, and

the states of the user include one or more of user attention lapses, eye activity, and head movements.