DETECTING BEHAVIORAL DEVIATIONS BY MEASURING EYE MOVEMENTS

• US 20090232357A1
• Filed: 03/17/2008
• Published: 09/17/2009
• Est. Priority Date: 03/17/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kraemer, James R., Angell, Robert Lee
• Granted Patent: US 8,345,935 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

According to one embodiment of the present invention, a computer implemented method, apparatus, and computer usable program product is provided for detecting behavioral deviations in members of a cohort group. A member of a cohort group is identified. Each member of the cohort group shares a common characteristic. Ocular metadata associated with the member of the cohort group is generated in real-time. The ocular metadata describes movements of an eye of the member of the cohort group. The ocular metadata is analyzed to identify patterns of ocular movements. In response to the patterns of ocular movements indicating behavioral deviations in the member of the cohort group, the member of the cohort group is identified as a person of interest. A person of interest may be subjected to an increased level of monitoring and/or other security measures.

Claims

1. A computer implemented method for detecting behavioral deviations in members of a cohort group, the computer implemented method comprising:

generating ocular metadata associated with a member of a cohort group using images of eye movements captured by a set of cameras, wherein the ocular metadata describes movements of an eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are received;

analyzing the ocular metadata using a set of data models to identify patterns of ocular movements; and

responsive to the patterns of ocular movements indicating behavioral deviations in the member of the cohort group, identifying the member of the cohort group as a person of interest, wherein the person of interest is subject to an increased level of monitoring.

2. The computer implemented method of claim 1 wherein the ocular metadata comprises metadata describing changes in a pupil size due to dilation and constriction of the pupil and further comprising:

analyzing the ocular metadata to identify patterns in the changes in the pupil dilation, wherein the patterns of the pupil dilation comprises a rate of changes in the pupil size, the degree of change in the pupil size, the number of changes in the pupil size, and wherein the patterns in the changes in the pupil dilation indicates any external stimuli associate with the changes in the pupil dilation.

3. The computer implemented method of claim 1 further comprising:

analyzing the ocular metadata to identify a given direction of gaze and a duration the gaze remains fixed in the given direction; and

responsive to the ocular movements indicating a direction of gaze in the given direction for a duration of time that is greater than a threshold duration, indicating an occurrence of a fixed stare in the patterns of ocular movements.

4. The computer implemented method of claim 1 further comprising:

analyzing the ocular metadata to identify visual lines of gaze, wherein a visual line of gaze indicates where the eye is looking;

responsive to the visual lines of gaze indicating the member of the cohort group is watching a single object or a type of object to form a target object or target type of object, indicating the member of the cohort group is watching the target object or the target type of object in the patterns of ocular movements; and

analyzing the patterns of ocular movements to determine whether the target object or target type of object indicates a behavioral deviation.

5. The computer implemented method of claim 1 further comprising:

analyzing the ocular metadata to identify a number of times the visual line of gaze is redirected from a first direction to a second direction over a time interval to form a rate of redirection of the gaze; and

responsive to the rate of redirection of the gaze exceeding a threshold rate, identifying the ocular movements as a furtive glance; and

responsive to identifying a set of furtive glances occurring over a threshold interval of time, identifying the set of furtive glances as a pattern of ocular movements in the patterns of ocular movements.

6. The computer implemented method of claim 1 further comprising:

analyzing the ocular metadata to identify a rate of blinking over an interval of time; and

responsive to the rate of blinking falling below a threshold rate of blinking, identifying a pattern of ocular movement comprising an unblinking stare, wherein the unblinking stare is a pattern of ocular movements in the patterns of ocular movements.

7. The computer implemented method of claim 1 wherein the patterns of ocular movements indicating behavioral deviations comprises at least one of a set of furtive glances, an unblinking stare, a fixed stare, an atypical rate of blinking, an atypical target object or type of object watched by the person, a flat or unemotional affect, atypical patterns in pupil dilation, atypical narrowing of the eyes, and atypical widening of the eyes.

8. The computer implemented method of claim 1 wherein the cohort group is a group of people in a transportation facility.

9. The computer implemented method of claim 1 wherein the transportation facility is an airport.

10. The computer implemented method of claim 1 wherein increasing a level of monitoring associated with the person of interest comprises at least one of continuing capturing video images of the eye movements and behavior of the person of interest, questioning the person of interest by security personnel, checking an identification of the person of interest, searching baggage associated with the person of interest, searching the person of interest, monitoring the activities of the person of interest by security personnel, scanning the person of interest with a metal detector or metal detector wand, x-ray the person of interest, and checking the person of interest with a drug sniffing dog.

11. A computer program product for detecting behavioral deviations in members of a cohort group, the computer program product comprising:

a computer readable medium;

program code stored on the computer readable medium for generating ocular metadata associated with a member of a cohort group using images of eye movements captured by a set of cameras, wherein the ocular metadata describes movements of an eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are received;

program code stored on the computer readable medium for analyzing the ocular metadata using a set of data models to identify patterns of ocular movements; and

program code stored on the computer readable medium for identifying the member of the cohort group as a person of interest in response to the patterns of ocular movements indicating behavioral deviations in the member of the cohort group, wherein the person of interest is subject to an increased level of monitoring.

12. The computer program product of claim 11 wherein the ocular metadata comprises metadata describing changes in a pupil size due to dilation and constriction of the pupil and further comprising:

program code stored on the computer readable medium for analyzing the ocular metadata to identify patterns in the changes in the pupil dilation, wherein the patterns of the pupil dilation comprises a rate of changes in the pupil size, the degree of change in the pupil size, the number of changes in the pupil size, and wherein the patterns in the changes in the pupil dilation indicates any external stimuli associate with the changes in the pupil dilation.

13. The computer program product of claim 11 further comprising:

program code stored on the computer readable medium for analyzing the ocular movements to identify a given direction of gaze and a duration the gaze remains fixed in the given direction; and

program code stored on the computer readable medium for indicating an occurrence of a fixed stare in the patterns of ocular movements in response to the ocular movements indicating a direction of gaze in the given direction for a duration of time that is greater than a threshold duration.

14. The computer program product of claim 11 further comprising:

program code stored on the computer readable medium for analyzing the ocular metadata to identify visual lines of gaze, wherein a visual line of gaze indicates where the eye is looking;

program code stored on the computer readable medium for indicating the member of the cohort group is watching the target object or the target type of object in the patterns of ocular movements in response to the visual lines of gaze indicating the member of the cohort group is watching a single object or a type of object to form a target object or target type of object; and

program code stored on the computer readable medium for analyzing the patterns of ocular movements to determine whether the target object or target type of object indicates a behavioral deviation.

15. The computer program product of claim 11 further comprising:

program code stored on the computer readable medium for analyzing the ocular metadata to identify a number of times the visual line of gaze is redirected from a first direction to a second direction over a time interval to form a rate of redirection of the gaze; and

program code stored on the computer readable medium for identifying the ocular movements as a furtive glance in response to the rate of redirection of the gaze exceeding a threshold rate; and

program code stored on the computer readable medium for identifying the set of furtive glances as a pattern of ocular movements in the patterns of ocular movements in response to identifying a set of furtive glances occurring over a threshold interval of time.

16. The computer program product of claim 11 further comprising:

program code stored on the computer readable medium for analyzing the ocular metadata to identify a rate of blinking over an interval of time; and

program code stored on the computer readable medium for identifying a pattern of ocular movement comprising an unblinking stare in response to the rate of blinking falling below a threshold rate of blinking, wherein the unblinking stare is a pattern of ocular movements in the patterns of ocular movements.

17. An apparatus comprising:

a bus system;

a communications system coupled to the bus system;

a memory connected to the bus system, wherein the memory includes computer usable program code; and

a processing unit coupled to the bus system, wherein the processing unit executes the computer usable program code to generate ocular metadata associated with a member of a cohort group using images of eye movements captured by a set of cameras, wherein the ocular metadata describes movements of an eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are received;

analyze the ocular metadata using a set of data models to identify patterns of ocular movements; and

identify the member of the cohort group as a person of interest, wherein the person of interest is subject to an increased level of monitoring in response to the patterns of ocular movements indicating behavioral deviations in the member of the cohort group.

18. The apparatus of claim 17 wherein the ocular metadata comprises metadata describing changes in a pupil size due to dilation and constriction of the pupil and wherein the processor unit further executes the computer usable program code to analyze the ocular metadata to identify patterns in the changes in the pupil dilation, wherein the patterns of the pupil dilation comprises a rate of changes in the pupil size, the degree of change in the pupil size, the number of changes in the pupil size, and wherein the patterns in the changes in the pupil dilation indicates any external stimuli associate with the changes in the pupil dilation.

19. The apparatus of claim 17 wherein the processor unit further executes the computer usable program code to analyze the patterns of ocular movements to analyze the ocular movements to identify a given direction of gaze and a duration the gaze remains fixed in the given direction;

and indicate an occurrence of a fixed stare in the patterns of ocular movements in response to the ocular movements indicating a direction of the gaze in the given direction for the duration of time is greater than a threshold duration.

20. The apparatus of claim 17 wherein the processor unit further executes the computer usable program code to analyze the ocular metadata to identify visual lines of gaze, wherein a visual line of gaze indicates where the eye is looking;

indicating the member of the cohort group is watching the target object or the target type of object in the patterns of ocular movements in response to the visual lines of gaze indicating the member of the cohort group is watching a single object or a type of object to form a target object or target type of object; and

analyze the patterns of ocular movements to determine whether the target object or target type of object indicates a behavioral deviation.

21. The apparatus of claim 17 wherein the processor unit further executes the computer usable program code to analyze the ocular metadata to identify a rate of blinking over an interval of time;

and identify a pattern of ocular movement comprising an unblinking stare in response to the rate of blinking falling below a threshold rate of blinking, wherein the unblinking stare is a pattern of ocular movements in the patterns of ocular movements.

22. A data processing system for detecting behavioral deviations in members of a cohort group, the data processing system comprising:

a video analysis system, wherein the video analysis system generates ocular metadata associated with a member of a cohort group using images of eye movements associated with the member of the cohort group, wherein the images of the eye movements are captured by a set of cameras, wherein the ocular metadata describes movements of an eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are received;

an analysis server, wherein the analysis server analyzes the ocular metadata using analytics and a set of data models to identify patterns of ocular movement, and wherein the analysis server identifies the member of the cohort group as a person of interest in response to the patterns of ocular movements indicating behavioral deviations in the member of the cohort group.

23. The data processing system of claim 22 further comprising:

the set of cameras, wherein the set of cameras generates the images of the eye movements associated with a member of a cohort group, and wherein the set of cameras are located in a transportation facility.

24. The data processing system of claim 22 wherein the video analysis system further comprises:

an ocular movement analytics engine, wherein the ocular movement analytics engine analyzes the ocular metadata to identify a number of times the visual line of gaze is redirected from a first direction to a second direction over a time interval to form a rate of redirection of the gaze;

identify the ocular movements as a furtive glance in response to the rate of redirection of the gaze exceeding a threshold rate; and

identify the set of furtive glances as a pattern of ocular movements in the patterns of ocular movements in response to identifying a set of furtive glances occurring over a threshold interval of time.

25. The data processing system of claim 22 wherein the ocular metadata comprises metadata describing changes in a pupil size due to dilation and constriction of the pupil and wherein the video analysis system further comprises:

a retinal/iris analytics engine, wherein the retinal/iris analytics engine analyzes the ocular metadata to identify patterns in the changes in the pupil dilation, wherein the patterns of the pupil dilation comprises a rate of changes in the pupil size, the degree of change in the pupil size, the number of changes in the pupil size, and wherein the patterns in the changes in the pupil dilation indicates any external stimuli associate with the changes in the pupil dilation.