Detecting behavioral deviations by measuring eye movements

• US 8,582,832 B2
• Filed: 09/13/2012
• Issued: 11/12/2013
• Est. Priority Date: 03/17/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Angell, Robert Lee, Kraemer, James R.
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US13/614,984
• Publication Number:
  US 20130022948A1
• Time in Patent Office: 425 Days
• Field of Search: 382/100, 382115-118, 351200-210, 359 15- 20, 359600-610, 359/643
• US Class Current: 382/117
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Detecting behavioral deviations in members of a cohort group is provided. Ocular metadata is analyzed to identify patterns in changes in a size of a pupil of an eye of a member of the cohort group captured by a set of cameras. The ocular metadata describes the changes in the size of the pupil. The patterns in the changes in the size of the pupil include a rate of changes in the size of the pupil, a degree of change in the size of the pupil, and a number of changes in the size of the pupil. The patterns in the changes in the size of the pupil indicate any external stimuli associated with the changes in the size of the pupil. In responsive to the changes in the size of the pupil indicating behavioral deviations in the member, the member is identified as a person of interest.

Claims

1. A computer implemented method for detecting behavioral deviations in members of a cohort group, the computer implemented method comprising:

analyzing ocular metadata to identify patterns in changes in a size of a pupil of an eye of a member of the cohort group captured by a set of cameras, wherein the ocular metadata describes the changes in the size of the pupil of the eye due to dilation and constriction of the pupil, and wherein the patterns in the changes in the size of the pupil include a rate of changes in the size of the pupil, a degree of change in the size of the pupil, and a number of changes in the size of the pupil, and wherein the patterns in the changes in the size of the pupil indicate any external stimuli associated with the changes in the size of the pupil; and

responsive to the changes in the size of the pupil of the eye indicating behavioral deviations in the member of the cohort group, identifying the member of the cohort group as a person of interest that is subject to an increased level of monitoring.

2. The computer implemented method of claim 1 further comprising:

generating the ocular metadata associated with the member of the cohort group using images of eye movements captured by the set of cameras, wherein the ocular metadata also describes movements of the eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are captured; and

analyzing the ocular metadata using a set of data models to identify patterns of ocular movements indicating the behavioral deviations.

3. The computer implemented method of claim 2 further comprising:

analyzing the ocular metadata to identify a given direction of gaze and a duration the gaze remains fixed in the given direction; and

responsive to the eye movements indicating a direction of gaze in the given direction for a duration of time that is greater than a threshold duration, indicating an occurrence of a fixed stare in the patterns of ocular movements.

4. The computer implemented method of claim 2 further comprising:

analyzing the ocular metadata to identify visual lines of gaze, wherein a visual line of gaze indicates where the eye is looking;

responsive to the visual lines of gaze indicating the member of the cohort group is watching a single object or a type of object to form a target object or target type of object, indicating the member of the cohort group is watching the target object or the target type of object in the patterns of ocular movements; and

analyzing the patterns of ocular movements to determine whether the target object or target type of object indicates a behavioral deviation.

5. The computer implemented method of claim 4 further comprising:

analyzing the ocular metadata to identify a number of times the visual line of gaze is redirected from a first direction to a second direction over a time interval to form a rate of redirection of the gaze; and

responsive to the rate of redirection of the gaze exceeding a threshold rate, identifying the ocular movements as a furtive glance; and

responsive to identifying a set of furtive glances occurring over a threshold interval of time, identifying the set of furtive glances as a pattern of ocular movements in the patterns of ocular movements.

6. The computer implemented method of claim 2 further comprising:

analyzing the ocular metadata to identify a rate of blinking over an interval of time; and

responsive to the rate of blinking falling below a threshold rate of blinking, identifying a pattern of ocular movement comprising an unblinking stare, wherein the unblinking stare is a pattern of ocular movements in the patterns of ocular movements.

7. The computer implemented method of claim 2 wherein the patterns of ocular movements indicating behavioral deviations comprises at least one of a set of furtive glances, an unblinking stare, a fixed stare, an atypical rate of blinking, an atypical target object or type of object watched by the person, a flat or unemotional affect, atypical patterns in pupil dilation, atypical narrowing of the eyes, and atypical widening of the eyes.

8. The computer implemented method of claim 1 wherein the cohort group is a group of people in a transportation facility.

9. The computer implemented method of claim 8 wherein the transportation facility is an airport.

10. The computer implemented method of claim 1 wherein increasing a level of monitoring associated with the person of interest includes at least one of continuing capturing video images of eye movements and behavior of the person of interest, questioning the person of interest by security personnel, checking an identification of the person of interest, searching baggage associated with the person of interest, searching the person of interest, monitoring activities of the person of interest by security personnel, scanning the person of interest with a metal detector or metal detector wand, x-ray the person of interest, and checking the person of interest with a drug sniffing dog.

11. A computer program product for detecting behavioral deviations in members of a cohort group, the computer program product comprising:

a computer readable storage device;

program code stored on the computer readable storage device for analyzing ocular metadata to identify patterns in changes in a size of a pupil of an eye of a member of the cohort group captured by a set of cameras, wherein the ocular metadata describes the changes in the size of the pupil of the eye due to dilation and constriction of the pupil, and wherein the patterns in the changes in the size of the pupil include a rate of changes in the size of the pupil, a degree of change in the size of the pupil, and a number of changes in the size of the pupil, and wherein the patterns in the changes in the size of the pupil indicate any external stimuli associated with the changes in the size of the pupil; and

program code stored on the computer readable storage device for identifying the member of the cohort group as a person of interest that is subject to an increased level of monitoring in response to the changes in the size of the pupil of the eye indicating behavioral deviations in the member of the cohort group.

12. The computer program product of claim 11 further comprising:

program code stored on the computer readable storage device for generating the ocular metadata associated with the member of the cohort group using images of eye movements captured by the set of cameras, wherein the ocular metadata also describes movements of the eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are captured; and

program code stored on the computer readable storage device for analyzing the ocular metadata using a set of data models to identify patterns of ocular movements indicating the behavioral deviations.

13. The computer program product of claim 12 further comprising:

program code stored on the computer readable storage device for analyzing the ocular movements to identify a given direction of gaze and a duration the gaze remains fixed in the given direction; and

program code stored on the computer readable storage device for indicating an occurrence of a fixed stare in the patterns of ocular movements in response to the eye movements indicating a direction of gaze in the given direction for a duration of time that is greater than a threshold duration.

14. The computer program product of claim 12 further comprising:

program code stored on the computer readable storage device for analyzing the ocular metadata to identify visual lines of gaze, wherein a visual line of gaze indicates where the eye is looking;

program code stored on the computer readable storage device for indicating the member of the cohort group is watching a target object or a target type of object in the patterns of ocular movements in response to the visual lines of gaze indicating the member of the cohort group is watching a single object or a type of object to form the target object or the target type of object; and

program code stored on the computer readable storage device for analyzing the patterns of ocular movements to determine whether the target object or target type of object indicates a behavioral deviation.

15. The computer program product of claim 14 further comprising:

program code stored on the computer readable storage device for analyzing the ocular metadata to identify a number of times the visual line of gaze is redirected from a first direction to a second direction over a time interval to form a rate of redirection of the gaze; and

program code stored on the computer readable storage device for identifying the ocular movements as a furtive glance in response to the rate of redirection of the gaze exceeding a threshold rate; and

program code stored on the computer readable storage device for identifying a set of furtive glances as a pattern of ocular movements in the patterns of ocular movements in response to identifying the set of furtive glances occurring over a threshold interval of time.

16. The computer program product of claim 12 further comprising:

program code stored on the computer readable storage device for analyzing the ocular metadata to identify a rate of blinking over an interval of time; and

program code stored on the computer readable storage device for identifying a pattern of ocular movement comprising an unblinking stare in response to the rate of blinking falling below a threshold rate of blinking, wherein the unblinking stare is a pattern of ocular movements in the patterns of ocular movements.

17. An apparatus comprising:

a bus system;

a communications system coupled to the bus system;

a storage device connected to the bus system, wherein the storage device stores computer usable program code; and

a processor unit coupled to the bus system, wherein the processor unit executes the computer usable program code to analyze ocular metadata to identify patterns in changes in a size of a pupil of an eye of a member of a cohort group captured by a set of cameras, wherein the ocular metadata describes the changes in the size of the pupil of the eye due to dilation and constriction of the pupil, and wherein the patterns in the changes in the size of the pupil include a rate of changes in the size of the pupil, a degree of change in the size of the pupil, and a number of changes in the size of the pupil, and wherein the patterns in the changes in the size of the pupil indicate any external stimuli associated with the changes in the size of the pupil; and

identify the member of the cohort group as a person of interest that is subject to an increased level of monitoring in response to the changes in the size of the pupil of the eye indicating behavioral deviations in the member of the cohort group.

18. The apparatus of claim 17 wherein the processor unit further executes the computer usable program code to generate the ocular metadata associated with the member of the cohort group using images of eye movements captured by the set of cameras, wherein the ocular metadata also describes movements of the eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are captured;

and analyze the ocular metadata using a set of data models to identify patterns of ocular movements indicating the behavioral deviations.

19. The apparatus of claim 18 wherein the processor unit further executes the computer usable program code to analyze the patterns of ocular movements to analyze the eye movements to identify a given direction of gaze and a duration the gaze remains fixed in the given direction;

and indicate an occurrence of a fixed stare in the patterns of ocular movements in response to the ocular movements indicating a direction of the gaze in the given direction for a duration of time is greater than a threshold duration.

20. The apparatus of claim 18 wherein the processor unit further executes the computer usable program code to analyze the ocular metadata to identify visual lines of gaze, wherein a visual line of gaze indicates where the eye is looking;

indicate the member of the cohort group is watching a target object or a target type of object in the patterns of ocular movements in response to the visual lines of gaze indicating the member of the cohort group is watching a single object or a type of object to form the target object or the target type of object;

and analyze the patterns of ocular movements to determine whether the target object or target type of object indicates a behavioral deviation.

21. The apparatus of claim 18 wherein the processor unit further executes the computer usable program code to analyze the ocular metadata to identify a rate of blinking over an interval of time;

and identify a pattern of ocular movement comprising an unblinking stare in response to the rate of blinking falling below a threshold rate of blinking, wherein the unblinking stare is a pattern of ocular movements in the patterns of ocular movements.

22. A data processing system for detecting behavioral deviations in members of a cohort group, the data processing system comprising:

a video analysis system that includes a retinal/iris analytics engine, wherein the retinal/iris analytics engine analyzes ocular metadata to identify patterns in changes in a size of a pupil of an eye of a member of the cohort group captured by a set of cameras, wherein the ocular metadata describes the changes in the size of the pupil of the eye due to dilation and constriction of the pupil, and wherein the patterns in the changes in the size of the pupil include a rate of changes in the size of the pupil, a degree of change in the size of the pupil, and a number of changes in the size of the pupil, and wherein the patterns in the changes in the size of the pupil indicate any external stimuli associated with the changes in the size of the pupil

an analysis server, wherein the analysis server analyzes the ocular metadata using analytics and a set of data models to identify the patterns in the changes in the size of the pupil, and wherein the analysis server identifies the member of the cohort group as a person of interest that is subject to an increased level of monitoring in response to the changes in the size of the pupil of the eye indicating behavioral deviations in the member of the cohort group.

23. The data processing system of claim 22, wherein the set of cameras generates images of eye movements associated with the member of the cohort group, and wherein the set of cameras are located in a transportation facility.

24. The data processing system of claim 22 wherein the video analysis system further comprises:

an ocular movement analytics engine, wherein the ocular movement analytics engine analyzes the ocular metadata to identify a number of times a visual line of gaze is redirected from a first direction to a second direction over a time interval to form a rate of redirection of the gaze;

identify ocular movements as a furtive glance in response to the rate of redirection of the gaze exceeding a threshold rate; and

identify a set of furtive glances as a pattern of ocular movements in patterns of ocular movements in response to identifying the set of furtive glances occurring over a threshold interval of time.

25. The data processing system of claim 22, wherein the video analysis system generates the ocular metadata associated with the member of the cohort group using images of eye movements associated with the member of the cohort group, wherein the images of the eye movements are captured by the set of cameras, and wherein the ocular metadata also describes movements of the eye of the member of the cohort group, and wherein the ocular metadata is generated automatically in real time as the images of the eye movements are captured.