Security system that detects atypical behavior

• US 20180116578A1
• Filed: 01/02/2018
• Published: 05/03/2018
• Est. Priority Date: 06/14/2015
• Status: Active Grant

• Current Assignee: Facense Ltd.
• Original Assignee: Facense Ltd.
• Inventors: Tzvieli, Arie, Thieberger, Gil, Frank, Ari M
• Granted Patent: US 10,154,810 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2560/0252 using ambient temperature
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/0533 Measuring galvanic skin response
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/6814 Head
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/7445 Display arrangements, e.g. multiple display units
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: A61B 5/748 : Selection of a region of interest, e.g. using a graphics tablet

Abstract

System and method that identify atypical behavior of a user. One embodiment of the system includes an eye tracker to perform tracking of a user'"'"'s gaze while viewing items, an inward-facing head-mounted thermal camera to take thermal measurements of a region of interest on the face (TH_ROI) of the user, and a computer. The computer generates feature values based on TH_ROI and the tracking, and utilizes a model to identify atypical behavior of the user based on the feature values. The model may be trained based on previous tracking and previous TH_ROI of the user, taken while viewing other items.

Claims

1. A system configured to identify atypical behavior, comprising:

an eye tracker configured to perform tracking of a user'"'"'s gaze while viewing items;

an inward-facing head-mounted thermal camera (CAM) configured to take thermal measurements of a region of interest on the face (TH_ROI) of the user; and

a computer configured to;

generate feature values based on TH_ROI and the tracking; and

utilize a model to identify atypical behavior of the user based on the feature values;

wherein the model was trained based on previous tracking and previous TH_ROI of the user, taken while viewing other items.

2. The system of claim 1, further comprising a frame configured to be worn on a user'"'"'s head;

wherein the eye tracker and CAM are coupled to the frame, the region of interest is on a periorbital area of the face, and CAM is located less than 15 cm from the face.

3. The system of claim 2, further comprising a second inward-facing head-mounted thermal camera configured to take thermal measurements of a region on the forehead (TH_ROI2);

and wherein the computer is further configured to generate one or more of the feature values also based on TH_ROI2.

4. The system of claim 2, further comprising a second inward-facing head-mounted thermal camera configured to take thermal measurements of a region on the nose (TH_ROI2);

and wherein the computer is further configured to generate one or more of the feature values also based on TH_ROI2.

5. The system of claim 1, wherein the user'"'"'s behavior during most of the viewing of the other items was typical, and the previous TH_ROI were taken during different days and in different situations that include first TH_ROI taken at most one hour before having a meal and second TH_ROI taken at most one hour after having a meal.

6. The system of claim 1, wherein when the user'"'"'s behavior while viewing the items is atypical, a stress level of the user during the viewing, which is calculated based on TH_ROI, reaches a threshold;

and wherein stress levels of the user during most of the time spent viewing the other items, as calculated based on the previous TH_ROI, do not reach the threshold.

7. The system of claim 1, wherein the user'"'"'s gaze is indicative of attention of the user in the items.

8. The system of claim 7, wherein the computer is further configured to generate additional feature values indicative of expected attention of the user in the items and to utilize the additional feature values to identify the atypical behavior;

wherein when the user'"'"'s behavior while viewing the items is atypical, a divergence between the attention of the user in the items and the expected attention of the user in the items is above a certain value; and

wherein during most of the time spent while viewing the other items, divergences between attention of the user in the other items and expected attention levels in the other items were not above the certain value.

9. The system of claim 8, wherein the expected attention levels in the other items are calculated utilizing a certain model of the user;

and wherein the certain model was trained based on previous tracking of the user'"'"'s gaze.

10. The system of claim 8, wherein the expected attention levels in the other items are calculated utilizing a certain model;

and wherein the certain model was trained based on tracking of other users gaze.

11. The system of claim 8, wherein the expected attention levels in the other items are calculated using a saliency mapping algorithm.

12. The system of claim 1, wherein the computer is further configured to generate one or more of the feature values based on a difference between TH_ROI and a baseline value determined based on a set of previous measurements taken by CAM;

and wherein most of the measurements belonging to the set were taken while the user'"'"'s behavior was not considered atypical according to the model.

13. The system of claim 1, further comprising a head-mounted display (HMD) configured to expose the user to video depicting the items;

and wherein the computer is further configured to;

(i) receive one or more values indicative of at least one of the following parameters of the user;

heart rate, heart rate variability, galvanic skin response, respiratory rate, and respiratory rate variability, and (ii) to generate some of the feature values based on the one or more values.

14. The system of claim 1, wherein the computer is further configured to:

(i) receive one or more values measured by at least one of the following sensors coupled to the user;

a photoplethysmogram (PPG) sensor, an electrocardiogram (ECG) sensor, an electroencephalography (EEG) sensor, a galvanic skin response (GSR) sensor, and a thermistor, and (ii) to generate some of the feature values based on the one or more values.

15. The system of claim 1, wherein the computer is further configured to:

(i) receive one or more values indicative of at least one of the following;

whether the user touched at least one of the eyes, and whether the user is engaged in physical activity, and (ii) to generate some of the feature values based on the one or more values.

16. The system of claim 1, wherein the computer is further configured to:

(i) receive one or more values measured by at least one of the following sensors;

an accelerometer, a pedometer, a humidity sensor, a miniature radar, a miniature active electro-optics distance measurement device, an anemometer, an acoustic sensor, and a light meter, and (ii) to generate some of the feature values based on the one or more values.

17. A method for identifying atypical behavior, comprising:

tracking a user'"'"'s gaze while viewing items;

taking, using an inward-facing head-mounted thermal camera, thermal measurements of a region of interest on the face (TH_ROI) of a user;

generating feature values based on TH_ROI and the tracking; and

utilizing a model to identify, based on the feature values, atypical behavior of the user;

wherein the model was trained based on previous tracking and previous TH_ROI of the user, taken while viewing other items.

18. The method of claim 17, further comprising calculating expected attention of the user in the items and generating at least some of the feature values based on the expected attention of the user in the items;

wherein the user'"'"'s gaze is indicative of attention of the user in the items, and when the user'"'"'s behavior while viewing the items is atypical, a divergence between the attention of the user in the items and the expected attention of the user in the items is above a certain value.

19. The method of claim 17, wherein the region of interest is on a periorbital area of the face, and further comprising taking, using a second inward-facing head-mounted thermal camera, thermal measurements of a region on the forehead (TH_ROI2);

and generating one or more of the feature values also based on TH_ROI2.

20. The method of claim 17, wherein the region of interest is on a periorbital area of the face, and further comprising taking, using a second inward-facing head-mounted thermal camera, thermal measurements of a region on the nose (TH_ROI2);

and generating one or more of the feature values also based on TH_ROI2.