Deception detection system and method

• US 10,779,760 B2
• Filed: 07/10/2019
• Issued: 09/22/2020
• Est. Priority Date: 02/08/2016
• Status: Active Grant

• Current Assignee: NuraLogix Corporation
• Original Assignee: NuraLogix Corporation
• Inventors: Lee, Kang, Zheng, Pu
• Primary Examiner(s): Bee, Andrew W
• Application Number:
  US16/507,565
• Publication Number:
  US 20200022631A1
• Time in Patent Office: 440 Days
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/02 Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/026 Measuring blood flow A61B3/1233, A61B3/1241 take precedence
• CPC Class Codes: A61B 5/0261 using optical means, e.g. infrared light
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/1032 Determining colour for diagnostic purposes measuring colour in general G01J3/46
• CPC Class Codes: A61B 5/14546 for measuring analytes not otherwise provided for, e.g. ions, cytochromes
• CPC Class Codes: A61B 5/1455 using optical sensors, e.g. spectral photometrical oximeters
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/164 Lie detection
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/443 Evaluating skin constituents, e.g. elastin, melanin, water
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61B 5/7405 using sound
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/044 : Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/08 : Learning methods
• CPC Class Codes: G06V 10/467 : Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/60 : relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/174 : Facial expression recognition
• CPC Class Codes: G06V 40/176 : Dynamic expression
• CPC Class Codes: G08B 5/38 : using flashing light
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/63 : for estimating an emotional state
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A system for detecting deception is provided. The system comprises a camera, an image processing unit, and a notification device. The camera is configured to capture an image sequence of a person of interest. The image processing unit is trained to determine a set of bitplanes of a plurality of images in the captured image sequence that represent the hemoglobin concentration (HC) changes of the person, and to detect the person'"'"'s invisible emotional states based on HC changes. The image processing unit is trained using a training set comprising a set of subjects for which emotional state is known. The notification device provides a notification of at least one of the person'"'"'s detected invisible emotional states.

Claims

1. A system for detecting an emotional state of a person of interest, the system comprising a processing unit and a storage device, the processing unit configurable to:

receive an image sequence captured of the person of interest;

determine hemoglobin concentration (HC) changes of the person of interest from the image sequence;

determine an emotional state associated with the person of interest using a trained classification machine learning model, the trained classification machine learning model receiving the HC changes as input, the training set for the trained classification machine learning model comprising previously determined HC changes from images of subjects with known emotional states; and

output the determined emotional state.

2. The system of claim 1, wherein the emotional state comprises one of positive, negative, and neutral.

3. The system of claim 2, wherein the trained classification machine learning model further determines an associated intensity measure for the positive emotional state and the negative emotional state, and wherein the associated intensity is outputted with the respective determined emotional state.

4. The system of claim 3, wherein the processing unit is further configurable to determine a probability of deception by the person being questioned based on the determined emotional state and the associated intensity as input to a trained deception machine learning model, and to output the determined probability of deception.

5. The system of claim 4, wherein the probability of deception is determined periodically as new images in the image sequence are received.

6. The system of claim 4, wherein the HC changes are determined using an HC machine learning model trained with a HC training set, bit values from a set of bitplanes in the captured image sequence that represent HC changes are fed as input to the HC machine learning model, the training set for the HC machine learning model comprising bit values from each bitplane of images captured from persons for which HC is known.

7. The system of claim 6, wherein the set of bitplanes being those that are determined to approximately maximize a signal-to-noise ratio (SNR).

8. The system of claim 6, wherein the HC machine learning model comprises a Long Short Term Memory (LSTM) neural network.

9. The system of claim 4, wherein outputting the determined emotional state and outputting the probability of deception collectively comprise communicating the determined emotional state and the probability of deception to a display for presentation to an attendant.

10. A method for detecting an emotional state of a person of interest, the method executed on a processing unit, the method comprising:

receiving an image sequence captured of the person of interest;

determining hemoglobin concentration (HC) changes of the person of interest from the image sequence;

determining an emotional state associated with the person of interest using a trained classification machine learning model, the trained classification machine learning model receiving the HC changes as input, the training set for the trained classification machine learning model comprising previously determined HC changes from images of subjects with known emotional states; and

outputting the determined emotional state.

11. The method of claim 10, wherein the emotional state comprises one of positive, negative, and neutral.

12. The method of claim 11, wherein the trained classification machine learning model further determines an associated intensity measure for the positive emotional state and the negative emotional state, and wherein the associated intensity is outputted with the respective determined emotional state.

13. The method of claim 12, further comprising determining a probability of deception by the person being questioned based on the determined emotional state and the associated intensity as input to a trained deception machine learning model, and outputting the determined probability of deception.

14. The method of claim 13, wherein the probability of deception is determined periodically as new images in the image sequence are received.

15. The method of claim 13, wherein the HC changes are determined using an HC machine learning model trained with a HC training set, bit values from a set of bitplanes in the captured image sequence that represent HC changes are fed as input to the HC machine learning model, the training set for the HC machine learning model comprising bit values from each bitplane of images captured from persons for which HC is known.

16. The method of claim 15, wherein the set of bitplanes being those that are determined to approximately maximize a signal-to-noise ratio (SNR).

17. The method of claim 15, wherein the HC machine learning model comprises a Long Short Term Memory (LSTM) neural network.

18. The method of claim 13, wherein outputting the determined emotional state and outputting the probability of deception collectively comprise displaying the determined emotional state and the probability of deception to an attendant.