IMAGE-BASED MENTAL STATE DETERMINATION

• US 20170188928A1
• Filed: 12/22/2016
• Published: 07/06/2017
• Est. Priority Date: 12/24/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Asli Arslan Esme, Cagri Tanriover, Ece Oktay, Eda Okur, Nese Alyuz Civitci
• Inventors: Tanriover, Cagri, Aslan, Sinem, Alyuz Civitci, Nese, Oktay, Ece, Okur, Eda, Arslan Esme, Asli
• Granted Patent: US 10,299,716 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/145 Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissuemeasuring of blood pressure or blood flow A61B5/02; non-radiation detecting or locating of foreign bodies in blood A61B5/06
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Apparatus, systems, and/or methods may provide a mental state determination. For example, a data collector may collect image data for a side of a face of a user from an image capture device on the user (e.g., a wearable device). The image data may include two or more perspectives of a feature on the side of the face of the user. In addition, a state determiner may determine a mental state of the user based on the image data. In one example, fields of view may be combined to determine a total region and/or a total overlap region. Changing the position that one or more image capture devices point may modulate the total region and/or the total overlap region. In addition, one or more sensors may be utilized to further improve mental data determinations.

Claims

1. A system to determine a mental state comprising:

a wearable device including one or more image capture devices that are to be moveably coupled to a user to capture image data for a side of a face of the user, wherein the image data is to include two or more perspectives of a feature on the side of the face of the user, and

a state determiner to determine a mental state of the user based on the image data.

2. The system of claim 1, wherein the image capture device is to include one or more of,

at least one image capture device moveably coupled with the user to pan across the side of the face of the user to capture the image data when the at least one image capture device is to pan, or

at least two image capture devices moveably coupled with the user to be positioned to concurrently capture the image data when the at least two image capture devices are to be at rest.

3. The system of claim 1, further including,

a collector to collect the image data,

a region determiner to determine a region on the side of the face of the user, wherein the region is to be based on two or more fields of view,

a feature identifier to identify the feature in the region,

an evaluator to one or more of,

evaluate the feature in the region based on characteristic mental state data to determine the mental state of the user, or

evaluate the feature in the region based on user specific mental state date to determine the mental state of the user, and

an adjuster to one or more of,

calibrate an angle of at least one image capture device of the one or more image capture devices for the user, or

adjust the region based on one or more of a perspective preference, a region preference, a feature preference, an accuracy preference, or an efficiency preference.

4. The system of claim 1, further including,

a physiological sensor to provide physiological data of the user,

a state evaluator to evaluate the feature and the physiological data to determine the mental state of the user, and

an interface to provide the mental state of the user to one or more of a state repository, a function, or an entity.

5. An apparatus to determine a mental state comprising:

a data collector to collect image data for a side of a face of a user from an image capture device on the user, wherein the image data is to include two or more perspectives of a feature on the side of the face of the user, and

a state determiner to determine a mental state of the user based on the image data.

6. The apparatus of claim 5, wherein the data collector is to one or more of,

collect image data including a plurality of features on the side of face of the user from the image capture device on the user, or

collect image data for a different side of the face of the user from a different image capture device on the user, wherein the image data for the different side of the face of the user is to include two or more perspectives of a feature on the different side of the face of the user.

7. The apparatus of claim 5, wherein the feature is to include a partial feature on the side of the face of the user, and wherein the image data is to include two or more perspectives of the partial feature.

8. The apparatus of claim 5, wherein a field of view of the image capture device on the user at a first angle relative to the face is to provide a first perspective, and wherein the field of view of the image capture device on the user at a second angle relative to the face is to provide a second perspective.

9. The apparatus of claim 5, wherein a field of view of a first image capture device on the user at a first angle relative to the face is to provide a first perspective, and wherein a field of view of a second image capture device on the user at a second angle relative to the face is to provide a second perspective.

10. The apparatus of claim 5, further including,

a region determiner to determine a total region on the side of the face of the user, wherein the total region is to be based on the two or more fields of view,

a feature identifier to identify the feature in the total region, and

a state evaluator to one or more of,

evaluate the feature in the total region based on characteristic mental state data to determine the mental state of the user, or

evaluate the feature in the total region based on user specific mental state data to determine the mental state of the user.

11. The apparatus of claim 10, wherein the region determiner is to determine a total overlap region on the side of the face of the user, wherein the feature identifier is to identify the feature in the total overlap region, and wherein the state evaluator is to evaluate the feature in the total overlap region.

12. The apparatus of claim 5, further including an adjuster to one or more of,

calibrate an angle of the image capture device for the user, or

adjust one or more of a total region or a total overlap region based on one or more of a perspective preference, a region preference, a feature preference, an accuracy preference, or an efficiency preference.

13. The apparatus of claim 5, further including,

a physiological data collector to collect physiological data of the user from a physiological sensor,

a state evaluator to evaluate the feature and the physiological data to determine the mental state of the user based, and

an interface to provide the mental state of the user to one or more of a state repository, a function, or an entity.

14. At least one computer readable storage medium comprising a set of instructions, which when executed by a computer, cause the computer to:

collect image data for a side of a face of a user from an image capture device on the user, wherein the image data is to include two or more perspectives of a feature on the side of the face of the user; and

determine a mental state of the user based on the image data.

15. The at least one computer readable storage medium of claim 14, wherein the instructions, when executed, cause the computer to one or more of:

collect image data including a plurality of features on the side of face of the user from the image capture device on the user;

or

collect image data for a different side of the face of the user from a different image capture device on the user, wherein the image data for the different side of the face of the user is to include two or more perspectives of a feature on the different side of the face of the user.

16. The at least one computer readable storage medium of claim 14, wherein the instructions, when executed, cause the computer to:

determine a total region on the side of the face of the user, wherein the total region is to be based on the two or more fields of view;

identify the feature in the total region; and

one or more of;

evaluate the feature in the total region based on characteristic mental state data to determine the mental state of the user;

or

evaluate the feature in the total region based on user specific mental state data to determine the mental state of the user.

17. The at least one computer readable storage medium of claim 14, wherein the instructions, when executed, cause the computer to:

determine a total overlap region on the side of the face of the user;

identify the feature in the total overlap region; and

evaluate the feature in the total overlap region.

18. The at least one computer readable storage medium of claim 14, wherein the instructions, when executed, cause the computer to one or more of:

calibrate an angle of the image capture device for the user;

or

adjust one or more of a total region or a total overlap region based on one or more of a perspective preference, a region preference, a feature preference, an accuracy preference, or an efficiency preference.

19. The at least one computer readable storage medium of claim 14, wherein the instructions, when executed, cause the computer to:

collect physiological data of the user from a physiological sensor;

evaluate the feature and the physiological data to determine the mental state of the user based; and

provide the mental state of the user to one or more of a state repository, a function, or an entity.

20. A method to determine a mental state comprising:

collecting image data for a side of a face of a user from an image capture device on the user, wherein the image data includes two or more perspectives of a feature on the side of the face of the user; and

determining a mental state of the user based on the image data.

21. The method of claim 20, further including one or more of:

collecting image data including a plurality of features on the side of face of the user from the image capture device on the user;

or

collecting image data for a different side of the face of the user from a different image capture device on the user, wherein the image data for the different side of the face of the user includes two or more perspectives of a feature on the different side of the face of the user.

22. The method of claim 20, further including:

determining a total region on the side of the face of the user, wherein the total region is based on the two or more fields of view;

identifying the feature in the total region; and

one or more of;

evaluating the feature in the total region based on characteristic mental state data to determine the mental state of the user;

or

evaluating the feature in the total region based on user specific mental state data to determine the mental state of the user.

23. The method of claim 20, further including:

determining a total overlap region on the side of the face of the user;

identifying the feature in the total overlap region; and

evaluating the feature in the total overlap region.

24. The method of claim 20, further including one or more of:

calibrating an angle of the image capture device for the user;

or

adjusting one or more of a total region or a total overlap region based on one or more of a perspective preference, a region preference, a feature preference, an accuracy preference, or an efficiency preference.

25. The method of claim 20, further including:

collecting physiological data of the user from a physiological sensor;

evaluating the feature and the physiological data to determine the mental state of the user based; and

providing the mental state of the user to one or more of a state repository, a function, or an entity.