Physiological cue processing

• US 9,640,218 B2
• Filed: 12/07/2012
• Issued: 05/02/2017
• Est. Priority Date: 12/07/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Shoemaker, Garth, Thompson, Jonathan
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US13/707,697
• Publication Number:
  US 20140161421A1
• Time in Patent Office: 1,607 Days
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0806 by whole-body plethysmography measuring blood flow using plethysmography A61B5/0295; impedance plethysmography A61B5/053; measuring volume of the body or parts thereof A61B5/1073
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G11B 27/028 with computer assistance
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Physiological cues, such as elevated heart or respiratory rates can be masked from video of a person before the video is sent to another party. The masking of human-perceptible and non-human-perceptible physiological cues removes information from the video that another party could use to determine the person'"'"'s emotional state. For example, variations in a person'"'"'s skin color that are typically imperceptible to the human eye and from which a person'"'"'s heart rate can be detected, can be removed or altered so that another party viewing the video cannot determine the person'"'"'s actual heart rate in an attempt to determine the person'"'"'s emotional state, even if they are performing computer analysis on the video. The presence of some physiological cues can be determined by detecting that a physiological measure is above a specified physiological measure threshold. Modified video can have the characteristic that physiological cues are substantially absent in the modified video or that physiological measures extracted from the modified video have a value below an associated physiological measure threshold.

Claims

1. One or more non-transitory computer-readable storage media storing computer-executable instructions for causing a computing device to perform a method of masking physiological cues, the method comprising:

receiving source video of at least one person, the source video comprising one or more physiological cues;

determining a physiological measure of the person from the source video;

determining whether the physiological measure satisfies a predefined threshold indicative of one or more of the physiological cues;

generating, in response to a determination that the physiological measure determined from the source video satisfies the predefined threshold, modified video in which at least one of the one or more physiological cues is masked; and

sending the modified video to a second computing device.

2. The one or more non-transitory computer-readable storage media of claim 1, wherein the physiological measure is a heart rate, the physiological measure threshold is a heart rate threshold, and the detecting comprises determining the heart rate based at least in part on fluctuations in average pixel intensities in one or more regions of the person'"'"'s face in the source video.

3. The one or more non-transitory computer-readable storage media of claim 2, wherein fluctuations in average pixel intensities in the one or more regions of the person'"'"'s face in the modified video fluctuate at a different rate than the heart rate determined from the source video.

4. The one or more non-transitory computer-readable storage media of claim 2, wherein fluctuations in average pixel intensities in the one or more regions of the person'"'"'s face in the modified video fluctuate at a rate below the heart rate threshold.

5. The one or more computer-readable storage media of claim 1, wherein the physiological measure is a respiratory rate, the physiological measure threshold is a respiratory rate threshold, and the detecting comprises determining the respiratory rate of the person based on a rate at which the chest and/or shoulders of the person rise and fall in the source video.

6. The one or more computer-readable storage media of claim 5, wherein a rate at which the shoulder and/or chest of the person rise and fall in the modified video is different from the rate at which the shoulders and/or chest of the person rise and fall in the source video.

7. The one or more computer-readable storage media of claim 5, wherein the shoulders and chest of the person do not substantially rise or fall in the modified video.

8. The one or more computer-readable storage media of claim 1, wherein the detecting comprises:

determining a skin redness of the person based at least in part on pixel intensities in one or more regions of the person'"'"'s face in the source video; and

determining that the skin redness exceeds a skin redness baseline by more than a skin redness shift threshold.

9. The one or more computer-readable storage media of claim 8, wherein a skin redness in the one or more regions of the person'"'"'s face in the modified video is different than the skin redness determined from the source video.

10. The one or more computer-readable storage media of claim 8, wherein a skin redness in the one or more regions of the person'"'"'s face in the modified video is within the skin redness shift threshold of the skin redness baseline.

11. The one or more computer-readable storage media of claim 1, wherein the detecting comprises identifying specular reflections in one or more regions of the person'"'"'s face in the source video, the generating the modified video comprising smoothing the one or more regions of the person'"'"'s face to at least reduce the intensity of the identified specular reflections.

12. The one or more non-transitory computer-readable storage media of claim 1, wherein the generating the modified video comprises substantially removing nervous tics and/or microexpressions from the source video.

13. The one or more non-transitory computer-readable storage media of claim 1, wherein the generating the modified video comprises passing the source video through a filter that substantially removes nervous tics or microexpressions lasting approximately 100 ms or less.

14. The one or more non-transitory computer-readable storage media of claim 1, wherein generating the modified video comprises to:

generate an intermediate video by removing a component of the source video, wherein the component is indicative of the physiological measure; and

add a signal to the intermediate video, wherein the signal is indicative of a physiological measure that does not satisfy the predefined threshold.

15. The one or more non-transitory computer-readable storage media of claim 1, wherein generating the modified video comprises to smooth one or more specular highlights on the skin of a person in the source video.

16. The one or more non-transitory computer-readable storage media of claim 1, wherein generating the modified video comprises to add one or more compression artifacts to the source video.

17. A method of masking physiological cues, the method comprising:

receiving source video of at least one person, the source video comprising one or more physiological cues;

determining a physiological measure of the person from the source video;

determining whether the physiological measure satisfies a predefined threshold indicative of one or more of the physiological cues;

generating, in response to a determination that the physiological measure satisfies the predefined threshold determined from the source video, modified video in which at least one of the one or more physiological cues is masked; and

sending the modified video to a second computing device.

18. A method comprising:

sending computer-executable instructions to one or more computing devices to cause the one or more computing devices to perform a method, the method comprising;

receiving source video of at least one person, the source video comprising one or more physiological cues;

determining a physiological measure of the person from the source video;

determining whether the physiological measure satisfies a predefined threshold indicative of one or more of the physiological cues;

generating, in response to a determination that the physiological measure determined from the source video satisfies the predefined threshold, modified video in which at least one of the one or more physiological cues is masked;

sending the modified video to a second computing device; and

storing the computer-executable instructions at the one or more computing devices.

19. One or more computing devices programmed to perform a method, the method comprising:

receiving source video of at least one person, the source video comprising one or more physiological cues;

determining a physiological measure of the person from the source video;

determining whether the physiological measure satisfies a predefined threshold indicative of one or more of the physiological cues;

generating, in response to a determination that the physiological measure determined from the source video satisfies the predefined threshold, modified video in which at least one of the one or more physiological cues is masked; and

sending the modified video to a second computing device.

20. The one or more computing devices of claim 19, wherein the modified video has the characteristic that the physiological measure as determined from the modified video is different from the physiological measure as determined from the source video.

21. The one or more computing devices of claim 19, wherein the physiological measure is a heart rate, the physiological measure threshold is a heart rate threshold, and the detecting comprises determining the heart rate based at least in part on fluctuations in average pixel intensities in one or more regions of the person'"'"'s face in the source video.

22. The one or more computing devices of claim 21, wherein fluctuations in average pixel intensities in the one or more regions of the person'"'"'s face in the modified video fluctuate at a different rate than the heart rate determined from the source video.

23. The one or more computing devices of claim 21, wherein fluctuations in average pixel intensities in the one or more regions of the person'"'"'s face in the modified video fluctuate at a rate below the heart rate threshold.

24. The one or more computing devices of claim 21, wherein the fluctuations in average pixel intensities in the one or more regions of the person'"'"'s face in the source video are substantially absent from the modified video.