System, method and computer program product for human presence detection based on audio

• US 10,082,574 B2
• Filed: 08/25/2011
• Issued: 09/25/2018
• Est. Priority Date: 08/25/2011
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Wouhaybi, Rita H., Yarvis, Mark D., Garg, Sharad K.
• Primary Examiner(s): Wang, Quan-Zhen
• Assistant Examiner(s): BLACK-CHILDRESS, RAJSHEED O
• Application Number:
  US13/994,770
• Publication Number:
  US 20140340992A1
• Time in Patent Office: 2,588 Days
• Field of Search: None
• CPC Class Codes: G01S 15/04 Systems determining presence of a target
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

Methods, systems and computer program products that allow for the determination of human presence in a room where content is being presented. The audio that is associated with the content may be captured, along with the audio that is being generated collectively by whatever sources may be in the room including the presentation of the content. Features may be extracted from both the content audio and the room audio. These features may then be compared, and the differences may be quantified. If the differences are significant, then human presence may be inferred.

Claims

1. A machine-implemented method, comprising:

sampling a content audio from a content;

sampling room audio from a microphone proximate to a consumer electronics device, the room audio comprising all sounds generated in a room, including the content audio;

dividing each of the sampled content audio and the sampled room audio into intervals;

computing a coefficient of variation for each of the intervals of each of the content audio and the room audio, by dividing a standard deviation of the interval by a mean of the interval, if the mean is non-zero;

comparing the coefficient of variation of each interval of the room audio with the corresponding coefficient of variation of the content audio; and

determining that a person is present if a difference between the coefficients of variation of a predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio exceeds a threshold.

2. The machine-implemented method of claim 1, wherein the determining includes:

inferring that the person is being exposed to content presented at the consumer electronics device if the difference between the coefficients of variation of the content audio and the coefficients of variation of the room audio exceeds the threshold.

3. The machine-implemented method of claim 1, further including:

normalizing results of the comparing over the predetermined number of intervals; and

performing the determining based on the normalized results of the comparing.

4. The machine-implemented method of claim 3, wherein, for each interval:

the comparing includes determining a percentage difference between the coefficients of variation of the predetermined number of intervals of the room audio and the corresponding coefficients of variation of the content audio; and

the normalizing includes averaging the percentage difference of the intervals with the percentage differences of a predetermined number of preceding intervals.

5. The machine-implemented method of claim 1, wherein the computing includes, for each interval of the room audio and the content audio:

determining the standard deviation of the interval; and

determining the mean of the interval.

6. The machine-implemented method of claim 1, wherein the determining includes:

determining a level of confidence that the person is present based upon an extent to which the difference between the coefficients of variation of the content audio and the coefficients of variation of the room audio exceeds the threshold.

7. The machine-implemented method of claim 1, further comprising:

generating a frequency domain representation of each interval of the room audio and the content audio; and

wherein;

the computing includes computing a coefficient of variation of the frequency domain representation of each interval of the content audio and the room audio; and

the determining includes determining that the person is present if a difference between the coefficients of variation of the predetermined number of intervals of the frequency domain representations of the content audio and the corresponding coefficients of variation of the frequency domain representation of the room audio exceeds the threshold.

8. The machine-implemented method of claim 7, further comprising:

band pass filtering the frequency domain representation of each interval of the content audio and the room audio to remove frequencies outside of a frequency range of human speech.

9. The machine-implemented method of claim 1, further including:

placing the consumer electronics device in a reduced power consumption state if the difference between the coefficients of variation of the predetermined number of intervals of content audio and the corresponding coefficients of variation of the room audio is below the threshold.

10. The machine-implemented method of claim 9, further including:

pausing presentation of content at the consumer electronics device if the difference between the coefficients of variation of the predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio is below the threshold.

11. An apparatus, comprising:

a programmable integrated circuit (PIC); and

an execution engine in communication with the PIC;

wherein the PIC is to;

sample a content audio from a content;

sample room audio from a microphone proximate to a consumer electronics device, the room audio comprising all sounds generated in a room, including the content audio;

divide each of the sampled content audio and the sampled room audio into intervals; and

compute a coefficient of variation for each interval of each of the content audio and the room audio, by dividing a standard deviation of the interval by a mean of the interval, if the mean is non-zero;

and wherein the execution engine is to;

compare each coefficient of variation of the room audio with the corresponding coefficient of variation of the content audio; and

determine that a person is present if a difference between the coefficients of variation of a predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio exceeds a threshold.

12. The apparatus of claim 11, wherein the execution engine is further to:

infer that the person is being exposed to content presented at the consumer electronics device if the difference between the coefficients of variation of the predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio exceeds the threshold.

13. The apparatus of claim 11, wherein the execution engine is further to:

normalize results of the comparisons over the predetermined number of intervals; and

determine that the person is present based on the normalized results of the comparisons.

14. The apparatus of claim 13, wherein the execution engine is further to, for each interval of the predetermined number of intervals:

determine a percentage difference between the coefficient of variation of the room audio and the corresponding coefficient of variation of the content audio; and

average the percentage difference of the interval with the percentage differences of a predetermined number of preceding intervals.

15. The apparatus of claim 11, wherein the PIC is further to, for each interval of the room audio and the content audio:

determine the standard deviation of the interval; and

determine the mean of the interval.

16. The apparatus of claim 11, wherein the execution engine is further to:

determine a level of confidence that the person is present based upon an extent to which the difference between the coefficients of variation of the predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio exceeds the threshold.

17. The apparatus of claim 11, wherein the PIC is further to:

generate a frequency domain representation of each interval of the room audio and the content audio;

compute the coefficient of variation of each interval of the content audio and the room audio as a coefficient of variation of the respective frequency domain representation; and

the execution engine is further configured to determine that the person is present if a difference between the coefficients of variation of the frequency domain representation of the content audio and the corresponding coefficients of variation of the frequency domain representation of the room audio exceeds the threshold.

18. The apparatus of claim 17, wherein the PIC is further to:

band pass filter the frequency domain representation of each interval of the content audio and the room audio to remove frequencies outside of a frequency range of human speech.

19. The apparatus of claim 11, wherein the execution engine is further to:

place the consumer electronics device in a reduced power consumption state if the difference between the coefficients of variation of each of the predetermined intervals of the content audio and the corresponding coefficients of variation of the room audio is below the threshold.

20. The apparatus of claim 11, wherein the execution engine is further to:

pause presentation of content at the consumer electronics device if the difference between the coefficients of variation of each of the predetermined intervals of the content audio and the corresponding coefficients of variation of the room audio is below the threshold.

21. A non-transitory computer readable media encoded with a computer program that includes instructions to cause a processor to:

sample a content audio from a content;

sample room audio from a microphone proximate to a consumer electronics device, the room audio comprising all sounds generated in a room, including the content audio;

divide each of the sampled content audio and the sampled room audio into intervals;

compute a coefficient of variation for each interval of the content audio and the room audio, by dividing a standard deviation of the interval with a mean of the interval, if the mean is non-zero;

compare the coefficient of variation of each interval of the room audio with the corresponding coefficient of variation of the content audio; and

determine that a person is present if a difference between the coefficients of variation of a predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio exceeds a threshold.

22. The non-transitory computer readable media of claim 21, further including instructions to cause the processor to:

infer that the person is being exposed to content presented at the consumer electronics device if the difference between the coefficients of variation of the predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio exceeds the threshold.

23. The non-transitory computer readable media of claim 21, further including instructions to cause the processor to:

normalize results of the comparisons over the predetermined number of intervals; and

determine that the person is present based on the normalized results of the comparison.

24. The non-transitory computer readable media of claim 23, further including instructions to cause the processor to, for each interval of the predetermined number of intervals:

determine a percentage difference between the coefficient of variation of the room audio and the coefficient of variation of the content audio; and

average the percentage difference of the interval with the percentage differences of a predetermined number of preceding intervals.

25. The non-transitory computer readable media of claim 21, further including instructions to cause the processor to, for each interval of the room audio and the content audio:

determine the standard deviation of the interval; and

determine the mean of the interval.

26. The non-transitory computer readable media of claim 21, further including instructions to cause the processor to:

determine a level of confidence that the person is present based upon an extent to which the difference between the coefficients of variation of the predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio exceeds the threshold.

27. The non-transitory computer readable media of claim 21, further including instructions to cause the processor to:

generate a frequency domain representation of each interval of the room audio and the content audio;

compute the coefficient of variation of each interval of the content audio and the room audio as a coefficient of variation of the respective frequency domain representation; and

determine that the person is present if a difference between the coefficients of variation of the frequency domain representation of the content audio and the corresponding coefficients of variation of the frequency domain representation of the room audio exceeds the threshold.

28. The non-transitory computer readable media of claim 27, further including instructions to cause the processor to:

bandpass filter the frequency domain representation of each interval of the content audio and the room audio to remove frequencies outside of a frequency range of human speech.

29. The non-transitory computer readable media of claim 21, further including instructions to cause the processor to:

place the consumer electronics device in a reduced power consumption state if the difference between the coefficients of variation of the predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio is below the threshold.

30. The non-transitory computer readable media of claim 21, further including instructions to cause the processor to:

pause presentation of content at the consumer electronics device if the difference between the coefficients of variation of the predetermined number of intervals of the content audio and the corresponding coefficients of variation of the room audio is below the threshold.