AUDIENCE STATE ESTIMATION SYSTEM, AUDIENCE STATE ESTIMATION METHOD, AND AUDIENCE STATE ESTIMATION PROGRAM

• US 20080263580A1
• Filed: 05/13/2008
• Published: 10/23/2008
• Est. Priority Date: 06/26/2002
• Status: Active Grant

• Current Assignee: Koichi Fujishima, Tetsujiro Kondo, Tomoyuki Ohtsuki, Yuji Okumura
• Original Assignee: Koichi Fujishima, Tetsujiro Kondo, Tomoyuki Ohtsuki, Yuji Okumura
• Inventors: Kondo, Tetsujiro, Ohtsuki, Tomoyuki, Fujishima, Koichi, Okumura, Yuji
• Granted Patent: US 8,244,537 B2
• Time in Patent Office: Days
• US Class Current: 725/12
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: H04H 60/33 Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44213 Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

Video signal relative to an imaged audience and audio signal according to voices from the audience are generated in an input unit. A characteristic amount detection unit detects information on a movement amount, movement periodicity, a volume, voice periodicity of the audience, and a frequency component of voices from the audience based on the video signal or the audio signal. An estimation unit estimates an audience state based on the detected result. An output unit outputs the estimated result of the audience state. The audience state can be easily estimated without observing the audience state by a person.

Claims

1-11. -11. (canceled)

12. An audience state estimation system comprising:

sound-obtaining device for obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

volume detection device for detecting a volume of the sound from said audience based on said audio signal; and

estimation device for estimating an audience state based on said volume.

13. The audience state estimation system according to claim 12, wherein said sound from the audience includes voice.

14. The audience state estimation system according to claim 12, wherein in said volume detection device, variance of the sound level of said audio signal is set as said volume.

15. The audience state estimation system according to claim 12, wherein said volume detection device calculates variance of the sound level of said audio signal, and wherein a time macro volume is set as said volume, said time macro volume being an average of the variance in a time direction thereof.

16. The audience state estimation system according to claim 12, wherein when said volume is larger than a predetermined level, said estimation device estimates said audience state to be in a state of laughing.

17. An audience state estimation system comprising:

sound-obtaining device for obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

sound periodicity detection device for detecting sound periodicity from said audience based on said audio signal; and

estimation device for estimating an audience state based on said sound periodicity.

18. The audience state estimation system according to claim 17, wherein said sound from the audience includes voice.

19. The audience state estimation system according to claim 17, wherein said sound periodicity detection device sequentially detects autocorrelation maximum positions from said audio signal, and wherein variance of the autocorrelation maximum position is set as said sound periodicity.

20. The audience state estimation system according to claim 17, further comprising volume detection device, wherein in said sound periodicity detection device, a ratio of low-frequency component in a volume of the sound from said audience detected by said volume detection device based on said audio signal is set as said sound periodicity.

21. The audience state estimation system according to claim 20, wherein a frequency range of said low-frequency component is decided according to the sound periodicity to be detected.

22. The audience state estimation system according to claim 17, wherein when said sound periodicity is larger than a predetermined level, said estimation device estimates said audience state to be in a state of clapping.

23. An audience state estimation system comprising:

sound-obtaining device for obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

spectrum detection device for performing Fourier transform on said audio signal and detecting the center of gravity of the Fourier transformed result; and

estimation device for estimating an audience state based on said center of gravity.

24. The audience state estimation system according to claim 23, wherein said sound from the audience includes voice.

25. The audience state estimation system according to claim 23, wherein when said center of gravity is larger than a predetermined level, said estimation device estimates said audience state to be in any one of states of beating time with the hands and of clapping.

26. An audience state estimation system comprising:

sound-obtaining device for obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

component ratio detection device for performing Fourier transform on said audio signal and detecting a high-frequency to low-frequency component ratio by calculating said ratio based on the Fourier transformed result; and

estimation device for estimating an audience state based on said high-frequency to low-frequency component ratio.

27. The audience state estimation system according to claim 26, wherein said sound from the audience includes voice.

28. The audience state estimation system according to claim 26, wherein when in said high-frequency to low-frequency component ratio, a ratio of said high-frequency component to said low-frequency component is larger than a predetermined level, said estimation device estimates said audience state to be in any one of states of beating time with the hands and of clapping.

29-38. -38. (canceled)

39. An audience state estimation method comprising:

obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

detecting a volume of the sound from said audience based on said audio signal; and

estimating an audience state based on said volume.

40. The audience state estimation method according to claim 39, wherein said sound from the audience includes voice.

41. The audience state estimation method according to claim 39, wherein variance of the sound level of said audio signal is set as said volume.

42. The audience state estimation method according to claim 39, wherein variance of the sound levels of said audio signals is calculated, and wherein a time macro volume is set as said volume, said time macro volume being an average of the variance in the time direction thereof.

43. The audience state estimation method according to claim 39, wherein when said volume is larger than a predetermined level, said audience state is estimated to be in a state of laughing.

44. An audience state estimation method comprising:

obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

detecting sound periodicity from said audience based on said audio signal; and

estimating an audience state based on said sound periodicity.

45. The audience state estimation method according to claim 44, wherein said sound from the audience includes voice.

46. The audience state estimation method according to claim 44, wherein autocorrelation maximum positions are sequentially detected from said volume, and wherein variance of the autocorrelation maximum position is set as said sound periodicity.

47. The audience state estimation method according to claim 44, wherein a ratio of low-frequency component in the volume of the sound from said audience detected based on said audio signal is set as said sound periodicity.

48. The audience state estimation method according to claim 44, wherein when said sound periodicity is larger than a predetermined level, said audience state is estimated to be in a state of clapping.

49. An audience state estimation method comprising:

obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

performing Fourier transform on said audio signal and detecting the center of gravity of the Fourier transformed result; and

estimating an audience state based on said center of gravity.

50. The audience state estimation method according to claim 49, wherein said sound from the audience includes voice.

51. The audience state estimation method according to claim 50, wherein when said center of gravity is larger than a predetermined level, said audience state is estimated to be in any one of states of beating time with the hands and of clapping.

52. An audience state estimation method comprising:

obtaining sound from an audience and generating an audio signal according to the sound thus obtained;

performing Fourier transform on said audio signal and calculating a high-frequency to low-frequency component ratio based on the Fourier transformed result; and

estimating an audience state based on said high-frequency to low-frequency component ratio.

53. The audience state estimation method according to claim 52, wherein said sound from the audience includes voice.

54. The audience state estimation method according to claim 52, wherein when in said high-frequency to low-frequency component ratio, a ratio of said high-frequency component to said low-frequency component is larger than a predetermined level, said audience state is estimated to be in any one of states of beating time with the hands and of clapping.

55-58. -58. (canceled)