Affective television monitoring and control

• US 20030093784A1
• Filed: 11/13/2001
• Published: 05/15/2003
• Est. Priority Date: 11/13/2001
• Status: Active Grant

• Current Assignee: Sisvel S.p.A. (Fineur International S.A.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Zimmerman, John, Dimitrova, Nevenka, Agnihotri, Lalitha
• Granted Patent: US 8,561,095 B2
• Time in Patent Office: Days
• US Class Current: 725/10
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/466 Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 7/163 by receiver means only
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A system and method for collecting, analyzing, and using sensory reactions and involuntary or spontaneous movements by members of a television viewing (or listening) audience. While known programming is displayed on a television receiver, a plurality of sensors monitor the viewer or viewers for recognizable evidence of an emotional response that can be associated with a discrete program segment. Where positive (or negative) responses can be associated with a certain type of program content, the system monitors subsequent programs for the opportunity to notify the viewer or simply present (or avoid presenting) the program automatically.

Claims

1. A system for assessing viewer response to television programming that can be associated with information describing the programming content, said system comprising:

a receiver for receiving and displaying television programming;

at least one sensor for sensing a physical reaction by the viewer viewing the displayed programming and generating a signal representative of the physical reaction; and

a processor for receiving the sensor signal and analyzing it to determine if it can be associated with at least one recognizable viewer response, whereupon the processor associates the recognized response with a descriptive information relating to the program that was being displayed when the physical reaction was sensed.

2. The system of claim 1, further comprising a memory device for storing the association between the programming description and sensed reaction as a viewer preference.

3. The system according to claim 1, wherein the at least one sensor comprises a plurality of sensors.

4. The system of claim 3, further comprising a sensor-signal receiver for receiving the signals generated by at least some of the plurality of sensors.

5. The system of claim 4, wherein the sensor-signal receiver combines the received sensor signals so that the processor may analyze an aggregation of sensors signals.

6. The system of claim 1, wherein the at least one sensor is a video image capturing device.

7. The system of claim 6, further comprising a video processor in communication with the video camera, the video processor comprising an image library for comparing to video images received from the video camera.

8. The system of claim 6, further comprising a video recorder for recording the images captured by the video camera.

9. The system of claim 1, wherein the at least one sensor is a microphone for picking up vocalizations made by the viewer.

10. The system of claim 1, further comprising an environmental sensor for sensing a change in the viewing environment and transmitting environmental information to the processor for use in analyzing viewer physical reactions.

11. A method of assessing viewer response to television programming that includes one or more distinct segments, said method comprising the steps of:

providing a receiver on which the viewer may view the programming;

monitoring at least one viewer physical condition; and

associating a perceived physical-condition status with a viewer response.

12. The method of claim 11, wherein in the monitoring step includes monitoring a plurality of viewer physical conditions.

13. The method of claim 11, wherein the physical condition status includes a change in the physical condition relative to a baseline level, the change being perceived during the monitoring step.

14. The method of claim 11, wherein the physical condition is body temperature.

15. The method of claim 11, wherein the physical condition is heart rate.

16. The method of claim 11, wherein the monitoring step is performed by an image-capturing device for capturing images of the viewer viewing the programming.

17. The method of claim 16, further comprising the step of providing a video processor for receiving the video images captured by the video camera and comparing them to reference data to interpret a viewer movement represented in the captured images.

18. The method of claim 11, further comprising the steps of:

determining at least one distinguishing characteristic of a displayed programming segment;

associating a viewer response corresponding to a physical condition perceived during the display of the programming segment with a viewer preference level; and

applying the preference level to enhance program selection.

19. The method of claim 18, wherein program selection is enhanced by providing a notification that specified future programming will contain at least one segment possessing the at least one distinguishing characteristic.

20. The method of claim 18, wherein the program selection is enhanced by inserting into a program a segment possessing the at least one distinguishing characteristic.

21. The method of claim 18, wherein the program distinguishing characteristic is derived from electronic program guide (EPG) information provided with the television programming.

22. The method of claim 18, wherein the program segment distinguishing characteristic is derived from audio, video and text signal properties of television programming.

23. The method of claim 11, further comprising the steps of:

providing a recorder coupled to the receiver to record selected program segments;

determining when a program segment is being received that corresponds to a pre-selected viewer response previously associated with a physical-condition status; and

automatically recording the program segment.

24. The method of claim 11, further comprising the steps of:

determining when a program segment is being received that corresponds to a pre-selected viewer response previously associated with a physical-condition status;

extracting information related to the program segment from the television programming; and

automatically displaying the information on the receiver.

25. The method of claim 11, wherein the monitored physical condition viewer physical condition is a biometric response.

26. The method of claim 25, wherein the biometric response is galvactic skin response.

27. The method of claim 11, wherein the monitoring step comprises monitoring a visually observable response.

28. The method of claim 27, wherein the visually observable response is related to the gaze of the viewer.

29. The method of claim 28, wherein the gaze-related response includes the direction of the viewer'"'"'s gaze.

30. The method of claim 28, wherein the gaze-related response includes the duration of the viewer'"'"'s gaze in a certain direction before changing to a different direction.

31. The method of claim 27, wherein the visually observable response includes the furrowing of the viewer'"'"'s brow.

32. The method of claim 31, wherein the monitoring step includes measuring the depth of any furrows in the viewer'"'"'s brow, tending to indicate confusion or lack of understanding.

33. The method of claim 11, wherein the associating step is performed at least in part by using the Hidden Markov Model technique.

34. A method of assessing listener response to audio programming that includes one or more distinct segments, said method comprising the steps of:

providing a receiver having a speaker for presenting the audio programming to the listener;

monitoring at least one listener physical condition; and

associating a perceived physical-condition status with a viewer response.

35. The method of claim 34, wherein the monitoring step comprises monitoring a audibly observable response.

36. The method of claim 34, wherein the audibly observable response is listener laughter.

37. The method of claim 34, wherein the audibly observable response is the inflection of a listener'"'"'s vocalization, tending to indicate a question has been enunciated.

38. The method of claim 34, wherein the associating step is performed at least in part by using the Hidden Markov Model technique.