System for parental control in video programs based on multimedia content information

• US 8,949,878 B2
• Filed: 03/30/2001
• Issued: 02/03/2015
• Est. Priority Date: 03/30/2001
• Status: Active Grant

• Current Assignee: Maxell, Ltd.
• Original Assignee: Funai Electric Co., Ltd.
• Inventors: Dimitrova, Nevenka, Jasinschi, Radu
• Primary Examiner(s): SHANG, ANNAN Q
• Application Number:
  US09/822,436
• Publication Number:
  US 20020147782A1
• Time in Patent Office: 5,058 Days
• Field of Search: 725 25- 31, 709/229
• US Class Current: 725/28
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4542 Blocking scenes or portions of the received content, e.g. censoring scenes
• CPC Class Codes: H04N 21/466 Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/4755 for defining user preferences, e.g. favourite actors or genre retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

A parental control system provides the ability to automatically filter a multimedia program content in real time based on stock and user specified criteria. The criteria are used to teach a learning module in the system what types of video program segments are to be considered sensitive or objectionable so that the module'"'"'s understanding of what is sensitive and what is not can be applied to other video programs to provide real-time filtering. The multimedia program is broken down into audio, video, and transcript components so that sound effects, visual components and objects, and language can all be analyzed collectively to make a determination of whether offending material is being passed along in the multimedia program. The user has the option of training the system for any type of objectionable material, not just sex and violence.

Claims

1. A method performed by a device to filter objectionable material, from a multimedia program including plural segments, on a segment-by-segment basis responsive to a filter criteria, comprising:

providing a learning module to generate the filter criteria learned based on user instructions by examples of objectionable content;

splitting the multimedia program into plural components;

extracting audio, video, and transcript features from segments within the plural components;

processing each segment amongst the segments, according to the learned filter criteria generated by the learning module and the extracted features, and generating a numeric ranking corresponding to the filter criteria learned based on the user instructions by examples of objectionable content and applied to the segment; and

when the respective numeric ranking for the segment exceeds a threshold, processing the segment to thereby eliminate material corresponding to the learned filter criteria.

2. The method as recited in claim 1, wherein:

the filter criteria corresponds to language included in the segment being processed; and

the audio portion of the segment is modified during the processing step.

3. The method as recited in claim 1, wherein:

the filter criteria corresponds to an image included in the segment being processed; and

the video portion of the segment is modified during the processing step.

4. The method as recited in claim 1, wherein:

the filter criteria corresponds to an image included in the segment being processed; and

the entire segment is skipped during the processing step.

5. The method as recited m claim 1, wherein the numeric ranking is a weighted numeric ranking.

6. The method as recited in claim 5, wherein each weighting factor employed in generating the weighted numeric ranking identifies a characteristic of a respective viewer of the multimedia program.

7. The method as recited in claim 1, wherein the numeric ranking for each segment is generated by comparing the content of each segment to the filter criteria.

8. The method as recited in claim 1, further comprising:

when the numeric rankings for proximate ones of the segments each exceed the threshold, merging the proximate ones of the segments and any intervening segments to thereby produce a merged segment; and

wherein the processing step comprises processing the merged segment to thereby eliminate material corresponding to the filter criteria.

9. The method as recited in claim 1, wherein:

the filter criteria comprises first and second filter criteria;

the generating step comprises generating first and second numeric rankings for respective first and second filter criteria for each of the segments;

the method further comprising the steps of;

when the respective first numeric ranking for the segment exceeds a first threshold, processing the segment to thereby eliminate material corresponding to the first filter criteria;

when the respective second numeric ranking for the segment exceeds a second threshold, processing the segment to thereby eliminate material corresponding to the second filter criteria.

10. The method recited in claim 9, wherein the first filter criteria is associated with a first passive user and wherein the second filter criteria is associated with a second passive user.

11. The method as recited in claim 10, wherein:

the first filter criteria comprises a first set of filter criteria;

the second filter criteria comprises a second set of filter criteria; and

the first set of filter criteria is a subset of the second set of filter criteria.

12. The method as recited in claim 1, further comprising:

providing training segments having content indicated by the user to be objectionable; and

learning, by the learning module from the training segments, to identify objectionable content matching the filter criteria,

wherein the learning step is performed by the electronic device.

13. The method as recited m claim 12, wherein the device comprises a software device.

14. The method as recited in claim 12, further comprising the steps of:

reviewing results generated during performance of the extracting and generating steps; and

providing feedback to the device corresponding to a review of the results by a controlling user.

15. A parental control system for an electronic device to filter objectionable material from a multimedia program including plural segments, comprising:

a learning module to generate filter criteria learned based on user instructions by examples of objectionable content;

a splitting mechanism that splits the multimedia program into plural components;

a transcript analysis module that extracts first audible features and text from a transcript analysis component within the plural components;

a visual analysis module that extracts video features from a visual analysis component within the plural components;

an audio analysis module that extracts second audible features from an audio analysis component within the plural components;

an analyzer which processes each segment amongst the segments, according to the learned filter criteria generated by the learning module and the extracted features, and generates a numeric ranking corresponding to the filter criteria learned based on the user instructions by examples of objectionable content and applied to the segment, and which generates a respective control signal when the numeric ranking exceeds a threshold; and

a filter, which processes one of the segments of the multimedia program in response to a received respective control signal.

16. The parental control system as recited in claim 15, wherein the filter modifies one of the first and second audible features of the respective segment.

17. The parental control system as recited in claim 15, wherein the filter modifies the video feature of the respective segment.

18. The parental control system as recited in claim 15, wherein the filter eliminates the respective segment from the filtered multimedia program output by the parental control system.

19. The parental control system as recited in claim 15, wherein:

numeric ranking is a weighted numeric ranking;

the analyzer employs a weight factor in generating the weighted numeric factor; and

the weighting factor corresponds to a characteristic of the intended viewer of the multimedia program.

20. The parental control system as recited in claim 19, wherein the weighting factor is selectable from a plurality of weighting factors.

21. The parental control system as recited in claim 15, wherein selected ones of the first audible features and text extracted by the transcript analysis module, the video features extracted by the visual analysis module, the second audible features extracted by the audio analysis module and user data provided by a controlling user of the parental control system are employed by the learning module to generate the filter criteria.

22. The parental control system as recited in claim 21, wherein the learning module comprises a neural network.

23. The parental control system as recited in claim 21, wherein the learning module instantiates a genetic algorithm.

24. A television set incorporating the parental control system as cited in claim 15.

25. A settop box incorporating the parental control system as recited in claim 15.

26. A personal video recorder incorporating the parental control system as recited in claim 15.

27. A client software device incorporating the parental control system as recited in claim 15.

28. A device for eliminating objectionable material from a multimedia program, the device comprising:

a learning module to generate filter criteria learned based on user instructions by examples of objectionable content;

circuitry configured to extract at least one of audio, video and text features from the multimedia program;

circuitry configured to process each segment amongst the segments, according to the learned filter criteria generated by the learning module and the extracted features, and generate a numeric ranking corresponding to the filter criteria learned based on the user instructions by examples of objectionable content and applied to the segment; and

circuitry configured to eliminate, in accordance with the numeric ranking, objectionable material from the multimedia program on a segment-by-segment basis.

29. The device of claim 28, where the circuitry configured to extract at least one of audio, video and text features is configured to extract at least two types of features selected from the set of audio features, video features or text features.

30. The device of claim 28, where the circuitry configured to extract at least one of audio, video and text features is configured to extract each of audio features, video features and text features from the multimedia program.

31. The device of claim 28, where the circuitry configured to extract at least one of audio, video and text features is configured to identify demarcation between plural segments predefined in the multimedia program.

32. The device of claim 28, further comprising circuitry configured to identify demarcation between plural segments in the multimedia program.

33. The device of claim 28, wherein the learning module learns the filter criteria based on training segments having content indicated by the user to be objectionable.

34. The device of claim 33, where the learning module includes circuitry configured to store predefined parameters and to permit user-modification of the predefined parameters.

35. The device of claim 34, where the device includes a cursor mechanism to permit user identification of image regions, and where the learning module is to permit user-modification of the predefined parameters based on identified image regions.

36. The device of claim 28, where the circuitry configured to extract at least one of audio, video and text features is configured to separate multiple components from the multimedia program, as is further configured to extract at least two types of features from the set of audio features, video features or text features.

37. The device of claim 36, where, for each of plural segments:

the circuitry configured to extract at least one of audio, video and text features is configured to extract at least two types of features from the set of audio features, video features or text features, for each segment of the multimedia program;

the circuitry configured to process each segment is configured to compare for each segment the extracted feature(s) to criteria; and

the circuitry configured to eliminate the objectionable material from the multimedia program, configured to eliminate the objectionable material on a segment-by-segment basis.

38. A device for filtering material from a multimedia program, comprising:

a learning module to generate filter criteria learned based on user instructions by examples of objectionable content;

at least two of a visual analysis module to extract image features;

an audio analysis module to extract second verbal audible features;

a transcript analysis module to extract text features based on a transcript;

and

a filter to process the multimedia program, according to the extracted features and the titter criteria generated by the learning module, and generate a numeric ranking corresponding to the filter criteria learned based on the user instructions by examples of objectionable content and applied to a portion of the multimedia program, and eliminate, in accordance with the numeric ranking, objectionable material from the multimedia program.

39. The device of claim 38, further comprising a splitting mechanism to split the multimedia program into a plurality of multimedia components.

40. The device of claim 38, where the device comprises each of:

a visual analysis module to extract image features represented by the multimedia components;

an audio analysis module to extract verbal audible features from an audio analysis component within the multimedia components; and

a transcript analysis module to extract text features represented by at least one of the multimedia components, based on a transcript.

41. The device of claim 38, embodied in a set top box.

42. The device of claim 38, embodied in a television set.

43. The device of claim 38, wherein the learning module learns the filter criteria based on training segments having content indicated by the user to be objectionable.

44. The device of claim 43, where the learning module includes circuitry configured to store predefined parameters and to permit user-modification of the predefined parameters.

45. The device of claim 44, where the device further includes a cursor mechanism to permit user to identify regions, and where the learning module is to permit user modification of the predefined parameters based on regions identified using the cursor mechanism.

46. The device of claim 38, further comprising circuitry configured to separate multiple components from the multimedia program, the device being configured to extract from the component respective features from the set of audio features, video features or text features.

47. The device of claim 38, where the multimedia program has plural segments, and where:

the at least two of a visual analysis module, an audio analysis module and a transcript analysis module are configured to extract respective features from the set of audio features, video features or text features, for each segment of the multimedia program;

and

the filter is to eliminate the objectionable material on a segment-by-segment basis.

48. A device for tittering material from a multimedia program, comprising:

a learning module to generate filter criteria learned based on user instructions by examples of objectionable content;

for each one of plural components of the multimedia program, each comprising plural segments, a mechanism to extract, on a segment-by-segment basis, at least one of image features, verbal audible features, and text features;

and

a filter to process each segment amongst the segments, according to the extracted features and the learned filter criteria generated by the learning module, and generate a numeric ranking corresponding to the filter criteria learned based on user instructions by examples of objectionable content and applied to the segment, and eliminate objectionable material in accordance with the numeric ranking on a segment-by-segment basis.

49. A device to eliminate objectionable material from a multimedia program having plural segments, comprising:

a learning module to generate filter criteria learned based on user instructions by examples of objectionable content;

means for extracting at least two of audio, video, or transcript features from respective multimedia components;

and

means for processing each segment amongst the segments, according to the learned filter criteria generated by the learning module and the extracted features, and generating a numeric ranking corresponding to the filter criteria learned based on the user instructions by examples of objectionable content and applied to the segment, and eliminating objectionable material, in accordance with the numeric ranking, on a segment-by-segment basis.

50. A device for eliminating objectionable material from a multimedia program, the device comprising:

a learning module to generate filter criteria learned based on user instructions by examples of objectionable content;

circuitry configured to extract at least one of audio, video and text features from the multimedia program;

and

circuitry configured to process the multimedia program, according to the extracted features and the learned filter criteria generated by the learning module, and generate a numeric ranking corresponding to the filter criteria learned based on the user instructions by examples of objectionable content and applied to a portion of the multimedia program, and eliminate, in accordance with the numeric ranking, objectionable material from the multimedia program.