Method and apparatus for video digest generation
First Claim
1. A digital video recorder system for storage and/or playback of video media content, said video media content comprising audio portions and video image portions, said recorder comprising:
- at least one video processor capable of analyzing at least the video image portions of said input media content and automatically generating descriptors that are associated with the video image and/or audio contents of various time portions of the input media content;
at least one sensor to detect and record user viewing preferences while the input media content is being played;
persistent memory capable of storing a persistent database that correlates the descriptors with the recorded user viewing preferences;
at least one control processor, and associated control software;
in which the control processor, under the control of the control software, controls the storage and/or playback of unknown media content comprising various time portions of a previously unanalyzed input media content by;
a) sending the unknown media content to the video processor, and directing the video processor to generate unknown video descriptors comprising descriptors of the video and or audio portion of various time portions of the unknown media content;
b) retrieving the user viewing preferences associated with the unknown video descriptors by using the unknown video descriptors as an index to the persistent database;
c) constructing an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function;
d) generating overall video identifiers comprising video signatures of the various time portions of the unknown media content;
e) retrieving the overall user viewing preferences associated with the overall video identifiers by using the overall video identifiers as an index to the persistent database, and;
f) inputting the overall user viewing preferences and the estimated user preference into a master user preference function; and
g) controlling the storage and playback of various time portions of the unknown media content with this master user preference function.
4 Assignments
0 Petitions
Accused Products
Abstract
Automated video digest system and method. The system decomposes video and audio portions of video media into low-level descriptors, producing a large number of such descriptors for each video image and scene. These descriptors are optimized to be both compatible with rapid automated descriptor creation, and also to have some positive or negative statistical correlation with the viewing preferences of typical human viewers. By comparing the number of descriptors correlated with positive human viewing preferences, versus the number of descriptors correlated with negative human viewing preferences, the automated system may analyze an unknown video program and make educated guesses as to which portions of the program are likely to be most interesting to the viewer on playback. Less interesting portions may be skipped or fast-forwarded though. The method may also be optimized to individual viewers. Such methods are useful for controlling the playback of digital video recorders and other systems.
-
Citations
49 Claims
-
1. A digital video recorder system for storage and/or playback of video media content, said video media content comprising audio portions and video image portions, said recorder comprising:
-
at least one video processor capable of analyzing at least the video image portions of said input media content and automatically generating descriptors that are associated with the video image and/or audio contents of various time portions of the input media content; at least one sensor to detect and record user viewing preferences while the input media content is being played; persistent memory capable of storing a persistent database that correlates the descriptors with the recorded user viewing preferences; at least one control processor, and associated control software;
in which the control processor, under the control of the control software, controls the storage and/or playback of unknown media content comprising various time portions of a previously unanalyzed input media content by;a) sending the unknown media content to the video processor, and directing the video processor to generate unknown video descriptors comprising descriptors of the video and or audio portion of various time portions of the unknown media content; b) retrieving the user viewing preferences associated with the unknown video descriptors by using the unknown video descriptors as an index to the persistent database; c) constructing an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function; d) generating overall video identifiers comprising video signatures of the various time portions of the unknown media content; e) retrieving the overall user viewing preferences associated with the overall video identifiers by using the overall video identifiers as an index to the persistent database, and; f) inputting the overall user viewing preferences and the estimated user preference into a master user preference function; and g) controlling the storage and playback of various time portions of the unknown media content with this master user preference function. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A digital video recorder device for storage and playback of video media content, said video media content comprising audio portions and video image portions, said recorder comprising:
-
at least one video processor capable of analyzing at least the video image portions of said input media content and automatically generating descriptors that are associated with the video image and audio contents of various time portions of the input media content; at least one sensor to detect and record user viewing preferences while the input media content is being played;
persistent memory capable of storing a persistent database that correlates the descriptors with the recorded user viewing preferences;
at least one control processor and associated operational software;in which the control processor, under the control of the control software, is configured to controls the storage and playback of unknown media content comprising various time portions of a previously unanalyzed input media content, the processor including a descriptor generator configured to receive the unknown media content and to generate unknown video descriptors comprising descriptors of the video and or audio portion of various time portions of the unknown media content; an index unit configured to utilize the unknown video descriptors as an index to the persistent database to retrieve user viewing preferences associated with the unknown video descriptors; a preference constructor configured to construct an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function; wherein the processor is further configured to utilize this estimated user preference to control the storage and/or playback of various time portions of the unknown media content; in which said at least one video processor or control processor is further configured to compute one or more video signatures that identify overall video identifiers comprising the overall appearance of the various time portions of the unknown media content; said at least one video processor or control processor is further configured to correlate these overall video identifiers with overall user viewing preferences, and also to store this data in the persistent database; in which said at least one video processor or control processor, under control of the control software, are further configured to control the storage and/or playback of unknown media content comprising various time portions of a previously unanalyzed input media content by a signature generator configured to generate overall video identifiers comprising video signatures of the various time portions of the unknown media content; a signature indexer configured to use the overall video identifiers as an index to the persistent database, and retrieve the overall user viewing preferences associated with the overall video identifiers; a signature preference constructor configured to use the overall user viewing preferences and the estimated user preference as input into a master user preference control unit; wherein said master user preference control unit is further configured to control the storage and/or playback of various time portions of the unknown media content. - View Dependent Claims (18, 19, 20, 21, 22)
-
-
23. A computerized method of controlling the storage and/or playback of video media content said video media content comprising audio portions and video image portions, comprising:
-
inputting a plurality of different video media contents, and using at least one processor to perform the steps of; automatically generating a plurality of descriptors that are associated with the video image portions and/or audio portions contents of various time portions of the media content; and
detecting and recording the user viewing preferences associated with these various time portions of the different media contents;correlating the descriptors with the user video viewing preferences in a persistent database so that the descriptors may be used as an index to the persistent database to retrieve user video viewing preferences; and
using this database to control the storage and/or playback of unknown media content comprising a previously unrated media content by;automatically generating unknown content descriptors comprising a plurality of descriptors that are associated with the unknown media content; using the unknown content descriptors as an index into the database;
retrieving the user viewing preferences;constructing an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function; and
controlling the storage and/or playback of the unknown media content based on said estimated user preference;wherein the persistent database that correlates the descriptors with the recorded user viewing preferences is supplemented with additional descriptors from a user viewing preference neutral database of descriptors. - View Dependent Claims (24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37)
-
-
38. A computerized method of controlling the storage and/or playback of video media content said video media content comprising audio portions and video image portions, said method comprising:
-
inputting a plurality of different media contents, and using at least one processor to perform the steps of; automatically generating a plurality of descriptors that are associated with the video image portions and/or audio portion contents of various time portions of the media content; said descriptors being selected from population of descriptors known to have a positive or negative statistical correlation with user preferences; said descriptors being the outputs of mathematical functions selected from the group consisting of three dimensional blocks of pixels, three dimensional edges, spatio-temporal regions with consistent motion, a priori objects, and collections of simple features; or said descriptors being the outputs of mathematical functions selected from the group consisting of functions that analyze sound frequency range, sound energy versus frequency range, voice frequency range, music frequency range, word recognition, and language recognition; and detecting and recording the user viewing preferences associated with these various time portions of the different media contents; correlating the descriptors with the user video viewing preferences in a persistent database so that the descriptors may be used as an index to the persistent database to retrieve user video viewing preferences; and controlling the storage and/or playback of an unknown media content comprising previously unrated media content with said persistent database by; automatically generating a plurality of unknown content descriptors comprising descriptors that are associated with the unknown media content; database indexing the database with the unknown content descriptors; retrieving the user viewing preferences; constructing an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function; and
controlling the storage and/or playback of the unknown media content by further;computing one or more overall video identifiers comprising video signatures that identify the overall appearance of the various time portions of the unknown media; correlating these overall video identifiers with overall user viewing preferences, and also storing this data in the persistent database; controlling the storage and/or playback of unknown media content comprising various time portions of a previously unanalyzed input media content by; a) generating overall video identifiers comprising video signatures of the various time portions of the unknown media content; b) retrieving the overall user viewing preferences associated with the overall video identifiers by using the overall video identifiers as an index to the persistent database; c) inputting the overall user viewing preferences and the estimated user preference into a master user preference function; d) and controlling the storage and playback of various time portions of the unknown media content with this master user preference function. - View Dependent Claims (39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49)
-
Specification