Method and apparatus for automatically recognizing input audio and/or video streams
First Claim
1. Apparatus for recognizing an input data stream, comprising:
- a receiver for receiving the input data stream;
an interface for randomly selecting any one portion of the received data stream, and forming a plurality of feature time series waveforms respectively corresponding to distinct portions of the received data stream;
a memory for storing a plurality of feature distinctiveness maps of stored candidate patterns; and
processor structure for;
(i) computing distinctiveness of each feature of the input stream and constructing a map of the rank ordering of said distinctiveness, (ii) computing a distance metric between the input feature distinctiveness map and the plurality of feature distinctiveness maps of stored candidate patterns, (iii) correlating the plurality of feature time series waveforms with the plurality of feature distinctiveness maps of stored candidate patterns in an order determined by the computed distance metrics, (iv) designating a recognition when a feature correlation joint probability between the plurality of feature time series waveforms and at least one of the plurality of feature distinctiveness maps of stored candidate patterns reaches a predetermined value, and outputting a recognition signal after the feature correlation joint probability reaches the predetermined value.
0 Assignments
0 Petitions
Accused Products
Abstract
A method and system for the automatic identification of audio, video, multimedia, and/or data recordings based on immutable characteristics of these works. The invention does not require the insertion of identifying codes or signals into the recording. This allows the system to be used to identify existing recordings that have not been through a coding process at the time that they were generated. Instead, each work to be recognized is “played” into the system where it is subjected to an automatic signal analysis process that locates salient features and computes a statistical representation of these properties. These features are then stored as patterns for later recognition of live input signal streams. A different set of features is derived for each audio or video work to be identified and stored. During real-time monitoring of a signal stream, a similar automatic signal analysis process is carried out, and many features are computed for comparison with the patterns stored in a large feature database. For each particular pattern stored in the database, only the relevant characteristics are compared with the real-time feature set. Preferably, during analysis and generation of reference patterns, data are extracted from all time intervals of a recording. This allows a work to be recognized from a single sample taken from any part of the recording.
87 Citations
17 Claims
-
1. Apparatus for recognizing an input data stream, comprising:
-
a receiver for receiving the input data stream; an interface for randomly selecting any one portion of the received data stream, and forming a plurality of feature time series waveforms respectively corresponding to distinct portions of the received data stream; a memory for storing a plurality of feature distinctiveness maps of stored candidate patterns; and processor structure for;
(i) computing distinctiveness of each feature of the input stream and constructing a map of the rank ordering of said distinctiveness, (ii) computing a distance metric between the input feature distinctiveness map and the plurality of feature distinctiveness maps of stored candidate patterns, (iii) correlating the plurality of feature time series waveforms with the plurality of feature distinctiveness maps of stored candidate patterns in an order determined by the computed distance metrics, (iv) designating a recognition when a feature correlation joint probability between the plurality of feature time series waveforms and at least one of the plurality of feature distinctiveness maps of stored candidate patterns reaches a predetermined value, and outputting a recognition signal after the feature correlation joint probability reaches the predetermined value. - View Dependent Claims (2, 3)
-
-
4. Apparatus for forming features from an input data stream, comprising:
-
a receiver for receiving the input data stream and extracting a plurality of feature time series waveforms; and processor structure for (i) determining the information content of each feature from each of a plurality of time interval segments of the feature time series waveforms, (ii) rank-ordering each of the features of each of the time interval segments according to their information content, and (iii) constructing a map of feature distinctiveness rankings; and a memory for storing the constructed feature distinctiveness maps for each of the time interval segments.
-
-
5. A processor-implemented method for recognizing an input data stream, comprising the steps of:
-
receiving the input data stream; randomly selecting any one time interval from the received data stream; forming a first plurality of feature time series waveforms respectively corresponding to distinct portions of the received data stream; rank ordering features of the first plurality of waveforms according to distinctiveness of their information content and constructing corresponding feature distinctiveness maps; retrieving from a storage device a second plurality of feature time series waveforms and their predetermined feature distinctiveness maps; computing a distance measure between the first feature distinctiveness map and each of the second plurality of feature distinctiveness maps, using a processor; constructing a list of second plurality of feature time series waveforms corresponding to the smallest distances between first feature distinctiveness map and each of the second feature distinctiveness maps; correlating the first plurality of feature time series waveforms with the second plurality of feature time series waveforms in an order corresponding to (i) the list of candidate patterns from the second plurality of feature time series waveforms having the smallest distances between the distinctiveness map of the first plurality of feature time series waveforms, and (ii) the distinctiveness maps of the second plurality of feature time series waveforms; and designating a recognition when a joint correlation probability value between the first plurality of feature time series waveforms and at least one of the second plurality of feature time series waveforms reaches a predetermined value. - View Dependent Claims (6, 7)
-
-
8. A processor-implemented method for forming feature waveforms from an input data stream comprising the steps of:
-
receiving the input data stream and separating the input data stream into a plurality of different feature waveforms; determining the most distinctive feature information from each of a plurality of time interval segments, utilizing a processor; rank-ordering features of the time interval segments according to their distinctiveness; and storing in a storage device a map corresponding to the rank-ordered features.
-
-
9. A computer readable storage device for storing a program which causes one or more computers to recognize an input data stream, the stored program causing the one or more computers to:
-
receive the input data stream; randomly select any time interval of the received data stream; form a first plurality of feature time series waveforms from the received data stream which respectively correspond to distinct portions of the received data stream; store a second plurality of feature time series waveforms; compute the distinctiveness of each feature of the input stream and constructing a map of the rank ordering; compute a distance metric between the input feature distinctiveness map and the second plurality of feature distinctiveness maps of stored candidate patterns; correlate the first plurality of feature time series waveforms with the second plurality of feature time series waveforms in an order determined by the distance metric between (i) a the feature distinctiveness map of the input data stream and (ii) the feature distinctiveness maps of the second plurality of feature time series waveforms; and designate a recognition when a joint correlation probability value between the first plurality of the feature time series waveforms and at least one of the second plurality of feature time series waveforms reaches a predetermined value. - View Dependent Claims (10, 11)
-
-
12. A processor-implemented method of using recognition features from an input data stream to achieve automatic signal identification, comprising the steps of:
-
receiving the input data stream; forming a plurality of time series waveforms which correspond to all features of the received input data stream; forming multiple feature streams from the plurality of feature time series waveforms; correlating, using a processor, the most distinctive features of plural stored candidate patterns with the multiple feature streams formed from the input data stream in an order corresponding to a map of candidate patterns that best match the rank ordering of the plurality of feature time series waveforms, said stored candidate patterns being stored in a memory device; and designating recognition of the input data stream when a joint probability of correlations between the input data stream and at least one of the stored candidate patterns indicates that random detection is not probable. - View Dependent Claims (13, 14)
-
-
15. A processor-implemented method of using recognition features from an input data stream to achieve automatic signal identification comprising the steps of:
-
receiving the input data stream; forming a plurality of feature time series waveforms which correspond to features of the received input data stream; forming multiple feature streams from the plurality of feature time series waveforms; rank-ordering the features streams according to their distinctiveness; constructing a map of the feature distinctiveness rankings, and storing said map in a storage medium; computing a distance metric between the input feature distinctiveness map and each of the maps corresponding to plural stored candidate patterns, utilizing a processor; correlating the most distinctive features of the plural stored candidate patterns with the multiple feature streams formed from the input data stream in an order corresponding to the distance metric between the input feature distinctiveness map and the maps of candidate patterns of the plurality of feature time series waveforms; and designating recognition of the input data stream when a joint probability of correlations between the input data stream and at least one of the stored candidate patterns indicates that random detection is not probable. - View Dependent Claims (16, 17)
-
Specification