Business method and apparatus for employing induced multimedia classifiers based on unified representation of features reflecting disparate modalities
First Claim
1. A business method comprising the steps of:
- monitoring one or more multimedia items accessed by a user, each multimedia item having two or more disparate modalities, the disparate modalities being at least one or more visual modalities and one or more textual modalities;
for each of the one or more multimedia items;
(a) creating a visual feature vector for each of the visual modalities and a textual feature vector for each of the textual modalities; and
(b) concatenating the visual feature vectors and the textual feature vectors into one or more unified feature vectors;
categorizing at least a portion of each of the multimedia items by categorizing respective ones of the unified feature vectors; and
assembling a user profile based on the categorization.
1 Assignment
0 Petitions
Accused Products
Abstract
This invention is a business system and method to perform categorization (classification) of multimedia items and to make business decisions based on the categorization of the item. The multimedia items are comprised of a multitude of disparate information sources, in particular, visual information and textual information. Classifiers are induced based on combining textual and visual feature vectors. Textual features are the traditional ones. Visual features include, but are not limited to, color properties of key intervals and motion properties of key intervals. The visual feature vectors are determined in such a fashion that the vectors are sparse. The text and the visual representation vectors are combined in a systematic and coherent fashion. This vector representation of a media item lends itself to well-established learning techniques and can be used for multimedia item categorization. The resulting business system, subject of this invention, can be used for many purposes. An example here are enforcement of copyright, trademark, intellectual property, parental guidance and common decency restrictions. Other uses are multimedia item classifier to determine routing of incoming items or building user profiles based on user multimedia preferences.
-
Citations
19 Claims
-
1. A business method comprising the steps of:
-
monitoring one or more multimedia items accessed by a user, each multimedia item having two or more disparate modalities, the disparate modalities being at least one or more visual modalities and one or more textual modalities; for each of the one or more multimedia items; (a) creating a visual feature vector for each of the visual modalities and a textual feature vector for each of the textual modalities; and (b) concatenating the visual feature vectors and the textual feature vectors into one or more unified feature vectors; categorizing at least a portion of each of the multimedia items by categorizing respective ones of the unified feature vectors; and assembling a user profile based on the categorization. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A business method comprising the steps of:
-
scanning one or more multimedia items in a database, each multimedia item having two or more disparate modalities, the disparate modalities being at least one or more visual modalities and one or more textual modalities; for each of the one or more multimedia items; (a) creating a visual feature vector for each of the visual modalities and a textual feature vector for each of the textual modalities; and (b) concatenating the visual feature vectors and the textual feature vectors into one or more unified feature vectors; categorizing at least a portion of each of the multimedia items by categorizing respective ones of the unified feature vectors; and creating one or more indices of the database based on the categorization. - View Dependent Claims (8, 9, 10, 11, 12, 13, 14)
-
-
15. A business method comprising the steps of:
-
scanning one or more multimedia items in a database, each multimedia item having two or more disparate modalities, the disparate modalities being at least one or more visual modalities and one or more textual modalities; for each of the one or more multimedia items; (a) creating a visual feature vector for each of the visual modalities and a textual feature vector for each of the textual modalities; and (b) concatenating the visual feature vectors and the textual feature vectors into one or more unified feature vectors; categorizing at least a portion of each of the multimedia items by categorizing respective ones of the unified feature vectors; comparing one or more of the unified feature vectors to one or more other feature vectors; and making a decision based on the comparison. - View Dependent Claims (16, 17, 18, 19)
-
Specification