Augmenting metadata of digital objects

• US 8,856,051 B1
• Filed: 04/08/2011
• Issued: 10/07/2014
• Est. Priority Date: 04/08/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Song, Yang, Zhao, Ming
• Primary Examiner(s): Wong, Lut
• Application Number:
  US13/083,452
• Time in Patent Office: 1,278 Days
• Field of Search: 706/12
• US Class Current: 706/12
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 20/48 Matching video sequences

Abstract

A metadata augmentation system determines similarities between digital objects, such as digital videos, that may or may not have metadata associated with them. Based on the determined similarities, the metadata augmentation system augments metadata of objects, such as augmenting metadata of objects lacking a sufficient amount of metadata with metadata from other objects having a sufficient amount of metadata.

In one embodiment, the similarities are used to determine training sets for training of classifiers that output degrees of more specific similarities between the corresponding video and an arbitrary second video. These classifiers are then applied to add metadata from one video to another based on a degree of similarity between the videos, regardless of their respective locations within the object similarity graph.

Claims

1. A computer-implemented method comprising:

accessing a set of videos;

determining degrees of similarity between pairs of the videos;

for each video of a plurality of the videos, training a classifier for the video, the training comprising;

forming, for the video, a training set comprising other ones of the videos based at least in part on the degrees of similarity; and

training the classifier for the video based at least in part on audiovisual features extracted from the videos in the training set;

applying the trained classifier for a first one of the videos to a second one of the videos to determine a degree of similarity between the second one of the videos and the first one of the videos; and

responsive to the degree of similarity determined by applying the trained classifier being above a threshold value;

based on the degree of similarity determined by applying the trained classifier, reducing cluster weights derived from user-supplied textual metadata of the first one of the videos, thereby obtaining first reduced cluster weight metadata; and

associating the first reduced cluster weight metadata as metadata of the second one of the videos.

2. The computer-implemented method of claim 1, further comprising:

applying the trained classifier for a third one of the videos to the second one of the videos to determine whether the third one of the videos is similar to the second one of the videos; and

responsive to the third one of the videos being similar to the second one of the videos;

based on a degree of similarity between the second and third videos, reducing cluster weights derived from user-supplied textual metadata of the third one of the videos, thereby obtaining second reduced cluster weight metadata;

combining the second reduced cluster weight metadata with the first reduced cluster weight metadata; and

associating the combined reduced cluster weight metadata as metadata of the second one of the videos.

3. The computer-implemented method of claim 1, wherein forming the training set for the video comprises:

creating a similarity graph comprising;

ones of the videos, and

edges between pairs of the videos, the edges associated with the determined degrees of similarity of the pairs;

computing path degrees of similarity between the video and other ones of the videos based on a predetermined decay factor and on shortest path lengths between the video and the other ones of the videos; and

adding videos to the training set based on their computed path degrees of similarity with the video.

4. The computer-implemented method of claim 1, wherein forming the training set for the video comprises:

creating a similarity graph comprising;

ones of the videos, and

edges between pairs of the videos, the edges associated with the determined degrees of similarity of the pairs;

computing path degrees of similarity between the video and other ones of the videos based on products of the degrees of similarity associated with the edges on a path between the video and the others one of the videos; and

adding videos to the training set based on their computed path degrees of similarity with the video.

5. The computer-implemented method of claim 1, wherein the degrees of similarity between pairs of the videos are determined based on co-play counts of the videos in the pair.

6. The computer-implemented method of claim 1, wherein the degrees of similarity between pairs of the videos are determined based on comparisons between content of the videos in the pair.

7. The computer-implemented method of claim 1, wherein associating the first reduced cluster weight metadata as metadata of the second one of the videos comprises:

obtaining the first reduced cluster weight metadata by scaling the cluster weights derived from user-supplied textual metadata of the first one of the videos by the degree of similarity; and

obtaining combined cluster weight metadata by adding the scaled cluster weights to cluster weights derived from user-supplied textual metadata of the second one of the videos;

storing the combined cluster weight metadata in association with the second one of the videos.

8. The computer-implemented method of claim 1, wherein associating the first reduced cluster weight metadata as metadata of the second one of the videos comprises:

obtaining the first reduced cluster weight metadata by scaling the cluster weights derived from user-supplied textual metadata of the first one of the videos by the degree of similarity; and

responsive to the second one of the videos lacking user-supplied textual metadata, storing the first reduced cluster weight metadata in association with the second one of the videos.

9. The computer-implemented method of claim 1, further comprising prompting a user that uploaded the second one of the videos to approve association of the first reduced cluster weight metadata as metadata of the second one of the videos.

13. The non-transitory computer-readable storage medium of claim 8, wherein forming the training set for the object comprises:

creating a similarity graph comprising;

ones of the objects, and

edges between pairs of the objects, the edges associated with the determined degrees of similarity of the pairs;

computing path degrees of similarity between the object and other ones of the objects based on products of the degrees of similarity associated with the edges on a path between the object and the others one of the objects; and

adding objects to the training set based on their computed path degrees of similarity with the object.

10. A non-transitory computer-readable storage medium having executable computer program instructions embodied therein, actions of the computer program instructions comprising:

accessing a set of digital objects;

determining degrees of similarity between pairs of the objects; and

for each object of a plurality of the objects, training a classifier for the object, the training comprising;

forming, for the object, a training set comprising other ones of the objects based at least in part on the degrees of similarity; and

training the classifier for the object based at least in part on features extracted from the objects in the training set;

applying the trained classifier for a first one of the objects to a second one of the objects to determine a degree of similarity between the second one of the objects and the first one of the objects; and

responsive to the degree of similarity being above a threshold value;

based on the degree of similarity, reducing cluster weights derived from user-supplied textual metadata of the first one of the objects, thereby obtaining first reduced cluster weight metadata; and

associating the first reduced cluster weight metadata as metadata of the second one of the objects.

11. The non-transitory computer-readable storage medium of claim 10, further comprising:

applying the trained classifier for a third one of the objects to the second one of the objects to determine whether the third one of the objects is similar to the second one of the objects; and

responsive to the third one of the objects being similar to the second one of the objects;

based on a degree of similarity between the second and third objects, reducing cluster weights derived from user-supplied textual metadata of the third one of the objects, thereby obtaining second reduced cluster weight metadata;

combining the second reduced cluster weight metadata with the first reduced cluster weight metadata values; and

associating the combined reduced cluster weight metadata as metadata of the second one of the objects.

12. The non-transitory computer-readable storage medium of claim 10, wherein forming the training set for the object comprises:

creating a similarity graph comprising;

ones of the objects, and

edges between pairs of the objects, the edges associated with the determined degrees of similarity of the pairs;

computing path degrees of similarity between the object and other ones of the objects based on a predetermined decay factor and on shortest path lengths between the object and the other ones of the objects; and

adding objects to the training set based on their computed path degrees of similarity with the object.

14. The non-transitory computer-readable storage medium of claim 10, wherein the degrees of similarity between pairs of the objects are determined based on co-play counts of the videos in the pair.

15. A computer system comprising:

a computer processor; and

a computer-readable storage medium storing a computer program executable by the computer processor and performing actions comprising;

accessing a set of videos;

determining degrees of similarity between pairs of the videos; and

for each video of a plurality of the videos, training a classifier for the video, the training comprising;

forming, for the video, a training set comprising other ones of the videos based at least in part on the degrees of similarity; and

training the classifier for the video based at least in part on audiovisual features extracted from the videos in the training set;

applying the trained classifier for a first one of the videos to a second one of the videos to determine a degree of similarity between the second one of the videos and the first one of the videos; and

responsive to the degree of similarity being above a threshold value;

based on the degree of similarity, reducing cluster weights derived from user-supplied textual metadata of the first one of the videos, thereby obtaining first reduced cluster weight metadata; and

associating the first reduced cluster weight metadata as metadata of the second one of the videos.

16. The computer system of claim 15, the actions further comprising:

applying the trained classifier for a third one of the videos to the second one of the videos to determine whether the third one of the videos is similar to the second one of the videos; and

responsive to the third one of the videos being similar to the second one of the videos;

based on a degree of similarity between the second and third videos, reducing cluster weights derived from user-supplied textual metadata of the third one of the videos, thereby obtaining second reduced cluster weight metadata;

combining the second reduced cluster weight metadata values with the first reduced cluster weight metadata; and

associating the combined reduced cluster weight metadata as metadata of the second one of the videos.

17. The computer system of claim 15, wherein forming the training set for the video comprises:

creating a similarity graph comprising;

ones of the videos, and

edges between pairs of the videos, the edges associated with the determined degrees of similarity of the pairs;

computing path degrees of similarity between the video and other ones of the videos based on a predetermined decay factor and on shortest path lengths between the video and the other ones of the videos; and

adding videos to the training set based on their computed path degrees of similarity with the video.