CONTENT ANNOTATION TOOL

• US 20140331264A1
• Filed: 05/01/2013
• Published: 11/06/2014
• Est. Priority Date: 05/01/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Schneiderman, Henry Will, Sipe, Michael Andrew, Paglia, Marco, Kser, Mikkel Crone
• Granted Patent: US 9,438,947 B2
• Time in Patent Office: Days
• US Class Current: 725/92
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04N 21/278 Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content

Abstract

A content annotation tool is disclosed. In a configuration, a portion of a movie may be obtained from a database. Entities, such as an actor, background music, text, etc. may be automatically identified in the movie. A user, such as a content producer, may associate and/or provide supplemental content for an identified entity to the database. A selection of one or more automatically identified entities may be received. A database entry may be generated that links the identified entity with the supplemental content. The selected automatically identified one or more entities and//or supplemental content associated therewith may be presented to an end user.

Claims

1. A computer-implemented method comprising:

obtaining a portion of a first movie;

automatically identifying an entity within the portion of the movie;

receiving supplemental content from a first user about the identified entity within the first movie;

storing a database entry that links the identified entity within the portion of the first movie and the supplemental content about the identified entity;

providing a portion of a second movie to a second user, the portion of the second movie including the identified entity;

receiving a request from the second user based upon the portion of the second movie and the identified entity; and

responsive to the request, providing the supplemental content to the second user.

2. The method of claim 1, wherein the database entry is based upon an identification of the portion of the movie, the supplemental content, and the identified entity to a database.

3. The method of claim 1, further comprising tracking the entity in the movie throughout the duration that the entity exists within the portion of the movie.

4. The method of claim 1, wherein the entity comprises an audio component of the movie.

5. The method of claim 4, wherein the audio component is selected from the group consisting of:

a song, a soundtrack, a voice, and a sound effect.

6. The method of claim 1, wherein the entity comprises a visual component.

7. The method of claim 6, wherein the visual component is selected from the group consisting of:

a scene break, a geographic location, a face, a person, an object, a physical object, and a text.

8. The method of claim 1, wherein the second movie comprises the first movie.

9. The method of claim 1, wherein supplemental content is selected from the group consisting of:

a text, an audio entity, a visual entity, a URL, a picture, an advertisement, and a location.

10. The method of claim 1, further comprising associating a time reference with the supplemental content.

11. The method of claim 1, wherein the supplemental content comprises a selection of one or more of the automatically identified entities.

12. The method of claim 1, wherein the step of automatically identifying the entity is performed using a technique selected from the group consisting of:

face recognition, voice recognition, object recognition, individual characteristic recognition, scene break recognition, object tracking, speech recognition, and audio recognition.

13. A computer-implemented method comprising:

providing a movie;

receiving a query from a user, where the query is received by an input method during playback of the movie;

determining a location reference for the query, where the location reference identifies an entity in the movie;

retrieving, from a database, supplemental content about the identified entity, where the database comprising a plurality of automatically identified entities for a plurality of movies and supplemental content; and

providing a response to the query to the user, where the response includes supplemental content.

14. The method of claim 13 further comprising tracking the entity while the entity is present in the content.

15. The method of claim 13, wherein the query is received by an input device.

16. The method of claim 13, wherein receipt of the query causes playback of the content to pause.

17. The method of claim 13, further comprising providing the supplemental content substantially near the location reference.

18. A computer-implemented method comprising:

receiving a query by a database, where the database comprising a plurality of automatically identified entities for a plurality of movies and supplemental content;

generating a response, where the response comprises supplemental content and is based on at least one of;

a user history and a user location; and

sending the response to a device.

19. The method of claim 18, further comprising providing an augmented reality to the device.

20. The method of claim 18, wherein the user history comprises content consumed by a user.

21. The method of claim 18, further comprising receiving an update to the supplemental content from the device.

22. The method of claim 18, wherein the query is automatically sent by the device.

23. The method of claim 18, wherein an entity comprises an audio component or visual component of a movie.

24. A computer-implemented method comprising:

receiving supplemental content;

storing the supplemental content;

obtaining content;

automatically identifying an entity in the content;

retrieving a portion of the supplemental content based on the entity;

storing a database entry that links the identified entity with a portion of the supplemental content while the entity is present in the content;

providing a portion of the content to a user, the portion of the content movie including the identified entity;

receiving a request from the user based upon the portion of the movie and the identified entity; and

responsive to the request, providing the supplemental content to the user.

25. The method of claim 24, wherein supplemental content is selected from the group consisting of:

a text, an audio entity, a visual entity, a URL, a picture, an advertisement, and a location.

26. The method of claim 24, wherein the content comprises a movie or multimedia data.

27. The method of claim 24, wherein entity comprises an audio component of the movie.

28. The method of claim 26, wherein the audio component is selected from the group consisting of:

a song, a soundtrack, a voice, and a sound effect.

29. The method of claim 24, wherein the entity comprises a visual component.

30. The method of claim 28, wherein the visual component is selected from the group consisting of:

a scene break, a geographic location, a face, a person, an object, a physical object, and a text.

31. The method of claim 24, further comprising associating a time reference with the supplemental content.

32. The method of claim 24, further comprising tracking the entity in the movie throughout the duration that the entity exists within the content.

33. A computer-implemented method comprising:

obtaining a portion of a first movie;

automatically identifying an entity within the portion of the movie;

receiving supplemental content from a first user about the identified entity within the first movie; and

storing a database entry that links the identified entity within the portion of the first movie and the supplemental content about the identified entity.

34. The method of claim 33, wherein the supplemental content comprises a selection of one or more of the automatically identified entities.

35. The method of claim 33, wherein the step of automatically identifying the entity is performed using a technique selected from the group consisting of:

face recognition, voice recognition, object recognition, individual characteristic recognition, scene break recognition, object tracking, speech recognition, and audio recognition.