Content annotation tool

• US 9,438,947 B2
• Filed: 05/01/2013
• Issued: 09/06/2016
• Est. Priority Date: 05/01/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kser, Mikkel Crone, Schneiderman, Henry Will, Sipe, Michael Andrew, Paglia, Marco
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Hicks, Charles N
• Application Number:
  US13/874,541
• Publication Number:
  US 20140331264A1
• Time in Patent Office: 1,224 Days
• Field of Search: 725/40, 725/51, 725/60, 725/92
• US Class Current: 1/1
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04N 21/278 Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content

Abstract

A content annotation tool is disclosed. In a configuration, a portion of a movie may be obtained from a database. Entities, such as an actor, background music, text, etc. may be automatically identified in the movie. A user, such as a content producer, may associate and/or provide supplemental content for an identified entity to the database. A selection of one or more automatically identified entities may be received. A database entry may be generated that links the identified entity with the supplemental content. The selected automatically identified one or more entities and/or supplemental content associated therewith may be presented to an end user.

Claims

1. A computer-implemented method comprising:

obtaining a portion of a movie;

automatically identifying an entity within the portion of the movie using a technique selected from the group consisting of;

face recognition, voice recognition, object recognition, individual characteristic recognition, scene break recognition, object tracking, speech recognition, and audio recognition;

receiving supplemental content from a first user about the identified entity within the first movie;

subsequent to receipt of the supplemental content, receiving an update to the supplemental content about the identified entity within the movie, wherein the update to the supplemental content modifies or replaces the supplemental content;

receiving a request from a second user for information related to a location of the second user; and

responsive to the request, providing, to the second user, the information related to the location and at least one item of the supplemental content, from the update to the supplemental content, related to the location.

2. The method of claim 1, further comprising storing a database entry that links the identified entity within the portion of the movie and the supplemental content about the identified entity wherein the database entry is based upon an identification of the portion of the movie, the supplemental content, and the identified entity to a database.

3. The method of claim 1, further comprising tracking the entity in the movie throughout a duration that the entity exists within the portion of the movie.

4. The method of claim 1, wherein the entity comprises an audio component of the movie.

5. The method of claim 4, wherein the audio component is selected from the group consisting of:

a song, a soundtrack, a voice, and a sound effect.

6. The method of claim 1, wherein the entity comprises a visual component.

7. The method of claim 6, wherein the visual component is selected from the group consisting of:

a scene break, a geographic location, a face, a person, an object, a physical object, and a text.

8. The method of claim 1, wherein the supplemental content is selected from the group consisting of:

a text, an audio entity, a visual entity, a URL, a picture, an advertisement, and a location.

9. The method of claim 1, further comprising associating a time reference with the supplemental content.

10. The method of claim 1, wherein the supplemental content comprises a selection of one or more of the automatically identified entities.

11. A computer-implemented method comprising:

automatically identifying an entity in a movie using a technique selected from the group consisting of;

face recognition, voice recognition, object recognition, individual characteristic recognition, scene break recognition, object tracking, speech recognition, and audio recognition;

obtaining supplemental content for the entity identified in the movie;

subsequently, receiving an update to the supplemental content for the identified entity in the movie, wherein the update to the supplemental content modifies or replaces the supplemental content;

providing the movie;

receiving, from a user, a query for information related to a location of the user, wherein the query is received by an input method during playback of the movie;

retrieving, from a database, the information related to the location and at least one item of the supplemental content, of the update to the supplemental content about the identified entity, related to the location, wherein the database comprises a plurality of automatically identified entities for a plurality of movies and corresponding supplemental content; and

providing, to the user, a response to the query wherein the response includes the update to the supplemental content.

12. The method of claim 11 further comprising tracking the entity while the entity is present in the content.

13. The method of claim 11, wherein the query is received by an input device.

14. The method of claim 11, wherein receipt of the query causes playback of the content to pause.

15. A computer-implemented method comprising:

receiving supplemental content;

storing the supplemental content;

obtaining content;

automatically identifying an entity in the content using a technique selected from the group consisting of;

face recognition, voice recognition, object recognition, individual characteristic recognition, scene break recognition, object tracking, speech recognition, and audio recognition;

subsequent to receipt of the supplemental content, receiving an update to the supplemental content about the identified entity within the content, wherein the update to the supplemental content modifies or replaces the supplemental content;

retrieving a portion of the update to the supplemental content based on the entity;

storing a database entry that links the identified entity with the portion of the update to the supplemental content while the entity is present in the content;

providing, to a user, a portion of the content, the portion of the content including the identified entity;

receiving, from the user, a request for information related to a location of the user and based upon the portion of the content and the identified entity; and

responsive to the request, providing, to the user, the information related to the location and at least one item of the supplemental content, of the update to the supplemental content, related to the location.

16. The method of claim 15, wherein the supplemental content is selected from the group consisting of:

a text, an audio entity, a visual entity, a URL, a picture, an advertisement, and a location.

17. The method of claim 15, wherein the content comprises a movie or multimedia data.

18. The method of claim 15, wherein entity comprises an audio component of the content.

19. The method of claim 17, wherein the audio component is selected from the group consisting of:

a song, a soundtrack, a voice, and a sound effect.

20. The method of claim 15, wherein the entity comprises a visual component.

21. The method of claim 19, wherein the visual component is selected from the group consisting of:

a scene break, a geographic location, a face, a person, an object, a physical object, and a text.

22. The method of claim 15, further comprising associating a time reference with the supplemental content.

23. The method of claim 15, further comprising tracking the entity in the content throughout a duration that the entity exists within the content.