Using digital fingerprints to associate data with a work

• US 10,698,952 B2
• Filed: 03/20/2017
• Issued: 06/30/2020
• Est. Priority Date: 09/25/2012
• Status: Active Grant

• Current Assignee: Audible Magic Corporation
• Original Assignee: Audible Magic Corporation
• Inventors: Garside, Vic, Friedman, Jay, Schrempp, James, Williams, Robert
• Primary Examiner(s): Al-Hashemi, Sana A
• Application Number:
  US15/464,196
• Publication Number:
  US 20170193104A1
• Time in Patent Office: 1,198 Days
• Field of Search: 707607, 707609, 707687, 707705, 707790, 707813, 707821
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: H04L 9/3247 involving digital signatures

Abstract

A computing device, during sampling or playback of a work, receives a command to associate data with the work at a particular point in the work. The computing device generates a digital fingerprint of a segment of the work, wherein the segment corresponds to the particular point in the work. The computing device then associates the data with the digital fingerprint.

Claims

1. A method comprising:

during sampling or a first playback of a work, receiving, by a user device, a command to associate data with the work at a segment of the work occurring at a particular point in time in the work;

generating, by the user device, a user digital fingerprint of the segment of the work, wherein the segment comprises at least one of a sequence of frames of the work or adjacent in time samples of the work that begin at the particular point in time in the work, and wherein the user digital fingerprint comprises a feature vector of the segment of the work generated by the user device; and

associating the data with the user digital fingerprint, wherein the user digital fingerprint associated with the data will cause at least one of the user device or an additional user device to identify the segment of the work at the particular point in time in the work or an additional point in time in the work during a subsequent playback of the work and to present the data during the subsequent playback of the segment at the particular point in time or the additional point in time in the work.

2. The method of claim 1, wherein said data comprises at least one of a text commentary, a digital image, a uniform resource locator (URL), a video, a sound effect, or a command.

3. The method of claim 1, wherein the work is one of an audio rendering, a video rendering, or a video/audio rendering.

4. The method of claim 1, further comprising:

sending a query to a server, the query comprising the user digital fingerprint;

receiving a master digital fingerprint of a segment of a copy of the work from the server, the segment of the copy corresponding to the segment of the work, wherein the master digital fingerprint is a digital fingerprint from a high quality version of the work that comprises a feature vector of the high quality version of the work; and

replacing the user digital fingerprint with the master digital fingerprint.

5. The method of claim 4, further comprising:

performing an analysis of a quality of the segment of the work based on at least one of characteristics of the segment or characteristics of the user digital fingerprint; and

querying the server in response to determining, based on the analysis, that the quality of the segment of the work fails to satisfy a quality threshold.

6. The method of claim 1, further comprising:

associating a playback time range with the user digital fingerprint, wherein the playback time range indicates a range of points in time during the subsequent playback at which to generate digital fingerprints for comparison with the user digital fingerprint.

7. The method of claim 1, wherein during the subsequent playback of the work by the user device or the additional user device, the user device or the additional user device is to:

generate a plurality of additional digital fingerprints of the work;

identify an approximate match between one of the plurality of additional digital fingerprints and the user digital fingerprint or a master digital fingerprint corresponding to the user digital fingerprint;

determine the particular point in time or the additional point in time of the work associated with the additional digital fingerprint that matches the user digital fingerprint or the master digital fingerprint; and

present the data at the particular point in time or the additional point in time in the work, wherein the master digital fingerprint is a digital fingerprint generated from a high quality version of the work.

8. The method of claim 1, wherein associating the data with the user digital fingerprint comprises generating a tag comprising the user digital fingerprint and the data.

9. A system comprising:

a first user device comprising a first memory and a first processing device operatively coupled to the first memory, wherein the first user device is to;

during sampling or a first playback of a work, receive a command to associate data with the work at a segment of the work occurring at a particular point in time in the work;

generate a user digital fingerprint of the segment of the work, wherein the segment comprises at least one of a sequence of frames of the work or adjacent in time samples of the work that begin at the particular point in time in the work, and wherein the user digital fingerprint comprises a feature vector of the segment of the work generated by the first user device; and

associate the data with the user digital fingerprint, wherein the user digital fingerprint associated with the data will cause at least one of the first user device or an additional user device to identify the segment of the work at the particular point in time in the work or an additional point in time in the work during a subsequent playback of the work and to present the data during the subsequent playback of the segment at the particular point in time or the additional point in time in the work.

10. The system of claim 9, wherein the work is one of an audio rendering, a video rendering, or a video/audio rendering.

11. The system of claim 9, further comprising:

a second user device comprising a second memory and a second processing device operatively coupled to the second memory, wherein the second user device is to;

receive the data and the user digital fingerprint or a master digital fingerprint corresponding to the user digital fingerprint;

initiate the subsequent playback of the work; and

perform the following during the subsequent playback of the work;

generate a plurality of additional digital fingerprints of the work;

identify an approximate match between one of the plurality of additional digital fingerprints and the user digital fingerprint or the master digital fingerprint corresponding to the user digital fingerprint;

determine the particular point in time or the additional point in time in the work associated with the additional digital fingerprint that matches the user digital fingerprint or the master digital fingerprint; and

present the data at the particular point in time or the additional point in time in the work, wherein the master digital fingerprint is a digital fingerprint generated from a high quality version of the work.

12. The system of claim 9, wherein the first user device is further to:

initiate the subsequent playback of the work; and

perform the following during the subsequent playback of the work;

generate a plurality of additional digital fingerprints of the work;

identify an approximate match between one of the plurality of additional digital fingerprints and the user digital fingerprint or a master digital fingerprint corresponding to the user digital fingerprint;

determine the particular point in time or the additional point in time in the work associated with the additional digital fingerprint that matches the user digital fingerprint or the master digital fingerprint; and

present the data at the particular point in time or the additional point in time in the work, wherein the master digital fingerprint is a digital fingerprint generated from a high quality version of the work.

13. The system of claim 9, wherein said data comprises at least one of a text commentary, a digital image, a uniform resource locator (URL), a video, a sound effect, or a command.

14. The system of claim 9, wherein the first user device is further to:

query a server, the query comprising the user digital fingerprint;

receive a master digital fingerprint of a segment of a copy of the work from the server, the segment of the copy corresponding to the segment of the work, wherein the master digital fingerprint is a digital fingerprint from a high quality version of the work that comprises a feature vector of the high quality version of the work; and

replace the user digital fingerprint with the master digital fingerprint.

15. The system of claim 14, wherein the first user device is further to:

perform an analysis of a quality of the segment of the work based on at least one of characteristics of the segment or characteristics of the user digital fingerprint; and

query the server in response to determining, based on the analysis, that the quality of the segment of the work fails to satisfy a quality threshold.

16. The system of claim 9, wherein the first user device is further to:

associate a playback time range with the user digital fingerprint, wherein the playback time range indicates a range of points in time during the subsequent playback at which to generate digital fingerprints for comparison with the user digital fingerprint.

17. The system of claim 9, wherein to associate the data with the user digital fingerprint the first user device generates a tag comprising the user digital fingerprint and the data.

18. A computer readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

during sampling or a first playback of a work, receiving, by the processing device, a command to associate data with the work at a segment of the work occurring at a particular point in time in the work;

generating, by the processing device, a user digital fingerprint of the segment of the work, wherein the segment comprises at least one of a sequence of frames of the work or adjacent in time samples of the work that begin at the particular point in time in the work, and wherein the user digital fingerprint comprises a feature vector of the segment of the work generated by the processing device; and

associating the data with the user digital fingerprint, wherein the user digital fingerprint associated with the data will cause at least one of the processing device or an additional processing device to identify the segment of the work at the particular point in time in the work or an additional point in time in the work during a subsequent playback of the work and to present the data during the subsequent playback of the segment at the particular point in time or the additional point in time in the work.

19. The computer readable storage medium of claim 18, the operations further comprising:

sending a query to a server, the query comprising the user digital fingerprint;

receiving a master digital fingerprint of a segment of a copy of the work from the server, the segment of the copy corresponding to the segment of the work, wherein the master digital fingerprint is a digital fingerprint from a high quality version of the work that comprises a feature vector of the high quality version of the work; and

replacing the user digital fingerprint with the master digital fingerprint.

20. The computer readable storage medium of claim 18, the operations further comprising:

performing an analysis of a quality of the segment of the work based on at least one of characteristics of the segment or characteristics of the user digital fingerprint; and

querying a server in response to determining, based on the analysis, that the quality of the segment of the work fails to satisfy a quality threshold.