Model-based media classification service using sensed media noise characteristics

• US 10,678,828 B2
• Filed: 06/17/2016
• Issued: 06/09/2020
• Est. Priority Date: 01/03/2016
• Status: Active Grant

• Current Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Original Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Inventors: Cramer, Jason, Cremer, Markus K., Popp, Phillip, Summers, Cameron Aubrey
• Primary Examiner(s): Shmatov, Alexey
• Assistant Examiner(s): Garner, Casey R.
• Application Number:
  US15/185,654
• Publication Number:
  US 20170193097A1
• Time in Patent Office: 1,453 Days
• Field of Search: None
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

A neural network-based classifier system can receive a query including a media signal and, in response, provide an indication that the query corresponds to a specified media type or media class. The neural network-based classifier system can select and apply various models to facilitate media classification. In an example embodiment, a query can be analyzed for various characteristics, such as a noise profile, before it is input to the network-based classifier. If the query has greater than a specified threshold noise characteristic, then a successful classification can be unlikely and a classification process based on the query can be terminated before computational resources are expended. Query signals that meet or exceed a threshold condition can be provided to the network-based classifier for media classification. In an example embodiment, a remote device or a central media classifier circuit can determine a noise profile for a query.

Claims

1. A method for classifying media, the method comprising:

accessing, using one or more processor circuits associated with a first device, first digital media data that represents a first media query to be identified;

determining, using the one or more processor circuits associated with the first device, a first noise characteristic corresponding to the first digital media data;

determining whether a prior classification attempt for a second media query was successful, the second media query corresponding to second digital media data having a similar noise characteristic to the first noise characteristic;

in response to determining the prior classification attempt was successful, transmitting the first digital media data to a remote media classification circuit that is configured to identify a source characteristic of the first digital media data; and

in response to determining the prior classification attempt unsuccessful, inhibiting the transmission of the first digital media data to the remote media classification circuit.

2. The method of claim 1, wherein the accessing the first digital media data includes receiving an audio signal captured via a microphone of a mobile device.

3. The method of claim 1, further including:

accessing, using the one or more processor circuits associated with the first device, a first context parameter that corresponds to the first media query to be identified;

determining, using the one or more processor circuits associated with the first device, a signal quality characteristic corresponding to the first context parameter; and

in response to determining the signal quality characteristic corresponding to the first context parameter is less than a specified threshold signal quality, inhibiting the transmission of the first digital media data to the remote media classification circuit.

4. The method of claim 1, further including:

retrieving a specified threshold amount of noise from a database of threshold noise characteristics, the database established by prior training of similar media queries with a neural network-based classifier system; and

determining if the first noise characteristic is less than the specified threshold amount of noise.

5. The method of claim 1, wherein the transmission of the first digital media data to the remote media classification circuit includes providing the first digital media data to an input of a convolutional neural network classifier system.

6. The method of claim 1, wherein the inhibiting the transmission of the first digital media data to the remote media classification circuit includes accessing subsequent digital media data that represents a different media query to be identified, determining an updated noise characteristic corresponding to the subsequent digital media data, and determining whether the updated noise characteristic indicates less than a specified threshold amount of noise.

7. The method of claim 1, wherein the accessing the first digital media data includes accessing an audio sample, using the first device, corresponding to the first media query;

wherein the determining the first noise characteristic includes determining the first noise characteristic for the audio sample; and

wherein the transmission of the first digital media data to the remote media classification circuit includes transmission of all or a portion of the audio sample to the remote media classification circuit.

8. The method of claim 1, wherein the accessing the first digital media data includes accessing a video signal sample corresponding to the first media query;

wherein the determining the first noise characteristic includes determining a visual characteristic of the video signal sample; and

wherein the transmission of the first digital media data to the remote media classification circuit includes transmission of all or a portion of the video signal sample to the remote media classification circuit.

9. The method of claim 1, further including receiving, at the first device and from the remote media classification circuit, a media characteristic of the first digital media data, and displaying the media characteristic of the first digital media data to a user of the first device.

10. A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor of a machine, cause the machine to perform operations comprising:

accessing, using one or more processor circuits, first digital media data that represents a first media query to be identified;

determining, using the one or more processor circuits, a first noise characteristic corresponding to the first digital media data;

determining whether a prior classification attempt for a second media query was successful, the second media query corresponding to second digital media data having a similar noise characteristic to the first noise characteristic;

in response to determining the prior classification attempt was successful;

providing the first digital media data to a first input of a neural network classifier;

receiving from the neural network classifier, in response to the first digital media data, a media type probability index for the first media query; and

providing information about the media type probability index to a remote device to provide an indication of a media type to a user of the remote device.

11. The non-transitory computer-readable storage medium of claim 10, wherein the operations further include:

accessing a context parameter associated with the first media query, wherein the context parameter and the first digital media data are accessed from the remote device; and

selecting a weighting model for the neural network classifier based on the context parameter.

12. The non-transitory computer-readable storage medium of claim 10, wherein the operations further include:

retrieving a specified threshold amount of noise from a database of threshold noise characteristics, the database established by prior training of similar media queries with a neural network-based classifier system; and

determining if the first noise characteristic is less than the specified threshold amount of noise.

13. The non-transitory computer-readable storage medium of claim 10, wherein accessing the first digital media data includes receiving an audio signal captured via a microphone.

14. The non-transitory computer-readable storage medium of claim 10, wherein the neural network classifier includes a convolutional neural network.

15. A media classification apparatus comprising:

an audio playback circuit to access first digital media data that represents a first media query to be identified;

a first processor circuit to;

determine a first noise characteristic corresponding to the first digital media data; and

determine whether a prior classification attempt for a second media query was successful, the second media query corresponding to second digital media data having a similar noise characteristic to the first noise characteristic; and

a transceiver circuit to;

in response to determining the prior classification attempt was successful, transmit the first digital media data to a remote media classification circuit that is to identify a source characteristic of the first digital media data; and

in response to determining the prior classification attempt was unsuccessful, inhibit the transmission of the first digital media data to the remote media classification circuit.

16. The media classification apparatus of claim 15, wherein accessing the first digital media data includes receiving an audio signal captured via a microphone.

17. The media classification apparatus of claim 15, wherein the first processor circuit is to:

access a first context parameter that corresponds to the first media query to be identified; and

determine a signal quality characteristic corresponding to the first context parameter; and

the transceiver circuit is to, in response to the signal quality characteristic corresponding to the first context parameter being less than a specified threshold signal quality, inhibit the transmission of the first digital media data to the remote media classification circuit.

18. The media classification apparatus of claim 15, wherein the first processor circuit is to:

retrieve a specified threshold amount of noise from a database of threshold noise characteristics, the database established by prior training of similar media queries with a neural network-based classifier system; and

determine if the first noise characteristic is less than the specified threshold amount of noise.

19. The media classification apparatus of claim 15, wherein transmission of the first digital media data to the remote media classification circuit includes providing the first digital media data to an input of a convolutional neural network classifier system.

20. The media classification apparatus of claim 15, wherein the first processor circuit is to receive, from the remote media classification circuit, a media characteristic of the first digital media data.