Music selection and organization using audio fingerprints

• US 10,225,328 B2
• Filed: 01/11/2018
• Issued: 03/05/2019
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Muvox LLC
• Original Assignee: Aperture Investments LLC
• Inventors: Fuzell-Casey, Jacquelyn, Fuzell-Casey, Skyler, Casey, Timothy D.
• Primary Examiner(s): Gilliard, Delomia L
• Application Number:
  US15/868,902
• Publication Number:
  US 20180139268A1
• Time in Patent Office: 418 Days
• Field of Search: None
• CPC Class Codes: G06F 16/64 Browsing; Visualisation therefor generation of a list or set of audio data G06F16/638
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/071 for rhythm pattern analysis or rhythm style recognition
• CPC Class Codes: G10H 2220/131 for abstract geometric visualisation of music, e.g. for interactive editing of musical parameters linked to abstract geometric figures
• CPC Class Codes: G10H 2240/085 Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
• CPC Class Codes: G10H 2240/135 Library retrieval index, i.e. using an indexing scheme to efficiently retrieve a music piece
• CPC Class Codes: G10H 2250/235 Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A content selection system and method for identifying and organizing moods in content using objectively measured scores for rhythm, texture and pitch (RTP) and clustered into six mood classifications based on an objective analysis of the measured scores. Digitized representations of the content may also be identified and organized based on the content'"'"'s frequency data, three-dimensional shapes derived from the digitized representations, and colors derived from the frequency data. Each piece of content may be identified by at least a mood shape, but may also be identified by a mood color and/or a mood based on the clustered RTP scores and/or the digitized representation. Users of the selection system may be able to view the moods identified in the different manners, or combinations of two or three mood identifying manners and select and organize content based on the identified moods.

Claims

1. A method for identifying mood in content, comprising:

generating a static visual representation of an item of content based on frequency data sampled from the item of content;

filtering the static visual representation to capture intensity differences at specific frequencies represented in the static visual representation and generating a filtered representation of the item of content;

encoding the filtered representation to create a digitized representation of the item of content;

identifying characteristics within the digitized representation that correspond to a rhythm (R), a texture (T), and a pitch (P) within the item of content; and

determining a mood among a plurality of moods that correspond to the R, the T and the P.

2. The method of claim 1, wherein determining a mood includes objectively determining a R-value corresponding to the R, a T-value corresponding to the T and a P-value corresponding to the P, collectively representing a RTP score.

3. The method of claim 2, wherein the RTP score is a RTP vector, and wherein determining includes identifying a mood that corresponds to the RTP vector.

4. The method of claim 2, wherein the RTP score is a RTP vector, further comprising training a machine learning system based on the characteristics within the digitized representation that correspond to the RTP vector.

5. The method of claim 4, further comprising identifying an unknown mood of different items of content by generating, filtering, encoding and identifying the different items of content in accordance with claim 1, and determining the unknown mood with the trained machine learning system.

6. The method of claim 4, further comprising identify unknown moods for a plurality of different items of content by generating, filtering, encoding and identifying each of the plurality of different items of content in accordance with claim 1, and determining the unknown moods of each of the plurality of different items of content with the trained machine learning system.

7. The method of claim 6, wherein the item of content and different items of content are songs or include songs.

8. The method of claim 2, wherein the RTP score is a RTP vector, wherein the item of content is a song or includes a song, wherein the item of content has a first unique identification that is associated with the mood of the item of content that corresponds to the RTP vector, further comprising:

receiving a request to perform generating, filtering, encoding, identifying and determining for a second item of content having a second unique identification that matches the first unique identification; and

based on the request, identifying the mood of the second item of content based on the RTP vector corresponding to the first unique identification.

9. The method of claim 1, wherein the static visual representation is one or more of a spectrogram, a chromagram or an audio fingerprint.

10. A method for identifying characteristics of a rhythm (R), a texture (T) and a pitch (P) for an item of content, comprising:

generating a static visual representation of the content based on frequency data sampled from the content;

filtering the static visual representation to capture intensity differences at specific frequencies represented in the static visual representation and generating a filtered representation of the content;

encoding the filtered representation to create a digitized representation of the content; and

identifying characteristics within the digitized representation that correspond to the R, the T and the P for the item of content.

11. The method of claim 10, further comprising:

sending the identifying characteristics from a first computer to a second computer; and

receiving from the second computer at the first computer a determination of a mood that correspond to the R, the T and the P for the item of content.

12. The method of claim 11, wherein the determination is based on an objective determination of a R-value corresponding to the R, a T-value corresponding to the T and a P-value corresponding to the P, collectively representing a RTP score.

13. The method of claim 12, wherein the RTP score is a RTP vector, and wherein the determination is based on identifying a mood that corresponds to the RTP vector.

14. The method of claim 11, wherein a R-value corresponds to the R, a T-value corresponds to the T and a P-value corresponds to the P, collectively representing a RTP score, wherein the determination is based on a comparison of the RTP score to a plurality of previously determined RTP scores for a plurality of items of content.

15. The method of claim 14, wherein the comparison is also based on a unique identifier associated with the item of content on other unique identifiers associated with each of the plurality of items of content.

16. The method of claim 10, wherein the item of content is a song or includes a song.

17. The method of claim 10, further comprising:

determining a mood that correspond to the R, the T and the P for the item of content; and

including the mood in metadata associated with the item of content.

18. The method of claim 16, wherein the item of content is a song or includes a song.

19. The method of claim 10, further comprising:

determining a mood that correspond to the R, the T and the P for the item of content; and

including the mood in streaming data associated with the item of content.

20. The method of claim 19, wherein the item of content is a song or includes a song.