Automated music composition and generation system driven by emotion-type and style-type musical experience descriptors

• US 10,163,429 B2
• Filed: 04/17/2017
• Issued: 12/25/2018
• Est. Priority Date: 09/29/2015
• Status: Active Grant

• Current Assignee: Shutterstock Incorporated
• Original Assignee: Andrew H. Silverstein
• Inventors: Silverstein, Andrew H.
• Primary Examiner(s): Warren, David
• Application Number:
  US15/489,707
• Publication Number:
  US 20170263227A1
• Time in Patent Office: 617 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0025 Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece automatically producing a series of tones G10H1/26
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/021 Background music, e.g. for video sequences, elevator music musical accompaniment G10H2210/005
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/105 Composing aid, e.g. for supporting creation, edition or modification of a piece of music
• CPC Class Codes: G10H 2210/111 Automatic composing, i.e. using predefined musical rules
• CPC Class Codes: G10H 2210/115 using a random process to generate a musical note, phrase, sequence or structure using a random process to build a rhythm pattern G10H2210/356; random rhythm pattern selection G10H2210/366
• CPC Class Codes: G10H 2210/341 Rhythm pattern selection, synthesis or composition Rhythm analysis G10H2210/071; accompaniment G10H2210/005
• CPC Class Codes: G10H 2220/101 for graphical creation, edition or control of musical data or parameters
• CPC Class Codes: G10H 2240/081 Genre classification, i.e. descriptive metadata for classification or selection of musical pieces according to style analysis genre G10H2210/036
• CPC Class Codes: G10H 2240/085 Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
• CPC Class Codes: G10H 2240/131 Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
• CPC Class Codes: G10H 2240/305 Internet or TCP/IP protocol use for any electrophonic musical instrument data or musical parameter transmission purposes
• CPC Class Codes: G10H 2250/311 Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation musical analysis G10H2210/031; neural networks G06N3/02
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information

Abstract

An automated music composition and generation system for automatically composing and generating digital pieces of music using an automated music composition and generation engine driven by a set of emotion-type and style-type musical experience descriptors and time and/or space parameters supplied by a system user during an automated music composition and generation process. The system includes a system user interface allowing a system user to input (i) linguistic and/or graphical icon based musical experience descriptors, and (ii) a video, audio-recording, image, slide-show, or event marker, as input through the system user interface.

Claims

1. An automated music composition and generation system comprising:

a system user interface for enabling system users to create a project for a digital piece of music to be composed and generated, and review and select one or more emotion-type musical experience descriptors, one or more style-type musical experience descriptors, as well as time and/or space parameters; and

an automated music composition and generation engine, operably connected to said system user interface, for receiving, storing and processing said emotion-type and style-type musical experience descriptors and time and/or space parameters selected by the system user;

wherein said automated music composition and generation engine includes a plurality of function-specific subsystems cooperating together to automatically compose and generate one or more digital pieces of music in response to said emotion-type and style-type musical experience descriptors and time and/or space parameters selected by said system user;

wherein said digital piece of music composed and generated has a rhythmic landscape and a pitch landscape and contains a set of musical notes arranged and performed using an orchestration of one or more musical instruments selected for said digital piece of music being composed;

wherein said plurality of function-specific subsystems include a rhythmic landscape subsystem, a pitch landscape subsystem, and a digital piece creation subsystem;

wherein said rhythmic landscape subsystem is configured to generate and manage the rhythmic landscape of said digital piece of music being composed;

wherein said pitch landscape subsystem is configured to generate and manage the pitch landscape of said digital piece of music being composed;

wherein said digital piece creation subsystem is configured for creating said digital piece of music, employing one or more automated music synthesis techniques, and delivering said digital piece of music to said system user interface; and

wherein said digital piece of music composed and generated has the emotional and stylistic characteristics expressed throughout the rhythmic and pitch landscapes of said digital piece of music as represented by said set of emotion-type and style-type musical experience descriptors and time and/or space parameters supplied by said system user to said automated music composition and generation engine.

2. The automated music composition and generation system of claim 1, wherein said plurality of function-specific subsystems further includes a music editability subsystem that is interfaced with said system user interface, and configured for allowing system users to edit and modify each generated digital piece of music by using said system user interface to (i) edit the stored set of emotion-type and style-type musical experience descriptors and time and/or space parameters, and (ii) initiate said automated music composition and generation engine to generate a new digital piece of music using said edited set of stored emotion-type and style-type musical experience descriptors and time and/or space parameters.

3. The automated music composition and generation system of claim 2, wherein said plurality of function-specific subsystems further includes a controller code creation subsystem configured to create controller code for controlling the expression of the actual notes, rhythms, and instrumentation of said digital piece of music being composed.

4. The automated music composition and generation system of claim 3, configured as a digital video scoring workstation, wherein said system user interface receives a piece of digital media selected from the group consisting of a video, an audio-recording, an image, and an event marker, and then said automated music composition and generation engine generates said digital piece of music for musically scoring said piece of digital media, and then said digital piece of music is automatically combined with said piece of digital media to produced a piece of musically-scored digital media, and wherein said piece of musically-scored digital media is supplied back to said system user via said system user interface for experiencing and reviewing.

5. The automated music composition and generation system of claim 2, wherein said system user uses said music editability subsystem to edit said digital piece of music, and then recombines the edited digital piece of music with said piece of digital media.

6. The automated music composition and generation system of claim 1,

wherein said rhythmic landscape subsystem generates and manages the rhythmic landscape of said digital piece of music being composed, including the arrangement in time of all events in said digital piece of music being composed, and organizable at a high level by the musical piece'"'"'s tempo, meter, and length, at a middle level by the structure, form, and phrase of said digital piece of music being composed, and at a low level by the specific organization of events of each instrument and/or other component of said digital piece of music being composed;

and

wherein said pitch landscape subsystem generates and manages the pitch landscape of said digital piece of music being composed, including the arrangement in space of all events in said digital piece of music being composed, and organizable at a high level by the key and tonality of said digital piece of the music being composed, at a middle level by the structure, form, and phrase of said digital piece of music, and at a low level by the specific organization of events of each instrument and/or other component of said digital piece of music being composed.

7. The automated music composition and generation system of claim 6, wherein said rhythmic landscape subsystem comprises a general rhythm generation subsystem for generating a general rhythm for said digital piece of music being composed, and a melody rhythm generation subsystem for generating a melody rhythm for said digital piece of music being composed;

and wherein said pitch landscape subsystem comprises a general pitch generation subsystem for generating chords for said digital piece of music being composed, and a melody pitch generation subsystem for generating a melody pitch for said digital piece of music being composed.

8. The automated music composition and generation system of claim 3, wherein said plurality of function-specific subsystems further includes an orchestration subsystem configured to automatically manipulate, arrange, and/or adapt said digital piece of music being composed for performance by an ensemble of one or more virtual-instruments to provide said digital piece of music.

9. The automated music composition and generation system of claim 1, wherein said system user is a human being.

10. The automated music composition and generation system of claim 1, wherein said system user is a computer system.

11. The automated music composition and generation system of claim 1, wherein said system user interface is supported by a client system operably connected to a digital communications network;

and wherein said automated music composition and generation engine is supported by an application server system and a database server system also operably connected to said digital communications network.

12. The automated music composition and generation system of claim 1, wherein said emotional-type and said style-type musical experience descriptors are expressed as linguistic elements displayed on a graphical user interface (GUI) surface.

13. The automated music composition and generation system of claim 1, wherein said emotional-type and said style-type musical experience descriptors are expressed as icons displayed on a graphical user interface (GUI) surface.

14. The automated music composition and generation system of claim 1, wherein said emotional-type and said style-type musical experience descriptors are expressed as physical buttons bearing linguistic expressions.

15. The automated music composition and generation system of claim 1, wherein said one or more automated music synthesis techniques are based on methods selected from the group consisting of digital audio sampling synthesis methods, partial timbre synthesis methods, frequency modulation (FM) synthesis methods, and other forms of virtual instrument synthesis methods.

16. The automated music composition and generation system of claim 1, wherein said system user interface is supported by a client system operably connected to a digital communications network, and said wherein said automated music composition and generation engine is supported by an application server system and a database server system also operably connected to said digital communications network.

17. The automated music composition and generation system of claim 1, wherein said automated music composition and generation engine is realized as system-on-a-chip based (SOC-based) device, operably connected to said system user interface.

18. An automated music composition and generation system comprising:

an automated music composition and generation engine realized on a semiconductor chip containing electronic circuits; and

a system user interface allowing a system user to automatically compose and generate a digital piece of music by providing one or more emotion-type musical experience descriptors, one or more style-type musical experience descriptors, as well as time and/or space parameters to said system user interface;

wherein said automated music composition and generation engine is operably connected to said system user interface, for receiving, storing and processing said emotion-type and style-type musical experience descriptors and time and/or space parameters selected by said system user;

wherein said automated music composition and generation engine includes a plurality of function-specific subsystems cooperating together to automatically compose and generate one or more digital pieces of music in response to said emotion-type and style-type musical experience descriptors and time and/or space parameters selected by said system user;

wherein said digital piece of music composed and generated has a rhythmic landscape and a pitch landscape and contains a set of musical notes arranged and performed using an orchestration of one or more musical instruments selected for said digital piece of music;

wherein said plurality of function-specific subsystems include a rhythmic landscape subsystem, a pitch landscape subsystem, and a digital piece creation subsystem;

wherein said rhythmic landscape subsystem is configured to generate and manage the rhythmic landscape of said digital piece of music being composed;

wherein said pitch landscape subsystem is configured to generate and manage the pitch landscape of said digital piece of music being composed;

wherein said digital piece creation subsystem is configured for creating said digital piece of music, employing one or more automated music synthesis techniques, and delivering said digital piece of music to said system user interface; and

wherein said digital piece of music composed and generated has the emotional and stylistic characteristics expressed throughout the rhythmic and pitch landscapes of said digital piece of music as represented by said set of emotion-type and style-type musical experience descriptors and time and/or space parameters supplied by said system user to said automated music composition and generation engine.

19. The automated music composition and generation system of claim 18, wherein said semiconductor chip comprises:

a subsystem architecture including a CPU, program memory (RAM) and persistent memory.

20. The automated music composition and generation system of claim 18, wherein said plurality of function-specific subsystems further includes a music editability subsystem that is interfaced with said system user interface, and configured for allowing system users to edit and modify each generated digital piece of music by using said system user interface to (i) edit the stored set of emotion-type and style-type musical experience descriptors and time and/or space parameters, and (ii) initiate said automated music composition and generation engine to generate a new digital piece of music using said edited set of stored emotion-type and style-type musical experience descriptors and time and/or space parameters.

21. The automated music composition and generation system of claim 20, wherein said plurality of function-specific subsystems further includes a controller code creation subsystem configured to create controller code for controlling the expression of the actual notes, rhythms, and instrumentation of said digital piece of music being composed.

22. The automated music composition and generation system of claim 21, wherein said system user uses said music editability subsystem to edit said digital piece of music, and then recombines the edited digital piece of music with said piece of digital media.

23. The automated music composition and generation system of claim 22, configured as a digital video scoring workstation, wherein said system user interface receives a piece of digital media selected from the group consisting of a video, an audio-recording, an image, and an event marker, and then said automated music composition and generation engine generates said digital piece of music for musically scoring said piece of digital media, and then said digital piece of music is automatically combined with said piece of digital media to produced a piece of musically-scored digital media, and wherein said piece of musically-scored digital media is supplied back to said system user via said system user interface for experiencing and reviewing by the system user.

24. The automated music composition and generation system of claim 20, configured as a consumer electronic device.