Machines, systems, processes for automated music composition and generation employing linguistic and/or graphical icon based musical experience descriptions

• US 9,721,551 B2
• Filed: 09/29/2015
• Issued: 08/01/2017
• Est. Priority Date: 09/29/2015
• Status: Active Grant

• Current Assignee: Shutterstock Incorporated
• Original Assignee: Amper Music, Inc. (Shutterstock Incorporated)
• Inventors: Silverstein, Andrew H.
• Primary Examiner(s): WARREN, DAVID S
• Application Number:
  US14/869,911
• Publication Number:
  US 20170092247A1
• Time in Patent Office: 672 Days
• Field of Search: 84600
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0025 Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece automatically producing a series of tones G10H1/26
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/021 Background music, e.g. for video sequences, elevator music musical accompaniment G10H2210/005
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/105 Composing aid, e.g. for supporting creation, edition or modification of a piece of music
• CPC Class Codes: G10H 2210/111 Automatic composing, i.e. using predefined musical rules
• CPC Class Codes: G10H 2210/115 using a random process to generate a musical note, phrase, sequence or structure using a random process to build a rhythm pattern G10H2210/356; random rhythm pattern selection G10H2210/366
• CPC Class Codes: G10H 2210/341 Rhythm pattern selection, synthesis or composition Rhythm analysis G10H2210/071; accompaniment G10H2210/005
• CPC Class Codes: G10H 2220/101 for graphical creation, edition or control of musical data or parameters
• CPC Class Codes: G10H 2240/081 Genre classification, i.e. descriptive metadata for classification or selection of musical pieces according to style analysis genre G10H2210/036
• CPC Class Codes: G10H 2240/085 Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
• CPC Class Codes: G10H 2240/131 Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
• CPC Class Codes: G10H 2240/305 Internet or TCP/IP protocol use for any electrophonic musical instrument data or musical parameter transmission purposes
• CPC Class Codes: G10H 2250/311 Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation musical analysis G10H2210/031; neural networks G06N3/02
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information

Abstract

Automated music composition and generation machine, systems and methods, and architectures that allow anyone, without possessing any knowledge of music theory or practice, or expertise in music or other creative endeavors, to instantly create unique and professional-quality music, synchronized to any kind of media content, including, but not limited to, video, photography, slideshows, and any pre-existing audio format, as well as any object, entity, and/or event, wherein the system user only requires knowledge of ones own emotions and/or artistic concepts which are to be expressed in a piece of music that will ultimately composed by the automated composition and generation system of the present invention.

Claims

1. An automated music composition and generation system for automatically composing and generating digital pieces of music using an automated music composition and generation engine controlled by emotion/style-indexed music-theoretic system operating parameters (SOP) produced for each digital piece of music being composed and generated in response to a set of emotion-type and style-type musical experience descriptors and time and/or space parameters supplied by a system user during an automated music composition and generation process, said automated music composition and generation system comprising:

a system user interface for enabling system users to (i) create a project for a digital piece of music to be composed and generated, and (ii) review and select one or more emotion-type musical experience descriptors, one or more style-type musical experience descriptors, as well as time and/or space parameters; and

an automated music composition and generation engine, operably connected to said system user interface, for receiving said emotion-type and style-type musical experience descriptors and time and/or space parameters selected by the system user at said system user interface;

wherein said automated music composition and generation engine includes a plurality of function-specific subsystems cooperating together to automatically compose and generate one or more digital pieces of music in response to said emotion-type and style-type musical experience descriptors and time and/or space parameters selected by the system user at said system user interface;

wherein each said digital piece of music to be composed and generated has a rhythmic landscape and a pitch landscape and contains a set of musical notes arranged and performed using an orchestration of one or more musical instruments selected for the digital piece of music;

wherein said digital piece of music has musical elements including (i) structure, form and phrase, (ii) tempo, meter and length, and (iii) key and tonality;

wherein said plurality of function-specific subsystems include a rhythmic landscape subsystem, a pitch landscape subsystem, and a controller code creation subsystem;

wherein each said function-specific subsystem supports and employs emotion/style-indexed music-theoretic system operating parameter tables for performing specific music theoretic operations during said automated music composition and generation process;

a parameter transformation subsystem for receiving said emotion-type and style-type musical experience descriptors and time and/or space parameters from said system user interface, and processing and transforming said emotion-type and style-type musical experience descriptors and time and/or space parameters and producing emotion/style-indexed music-theoretic system operating parameters for use by said function-specific subsystems employing emotion/style-indexed music-theoretic system operating parameter tables during said automated music composition and generation process;

a parameter storage subsystem for persistent storage and archiving of system user accounts and music composition projects, and all emotion/style-indexed music-theoretic system operating parameters generated by said parameter transformation subsystem for said music composition projects created by system users;

a parameter handling and processing subsystem operably connected to said parameter storage subsystem and said parameter transformation subsystem for (i) receiving emotion/style-specific music-theoretic system operating parameters produced by said parameter transformation subsystem, and (ii) loading said emotion/style-indexed music-theoretic system operating parameters within said function-specific subsystems employing said emotion/style-indexed musical-theoretic system operating parameter tables;

wherein said rhythmic landscape subsystem is configured to generate and manage the rhythmic landscape of the digital piece of music being composed, including the arrangement in time of all events in the digital piece of music being composed, and organizable at a high level by the tempo, meter, and length of the digital piece of music, at a middle level by the structure, form, and phrase of the digital piece of music, and at a low level by a specific organization of events of each musical instrument and/or other components of the digital piece of music being composed;

wherein said pitch landscape subsystem is configured to generate and manage the pitch landscape of the digital piece of music being composed, including the arrangement in space of all events in the digital piece of music being composed, and organizable at a high level by the key and tonality of the digital piece of the music, at a middle level by the structure, form, and phrase of the digital piece of music, and at a low level by a specific organization of events of each musical instrument and/or other components of the digital piece of music being composed;

wherein said controller code creation subsystem is configured to create controller code to control the expression of the musical notes, rhythms, and musical instruments orchestrated in said digital piece of music being composed;

a digital piece creation subsystem for creating the digital piece of music, employing one or more automated virtual-instrument music synthesis techniques;

wherein during said automated music composition and generation process, said function-specific subsystems are controlled by the emotion/style-indexed music-theoretic system operating parameters loaded within said emotion/style-indexed music-theoretic system operating parameters (SOP) tables supported within said function-specific subsystems, while the digital piece of music composed and generated has the emotional and stylistic characteristics expressed throughout the rhythmic and pitch landscapes of the digital piece of music as represented by said set of emotion-type and style-type musical experience descriptors and time and/or space parameters supplied by said system user; and

a music editability subsystem, interfaced with said system user interface, allowing system users to edit and modify generated digital pieces of music by (i) using said system user interface to edit the set of emotion-type and style-type musical experience descriptors and time and/or space parameters stored in said parameter storage subsystem, (ii) using said parameter transformation subsystem to transform said edited set of emotion-type and style-type musical experience descriptors and time and/or space parameters into a new set of emotion/style-indexed music-theoretic system operating parameters (SOP) for storage in said parameter storage subsystem and loading within said function-specific subsystems by said parameter handling and processing subsystem, and (iii) using said automated music composition and generation engine to generate a new digital piece of music using said new set of emotion/style-indexed music-theoretic system operating parameters.

2. The automated music composition and generation system of claim 1 which is configured as a digital video scoring workstation, and wherein said system user interface receives a piece of digital media selected from the group consisting of a video, an audio-recording, an image, or an event marker, and then said automated music composition and generation engine generates said digital piece of music for musically scoring said piece of digital media, and then the digital piece of music is automatically combined with said piece of digital media to produced a piece of musically-scored digital media, wherein said piece of musically-scored digital media is supplied back to said system user via said system user interface for experiencing and reviewing.

3. The automated music composition and generation system of claim 2, wherein said system user uses said music editability subsystem to edit said digital piece of music, and then recombines the edited digital piece of music with said piece of digital media.

4. The automated music composition and generation system of claim 1, wherein said rhythmic landscape subsystem comprises a general rhythm generation subsystem for generating a general rhythm for the piece of music being composed, and a melody rhythm generation subsystem for generating a melody rhythm for the piece of music being composed;

and wherein said pitch landscape subsystem comprises a general pitch generation subsystem for generating chords for the piece of music being composed, and a melody pitch generation subsystem for generating a melody pitch for the piece of music being composed.

5. The automated music composition and generation system of claim 1, wherein said digital piece creation subsystem comprises a piece deliverer subsystem for delivering said digital piece of music to said system user interface.

6. The automated music composition and generation system of claim 1, wherein said plurality of function-specific subsystems further include an orchestration subsystem configured to automatically manipulate, arrange, and/or adapt a digital piece of music being composed for performance by an ensemble of one or more virtual-instruments to provide said digital piece of music.

7. The automated music composition and generation system of claim 1, wherein said system user is a human being.

8. The automated music composition and generation system of claim 1, wherein said system user is a computer system.

9. The automated music composition and generation system of claim 1, wherein one or more of said emotion/style-indexed music-theoretic system operating parameters comprise probability-based music-theoretic system operating parameters.

10. The automated music composition and generation system of claim 1, wherein said system user interface is supported by a client system operably connected to a digital communications network, and said wherein said automated music composition and generation engine is supported by an application server system and a database server system also operably connected to said digital communications network.

11. The automated music composition and generation system of claim 10, which further comprises one or more remote workstation client systems allowing one or more system designers to remotely configure said parameter transformation subsystem so that said automated music composition and generation system supports a first plurality of said emotion-type musical experience descriptors and a second plurality of said style-type musical experience descriptors.

12. The automated music composition and generation system of claim 1, wherein said emotional-type and said style-type musical experience descriptors are expressed as linguistic elements displayed on a graphical user interface (GUI) surface.

13. The automated music composition and generation system of claim 1, wherein said emotional-type and said style-type musical experience descriptors are expressed as icons displayed on a graphical user interface (GUI) surface.

14. The automated music composition and generation system of claim 1, wherein said emotional-type and said style-type musical experience descriptors are expressed as physical buttons bearing linguistic expressions.

15. The automated music composition and generation system of claim 1, wherein said one or more automated virtual-instrument music synthesis techniques are based on methods selected from the group consisting of digital audio sampling synthesis methods, partial timbre synthesis methods, frequency modulation (FM) synthesis methods, and other forms of virtual instrument synthesis methods.