Music composition and generation instruments and music learning systems employing automated music composition engines driven by graphical icon based musical experience descriptors

• US 10,262,641 B2
• Filed: 04/17/2017
• Issued: 04/16/2019
• Est. Priority Date: 09/29/2015
• Status: Active Grant

• Current Assignee: Shutterstock Incorporated
• Original Assignee: Amper Music, Inc. (Shutterstock Incorporated)
• Inventors: Silverstein, Andrew H.
• Primary Examiner(s): Warren, David S
• Application Number:
  US15/489,672
• Publication Number:
  US 20170263225A1
• Time in Patent Office: 729 Days
• Field of Search: 84609
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0025 Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece automatically producing a series of tones G10H1/26
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/021 Background music, e.g. for video sequences or elevator music
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/105 Composing aid, e.g. for supporting creation, edition or modification of a piece of music
• CPC Class Codes: G10H 2210/111 Automatic composing, i.e. using predefined musical rules
• CPC Class Codes: G10H 2210/115 using a random process to generate a musical note, phrase, sequence or structure
• CPC Class Codes: G10H 2210/341 Rhythm pattern selection, synthesis or composition
• CPC Class Codes: G10H 2220/101 for graphical creation, edition or control of musical data or parameters
• CPC Class Codes: G10H 2240/081 Genre classification, i.e. descriptive metadata for classification or selection of musical pieces according to style
• CPC Class Codes: G10H 2240/085 Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
• CPC Class Codes: G10H 2240/131 Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
• CPC Class Codes: G10H 2240/305 Internet or TCP/IP protocol use for any electrophonic musical instrument data or musical parameter transmission purposes
• CPC Class Codes: G10H 2250/311 Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information

Abstract

A toy musical instrument having a compact housing supporting an automated music composition and generation engine that is driven by icon-based musical experience descriptors and musical style descriptors, selected by a child or adult during a video scoring process.

Claims

1. A portable automated music composition and generation system for scoring videos with custom pieces of digital music, said portable automated music composition and generation system comprising:

a compact housing;

an automated music composition and generation engine, mounted in said compact housing, and driven by graphical-icon based emotion-type musical experience descriptors and graphical-icon based style type musical experience descriptors selected by a system user during a video scoring process; and

a computing platform, mounted in said compact housing, and supporting a system user interface operably coupled to said automated music composition and generation engine;

wherein said computing platform includes

a processor with program memory for storing a control program, and persistent memory for storing a library of videos and/or other photographic media to be scored by the system user with custom pieces of digital music;

a touch-screen display panel for displaying a selected video and/or photographic media to be scored by the system user with custom pieces of digital music that have been automatically generated by said automated music composition and generation engine driven by said graphical-icon based emotion-type musical experience descriptors graphical-icon based style-type musical experience descriptors;

a keyboard allowing the system user to select graphical-icon based emotion-type musical experience descriptors and graphical-icon based style-type musical experience descriptors that emotionally and stylistically characterize custom pieces of digital music to be automatically composed and generated by said automated music composition and generation engine, for scoring segmented scenes of the selected video and/or photographic media;

an audio speaker for producing audio representations of custom pieces of digital music automatically composed by said automated music composition and generation engine driven by said graphical-icon based emotion-type musical experience descriptors and said graphical-icon based style-type musical experience descriptors selected by the system user; and

a wireless network adapter, disposed within said compact housing, allowing said computing platform to establish wireless communication with one or more devices operably connected to a wireless data communications network;

wherein said automated music composition and generation engine automatically generates custom pieces of digital music for scoring segmented scenes in the selected video and/or photographic media, based on and using said graphical-icon based emotion-type musical experience descriptors and said graphical-icon based style-type musical experience descriptors selected by the system user using said keyboard, and provided to said automated music composition and generation engine;

wherein said touch screen display panel allows the system user to select and load a video and/or photographic media from said library and view segmented scenes of the selected video and/or photographic media, and the system user can then select said graphical-icon based emotion-type musical experience descriptors and said graphical-icon based style-type musical experience descriptors using said keyboard, and supply the selected graphical-icon based emotion-type and style-type musical experience descriptors to said automated music composition and generation engine for use in composing and generating custom pieces of digital music for scoring segmented scenes in the selected video and/or photographic media; and

wherein said automated music composition and generation engine, once initiated by the system user, automatically transforms said selected graphical-icon based emotion-type and style-type musical experience descriptors into a set of music-theoretic system operating parameters, which are used by said automated music composition and generation engine to automatically compose and generate custom pieces of digital music for scoring the segmented scenes in said selected video and/or photographic media.

2. The portable automated music composition and generation system of claim 1, wherein said automated music composition and generation engine employs virtual-instrument music synthesis during automated music composition.

3. An automated music composition and generation process supported within a portable music composition and generation system, said automated music composition and generation process comprising the steps of:

(a) providing a graphical-icon supporting system user interface, including a video display, operably connected to an automated music composition and generation engine driven by graphical-icon based emotion-type and style-type musical experience descriptors supplied by a system user, interacting with said graphical-icon supporting system-user interface and selecting said graphical-icon based emotion-type and style-type musical experience descriptors from said graphical-icon supporting system user interface;

(b) a system user accessing said automated music composition and generation engine, and then selecting a video to be scored with custom pieces of digital music composed and generated by said automated music composition and generation engine;

(c) said system user selecting graphical-icon based emotion-type and style-type musical experience descriptors that emotionally and stylistically characterize custom pieces of digital music to be automatically composed and generated by said automated music composition and generation engine, for scoring segmented scenes in the selected video;

(d) said system user initiating said automated music composition and generation engine to compose and generate custom pieces of digital music for the segmented scenes in said selected video, using said graphical-icon based emotion-type and style-type musical experience descriptors selected for the segmented scenes in the selected video;

(e) said portable music composing and generation system combining the composed custom pieces of digital music with the selected video so as to create a video file for display and review by said system user; and

(f) reviewing and assessing said video file and making modifications to the selected graphical-icon based emotion-type and style-type musical experience descriptors and providing the modified graphical-icon based emotion-type and style-type musical experience descriptors to said automated music composition and generation engine, and then creating a new video file for display and audio-visual review;

wherein said automated music composition and generation engine, once initiated by the system user, automatically transforms said selected graphical-icon based emotion-type and style-type musical experience descriptors into a set of music-theoretic system operating parameters, which are used by said automated music composition and generation engine to automatically compose and generate custom pieces of digital music for scoring the segmented scenes in said selected video.

4. A portable automated music composition and generation system for scoring videos with custom pieces of digital music, said portable automated music composition and generation system comprising:

a portable housing;

an automated music composition and generation engine, disposed in said portable housing, driven by graphical-icon based musical experience descriptors selected by a system user using said portable automated music composition and composition system;

a manual selection mechanism, integrated with said portable housing, for enabling said system user to manually select said graphical-icon based musical experience descriptors that emotionally and stylistically characterize custom pieces digital music to be automatically composed and generated by said automated music composition and generation engine, for scoring segmented scenes in a video selected by the system user;

a computing platform, disposed in said portable housing, operably connected to said automated music composition and generation engine, for controlling said portable automated music composition and generation system, and having a storage device for storing a video library of videos and custom pieces of digital music generated by said automated music composition and generation engine;

a video display screen, integrated with said portable housing, for displaying videos selected from said video library maintained within said storage device, or on a local or remote video file server connected to an information network, and for displaying segmented scenes in the selected video being scored with said graphical-icon based musical experience descriptors;

wherein the system user uses said manual selection mechanism to select one or more graphical-icon based musical experience descriptors for supply to said automated music composition and generation engine for use in automatically composing and generating custom pieces of digital music for scoring segmented scenes in the selected video; and

an audio transducer for producing audio representations of said custom pieces of digital music automatically composed and generated by said automated music composition and generation engine; and

wherein said automated music composition and generation engine, once initiated by the system user, automatically transforms said selected graphical-icon based musical experience descriptors into a set of music-theoretic system operating parameters, which are used by said automated music composition and generation engine to automatically compose and generate said custom pieces of digital music for scoring segmented scenes in said selected video.

5. The portable automated music composition and generation system of claim 4, wherein said manual selection mechanism comprises at least one of a physical keyboard and a virtual keyboard.

6. The portable automated music composition and generation system of claim 5, wherein said graphical-icon based musical experience descriptors comprises graphical-icon based emotion-type musical experience descriptors and graphical-icon based style-type musical experience descriptors.

7. A portable automated music composition and generation system for producing musically-scored videos, said portable automated music composition and generation system comprising:

a portable housing;

an automated music composition and generation engine, disposed in said portable housing, for use in scoring segmented scenes in a video; and

a system user interface, disposed in said portable housing, including (i) a video display screen for displaying a video selected by a system user and scored with custom pieces of digital music automatically composed and generated by said automated music composition and generation engine, and (ii) a manual selection mechanism, integrated with said portable housing, for use by said system user in selecting graphical-icon based emotion-type and style-type musical experience descriptors that emotionally and stylistically characterize custom pieces of digital music to be automatically composed and generated by said automated music composition and generation engine, for scoring segmented scenes in the video selected by the system user, and providing said graphical-icon based emotion-type and style-type musical experience descriptors to said automated music composition and generation engine for use in automatically composing and generating said custom pieces of digital music;

an audio transducer for producing audio representations of said custom pieces of digital music automatically composed and generated by said automated music composition and generation engine, for scoring segmented scenes in said selected video;

wherein said musically-scored video is then supplied back to the system user by way of said system user interface for audio-visual review on said video display screen and using said audio transducer; and

wherein said automated music composition and generation engine, once initiated by the system user, automatically transforms said selected graphical-icon based emotion-type and style-type musical experience descriptors into a set of music-theoretic system operating parameters, which are used by said automated music composition and generation engine to automatically compose and generate custom pieces of digital music for scoring segmented scenes in said video and producing said musically-scored video.

8. The portable automated music composition and generation system of claim 7, wherein said system user interface, allowing the system user to select and load videos from a video library maintained within a storage device within said portable automated music composition and generation system, or from a local or remote video file server connected to an information network, and the system user can then select said graphical-icon based emotion-type and style-type musical experience descriptors using said manual selection mechanism for supply to said automated music composition and generation engine and subsequent use in automatically composing and generating custom pieces of digital music for scoring segmented scenes in said selected video so as to produce said musically-scored video.

9. The portable automated music composition and generation system of claim 8, wherein said automated music composition and generation engine supports automated virtual-instrument music synthesis driven by said graphical-icon based emotion-type and style-type musical experience descriptors selected by the system user for use in said portable automated music composition and generation system.