Systems and methods for the creation and playback of animated, interpretive, musical notation and audio synchronized with the recorded performance of an original artist

• US 9,472,177 B2
• Filed: 12/27/2013
• Issued: 10/18/2016
• Est. Priority Date: 09/19/2002
• Status: Active Grant

• Current Assignee: Fiver LLC
• Original Assignee: Family Systems, Ltd.
• Inventors: Hudak, William B., Reynolds, Brian
• Primary Examiner(s): FLETCHER, MARLON T
• Application Number:
  US14/142,357
• Publication Number:
  US 20140109750A1
• Time in Patent Office: 1,026 Days
• US Class Current: 1/1
• CPC Class Codes: G09B 15/00 Teaching music metronomes G04F5/02
• CPC Class Codes: G09B 15/02 Boards or like means for providing an indication of notes
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/381 Manual tempo setting or adjustment
• CPC Class Codes: G10H 2220/015 Musical staff, tablature or score displays, e.g. for score reading during a performance
• CPC Class Codes: G10H 2220/101 for graphical creation, edition or control of musical data or parameters
• CPC Class Codes: G10H 2240/056 MIDI or other note-oriented file format

Abstract

A music application guides a user with some musical, experience through the steps of creating and editing a musical enhancement file that enhances and plays in synchronicity with an audio signal of an original artist'"'"'s recorded performance. This enables others, perhaps with lesser musical ability than the original artist, to play-along with the original artist by following melodic, chordal, rhythmic, and verbal prompts. The music application accounts for differences in the timing of the performance from a standard tempo by guiding the user through the process of creating a tempo map for the performance and by associating the tempo map with MIDI information of the enhancement file. Enhancements may contain. MIDI information, audio signal information, and/or video signal information which may be played back in synchronicity with the recorded performance to provide an aural and visual, aid to others playing-along who may have less musical experience.

Claims

1. A method comprising:

capturing content from a user via an input device while the user is listening to audio or viewing video;

as the content from the user is captured, temporally linking the captured content to the audio or video by creating one or more time codes, wherein each time code is indicative of a difference between a first time that the audio or video begins and a second time that the content from the user is captured;

playing the captured content in synchronicity with the audio or video during playback of the audio or video, wherein the captured content is played at a time relative to the audio or video specified by the one or more time codes; and

adjusting at least one of the one or more time codes based on an input from the user.

2. The method of claim 1, wherein the content is audio content and the input device is an audio input device.

3. The method of claim 1 further comprising:

receiving user input of additional audio content to be associated with the audio or video;

assigning the additional audio content to respective portions of the captured content; and

playing the additional content in synchronicity with the playing of the audio or video based on the temporal link between the audio or video and the captured content.

4. The method of claim 1 further comprising:

receiving user input of additional visual content to be associated with the audio or video;

assigning the additional visual content to respective portions of the captured content; and

displaying the additional visual content in synchronicity with the playing of the audio or video based on the temporal link between the audio or video and the captured content.

5. The method of claim 4, wherein the additional visual content is at least one of text and video.

6. The method of claim 4, wherein the additional visual content is textual information corresponding to lyrics of the audio.

7. The method of claim 1, wherein the user is a first user, the method further comprising:

enabling a second user to play the captured content in synchronicity with the audio or video, wherein the captured content is played at a time relative to the audio or video specified by the one or more time codes.

8. A system comprising:

an input device that captures content from a user while the user is listening to audio or viewing video;

a processor that, as the content from the user is captured, temporally links the captured content to the audio or video by creating one or more time codes, wherein each time code is indicative of a difference between a first time that the audio or video begins and a second time that the content from the user is captured and that adjusts at least one of the one or more time codes based on an input from the user; and

an output device that plays the captured content in synchronicity with the audio or video during playback of the audio or video, wherein the captured content is played at a time relative to the audio or video specified by the one or more time codes.

9. The system of claim 8, wherein the content is audio content and the input device is an audio input device.

10. The system of claim 8, wherein:

the input device further receives user input of additional audio content to be associated with the audio or video;

the processor further assigns the additional audio content to respective portions of the captured content; and

the output device further plays the additional content in synchronicity with the playing of the audio or video based on the temporal link between the audio or video and the captured content.

11. The system of claim 8, wherein:

the input device further receives user input of additional visual content to be associated with the audio or video;

the processor further assigns the additional visual content to respective portions of the captured content; and

the output device further displays the additional visual content in synchronicity with the playing of the audio or video based on the temporal link between the audio or video and the captured content.

12. The system of claim 11, wherein the additional visual content is at least one of text and video.

13. The system of claim 11, wherein the additional visual content is textual information corresponding to lyrics of the audio.

14. The system of claim 8, wherein the user is a first user, the system further comprising:

a server that enables a second user to play the captured content in synchronicity with the audio or video, wherein the captured content is played at a time relative to the audio or video specified by the one or more time codes.

15. A method for enhancing audio or video, the method comprising:

enabling a first user to create at least one enhancement that accompanies the audio or video, wherein the enhancement comprises at least one of a visual or aural representation of at least one aspect of the content of the audio or video or content associated with the audio or video;

enabling the first user or a second user to synchronize the at least one enhancement to the audio or video by temporally linking the at least one enhancement to the audio or video by creating one or more time codes, wherein each time code is indicative of a difference between a first time that the audio or video begins and a second time that the content is captured;

enabling the first user or the second user to provide input to adjust at least one of the one or more time codes;

enabling the first user or the second user to store the at least one synchronized enhancement; and

enabling at least one of the first user or the second user to play the stored synchronized enhancement synchronously with the audio or video.

16. The method of claim 15 further comprising playing the synchronized enhancement without playing the audio or video.

17. The method of claim 15, wherein enabling the first user to synchronize the at least one enhancement to the audio or video comprises enabling the first user to specify that the enhancement starts playing a set time from the start of playback of the audio or video.

18. The method of claim 15 further comprising:

storing the at least one synchronized enhancement at a remote server; and

retrieving the at least one synchronized enhancement from the remote server in response to a user request to play the stored synchronized enhancement.

19. The method of claim 15, wherein playing the stored synchronized enhancement further comprises presenting at least one of textual, video, visual, audio, melodic, harmonic, chordal, lyrical, rhythmical, and instrumental data.