Analyzing or emulating a keyboard performance using audiovisual dynamic point referencing

• US 10,170,017 B2
• Filed: 04/04/2018
• Issued: 01/01/2019
• Est. Priority Date: 10/15/2010
• Status: Active Grant

• Current Assignee: Jammit, Inc.
• Original Assignee: Jammit, Inc.
• Inventors: Humphrey, Scott, Gryner, Frank
• Primary Examiner(s): Hull, James
• Application Number:
  US15/945,575
• Publication Number:
  US 20180225986A1
• Time in Patent Office: 272 Days
• Field of Search: 434250
• CPC Class Codes: G06Q 50/20 Education
• CPC Class Codes: G09B 15/04 with sound emitters
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 19/0015 Dancing
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems

Abstract

The systems and methods taught herein are generally directed to a dynamic point referencing of an audiovisual performance for an accurate and precise selection and controlled cycling of portions of the performance. The dynamic point referencing can be used by a learning artist, for example, in analyzing or performing a portion of the work through an accurate and precise digital audio/video instructional method having the controlled cycling feature. Such systems and methods will be appreciated, for example, by musicians, dancers, and other enthusiasts of the performing arts.

Claims

1. An audiovisual work including a video keyboard performance, the work comprising:

a video portion and an audio portion, wherein the work includes a video keyboard performance of a musician displaying combinations of finger movements that follow the audio portion; and

,

a dynamic point reference grid within the work for an accurate and precise selection and point-cycling of a desired portion of the work by a user, wherein the dynamic point reference includes a custom tempo map of the audio portion that is adaptable for a plurality of tempos within the work;

wherein

the video portion and the audio portion are stored on a non-transitory computer readable medium with instructions for execution by a processor;

the audio portion comprises a recording of a musical instrument having a fluctuation in tempos;

the custom tempo map is configured to include a mapping of the distance between bars or beats, the distance measured as time;

where, the mapping captures the fluctuation in tempos and composes the dynamic point reference grid for the accurate and precise selection of a point of reference for point-cycling of the desired portion of the work.

2. The work of claim 1, wherein the audio data comprises an originally recorded work.

3. The work of claim 1, wherein the audio data comprises a prerecorded audio data.

4. The work of claim 1, wherein the desired portion comprises a single musical note indexed to the custom tempo map.

5. The work of claim 1, wherein the desired portion consists of a single repeating rhythmic pattern indexed to the custom tempo map.

6. The work of claim 1, wherein the dynamic point reference grid is configured to facilitate a series of accurate and precise selections from the custom tempo map, each selection in the series of selections consisting of a musical note, a plurality of beats, a bar, a plurality of bars, a repeating rhythmic pattern, or any combination thereof.

7. The work of claim 1, wherein the video includes a plurality of instructional views of the musician performing the work for selection by the user.

8. The work of claim 1, wherein the work includes a multi-track audio portion comprising an isolated audio track consisting of a solo performance.

9. The work of claim 1, wherein the work includes a multi-track audio portion comprising a plurality of isolated audio tracks consisting of a plurality of solo performances.

10. The work of claim 1, wherein the custom tempo map comprises an assignment of a transient to a period of silence in the audio track having the fluctuation in tempos.

11. A method of creating the work of claim 1, the method comprising:

selecting the work with the fluctuation of tempos; and

,

custom-mapping the audio portion of the work to create the dynamic point reference to facilitate an accurate and precise selection, and point-cycling, of the desired portion of the work.

12. A method of analyzing or emulating a keyboard performance through an accurate and precise audiovisual process, the method comprising:

obtaining an audiovisual work having an audio portion having a fluctuation in tempos, a video portion having a video keyboard performance of a musician displaying combinations of finger movements that follow the audio portion, and a dynamic point reference grid having a custom tempo map of the audio portion for selecting a desired portion of the audiovisual work;

selecting a point of reference in the desired portion of the work, the point of reference defined by the dynamic point reference grid;

analyzing or emulating the performance; and

, point-cycling the desired portion;

wherein, the custom tempo map is configured to include a mapping of the distance between bars or beats, the distance measured as time;

where, the mapping captures the fluctuation in tempos and composes the dynamic point reference grid for an accurate and precise selection of a point of reference for the point-cycling of the desired portion of the work.

13. The method of claim 12;

wherein,

the audiovisual work has a multi-track audio file derived from a multi-track audio recording;

the multi-track audio file comprises an isolated instrument audio track and a residual component track, the isolated instrument audio track comprising a single musical instrument playing a preselected piece of music that the user desires to hear when analyzing or emulating the keyboard performance;

the analyzing or emulating includes

transforming the composition of the multi-track audio file to include selecting a gain ratio of volumes of (i) the isolated instrument audio track to (ii) the residual component track, wherein the residual component track represents a subtraction of the isolated instrument audio track from the plurality of audio tracks; and

,

selecting the desired portion, watching the video, listening to the isolated instrument audio track in the desired portion, and analyzing or emulating the desired portion.

14. The method of claim 13, the method further comprising:

recording an emulated keyboard video track on a non-transitory computer readable storage medium;

combining the user'"'"'s keyboard video track with the residual component track to create an educational audio file; and

,

comparing the emulated keyboard video track to that of the performer'"'"'s keyboard video track.

15. The method of claim 12, wherein the method further comprises emulating the audio portion which is controlled and indexed through the custom tempo map, wherein the keyboard performance includes an instrument performance of the audio portion.

16. The method of claim 12, wherein the desired keyboard video portion includes a musical note, a plurality of beats, a bar, a plurality of bars, a repeating rhythmic pattern, or any combination thereof, of the isolated instrument audio track.

17. The method of claim 12, wherein the emulating further comprises reading a digital musical transcription and tablature display corresponding to the isolated instrument audio track.

18. The method of claim 12, wherein the audio portion further comprises an audible dynamic point reference track, the method comprising listening to the audible dynamic point-reference track that is designed for the preselected piece of music.

19. The method of claim 12, wherein the desired portion plays back at a desired speed that matches the ability of the user.

20. An audiovisual system for analyzing or emulating a keyboard video track displaying combinations of finger movements that follow an audio portion, the system comprising:

a processor;

an input device operable to receive audio and video data on a non-transitory computer readable storage medium;

a database operable to store audiovisual files for access on a non-transitory computer readable storage medium;

an audio engine embodied in a non-transitory computer readable storage medium, wherein the audio engine is operable to transform input audio data to output audio data;

a video engine embodied in a non-transitory computer readable storage medium, wherein the video engine is (i) operable to transform input video data to output video data having a keyboard video track displaying combinations of movements that follow the audio portion and (ii) configured to display an indexed version of the keyboard video track that can be accurately and precisely parsed, and adjusted for a desired tempo;

a dynamic point reference module embodied in a non-transitory computer readable storage medium, wherein the dynamic point reference module is operable through a custom tempo map to create a dynamic point reference grid for the video keyboard performance having a video portion and an audio portion, the dynamic point reference grid indexed to the custom tempo map of the audio portion, the audio portion comprising a recording of a musical instrument having a fluctuation in tempos;

an output module embodied in a non-transitory computer readable medium, wherein the output module is operable to (i) transmit the audio data and the video data to an output device in the form of an accurate and precise selection of a desired portion of the keyboard video track and (ii) transmit a point-cycling of the desired portion of the keyboard video track to a user; and

,

the output device operable to provide the audio data and the video data to the user;

wherein, the custom tempo map is configured to include a mapping of the distance between bars or beats, the distance measured as time;

where, the mapping captures the fluctuation in tempos and composes the dynamic point reference grid.

21. The system of claim 20, wherein the dynamic point reference is constructed to include a transient in the custom tempo map representing a point of silence as a reference point for the point-cycling of the desired portion.

22. The system of claim 20, further comprising

a transformation module embodied in a non-transitory computer readable storage medium, wherein the audio portion comprises a multi-track file having an isolated instrument audio track and a residual component track, and the transformation module is operable to transform the audio portion into a ratio of (i) the isolated instrument audio track to (ii) the residual component track, the residual component track representing a subtraction of the isolated instrument audio track from the multi-track file, and the transforming resulting from a user selecting a gain ratio of volumes between the isolated instrument audio track and the residual component track;

an emulation recording module embodied in a non-transitory computer readable storage medium, wherein the emulation recording module is operable to record an emulated keyboard video track;

an integration engine embodied in a non-transitory computer readable storage medium, wherein the integration engine is operable to combine the emulated keyboard video track with the audio track.

23. The system of claim 20, wherein the input device comprises a microphone and a camera, and the output device comprises a speaker and a graphical user interface.

24. The system of claim 20, wherein the output module transmits music transcription and tablature data to a graphical user interface, the music transcription and tablature data reflecting the fluctuation in tempos from the multi-track audio file.

25. The system of claim 20, wherein the output module has a recalibration function operable to recalibrate an audio data track output to adjust tempo in the output of the audio track data.

26. The system of claim 20, wherein the output module has a synchronization function operable to synchronize the music transcription and tablature data display on the graphical user interface with the isolated instrument audio track provided to the listener through the speaker.

27. The system of claim 22, wherein the multi-track audio file further comprises an audible dynamic point reference track, and the transformation module is operable to transform the multi-track audio file into a gain ratio of (i) the isolated instrument audio track, (ii) the residual component, and (iii) the audible dynamic point reference track.

28. The system of claim 20 further comprising a data exchange module embodied in a non-transitory computer readable medium, wherein the data exchange module is operable to exchange data with external computer readable media.

29. The system of claim 20, wherein the system is operable using a hand-held device.

30. The system of claim 20, wherein the system is configured to display a set of emulated keyboard performances for the user to compare them for purposes of analyzing the performances.

31. The system of claim 20, wherein the system includes a custom tempo map to reflect the variable and unpredictable tempo changes of an original, multi-track master recording.

32. The system of claim 20, wherein the system consists of a handheld apparatus.

33. The system of claim 20, wherein the system has an external choreographer control configured to allow a choreographer to (i) select a section of the performer'"'"'s keyboard video track to point cycle or (ii) select a tempo for a musician among a group of musicians to follow in the process of learning a keyboard piece.

34. The system of claim 28, wherein the system is configured to include a voting or rating function for the user in the analyzing of the keyboard performance and transmitting the voting or rating using the data exchange module to the external computer readable media.

35. The system of claim 20, wherein the system consists of a game console apparatus.