Interactive music training and entertainment system and multimedia role playing game platform

• US 7,772,480 B2
• Filed: 06/25/2008
• Issued: 08/10/2010
• Est. Priority Date: 08/10/2007
• Status: Expired due to Fees

• Current Assignee: SonicJam, Inc. (JAPAN Creative Platform Group Co., Ltd.)
• Original Assignee: SonicJam, Inc. (JAPAN Creative Platform Group Co., Ltd.)
• Inventors: Brennan, Edward W.
• Primary Examiner(s): Donels; Jeffrey
• Assistant Examiner(s): Millikin; Andrew R
• Application Number:
  US12/146,378
• Publication Number:
  US 20090038468A1
• Time in Patent Office: 776 Days
• Field of Search: 434/307.R, 434/307.A, 844/70.R, 846/09, 846/10, 846/15, 846/16, 846/18, 846/25, 846/26, 846/34, 463/7, 463/23, 463 35- 38, 463/43, 700/94
• US Class Current: 84/615
• CPC Class Codes: G09B 15/00 Teaching music metronomes G04F5/02
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/046 for differentiation between music and non-music signals, based on the identification of musical parameters, e.g. based on tempo detection
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/071 for rhythm pattern analysis or rhythm style recognition rhythm pattern per se G10H2210/341
• CPC Class Codes: G10H 2210/076 for extraction of timing, tempo; Beat detection tempo display G10H2220/081; tempo control G10H2210/375
• CPC Class Codes: G10H 2210/081 for automatic key or tonality recognition, e.g. using musical rules or a knowledge base
• CPC Class Codes: G10H 2210/091 for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
• CPC Class Codes: G10H 2220/011 Lyrics displays, e.g. for karaoke applications

Abstract

A music training and entertainment system allows a user to input a selection of source music in a native format, and isolates a desired vocal or instrument component within the source music. The user selects the component of the source music against which to be compared. The system suppresses the non-selected components of the source content to isolate the user-selected component. The system receives user input corresponding to the isolated component, and matches the user'"'"'s performance to the original performance on the basis of pitch, rhythm, and quality comparisons. In a graphical user interface, avatars representing the user and judges are displayed in relation to an audience. Audience reaction is configured to reflect a score obtained by the user. A lyric extraction and generation process displays lyrics to a song in real time to the user as the music track is played.

Claims

1. A method of providing an interactive entertainment system comprising:

receiving a source stream from a source, the source stream comprising a plurality of components;

suppressing a user selected portion of the source stream during playback of the source stream to a user;

receiving a user provided component corresponding to the suppressed component of the source stream simultaneously with the source stream;

determining a first waveform area for the suppressed component and a second waveform area for the user provided component;

comparing the pitch, rhythm, and sonic quality of the user provided component with the pitch, rhythm, and sonic quality of the suppressed component of the source stream to derive an aggregate score corresponding to the similarity of the user provided component with the suppressed component, wherein the quality is compared by comparing the first waveform area to the second waveform area;

displaying an avatar representing the user on a stage in a concert setting including an audience and a panel of one or more judges; and

displaying, through the panel of the one or more judges, the aggregate score corresponding to the similarity of the user provided component with the suppressed component.

2. The method of claim 1 further comprising:

correlating the aggregate score to a qualitative text message, wherein the qualitative text message is provided to the user through the panel of one or more judges; and

manifesting a correlation between the similarity of the user provided component with the suppressed component as an audience reaction.

3. The system of claim 2 wherein the source stream comprises music, and the source is selected from the group consisting of:

streaming music content, pre-recorded digital media, pre-recorded analog media, and a live performance by one or more performers.

4. The system of claim 3 wherein the user selected component is selected from the group consisting of vocal, guitar, keyboard, drums, and an ensemble of instruments.

5. The system of claim 1 wherein the aggregate score is generated by averaging separate scores created for each of:

a comparison of the pitch of the suppressed component with the pitch of the user provided component, a comparison of the rhythm of the suppressed component with the rhythm of the user provided component, and a comparison of the quality of the suppressed component with the quality of the user provided component.

6. The system of claim 1 further comprising:

displaying a representation of the user performance on one or more virtual screens adjacent the displayed stage; and

panning the displayed representation across a plurality of camera angles relative to the displayed stage.

7. A graphical user interface comprising:

a virtual stage containing an avatar of a user providing a performance of a song, wherein the performance comprises a user provided component of a song that has been suppressed from an original source recording of the song, and wherein the user provided component is selected from the group consisting of vocal, guitar, keyboard, and drums, and further wherein the avatar is displayed as playing an instrument corresponding to the user selected component;

an interactive audience configured to respond to the performance based on the similarity of the user provided component with the original suppressed component;

one or more virtual big screen monitors displaying a depiction of the user performance from the vantage points of a plurality of cameras surrounding the virtual stage; and

a virtual panel of one or more judges, the judges displaying a score corresponding to the similarity of the user provided component with the original suppressed component, wherein the score is derived by comparing the user provided component with the original suppressed component across a plurality of characteristics comprising pitch, rhythm, and sonic quality.

8. The user interface of claim 7 further comprising an input process configured to receive a graphic file representing the user and display the graphic file as an overlay of the avatar.

9. The user interface of claim 7 wherein the sonic quality is determined by comparing a first waveform area for the suppressed component to a second waveform area for the user provided component.