Gestural generation, sequencing and recording of music on mobile devices

• US 8,111,241 B2
• Filed: 07/23/2008
• Issued: 02/07/2012
• Est. Priority Date: 07/24/2007
• Status: Expired due to Fees

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Weinberg, Gil, Loesch, Benedikt, Beck, Andrew
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US12/178,499
• Publication Number:
  US 20090027338A1
• Time in Patent Office: 1,294 Days
• Field of Search: 345156-179, 345/1.1, 178 1901- 1905, 178/18.05, 463/37, 463/30, 463/39, 463/16, 715/702
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G10H 1/0083 using wireless transmission, e.g. radio, light, infrared
• CPC Class Codes: G10H 2210/191 Tremolo, tremulando, trill or mordent effects, i.e. repeatedly alternating stepwise in pitch between two note pitches or chords, without any portamento between the two notes other common forms of tremolo, e.g. same note repetition, bisbigliando, amplitude tremolo, tremulants, percussion roll G10H2210/161 or G10H2210/205
• CPC Class Codes: G10H 2210/381 Manual tempo setting or adjustment tempo setting by interpretation of conducting movements G10H2220/206
• CPC Class Codes: G10H 2220/096 using a touch screen touch screen note input, e.g. using a displayed keyboard G10H2220/241; personal digital assistant [PDA] G10H2230/015
• CPC Class Codes: G10H 2220/261 Numeric keypad used for musical purposes, e.g. musical input via a telephone or calculator-like keyboard
• CPC Class Codes: G10H 2220/395 Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing. conductor baton movement sensing G10H2220/206, angle sensing without reference to gravity G10H2220/391; player body joint angle sensing G10H2220/321
• CPC Class Codes: G10H 2240/175 for jam sessions or musical collaboration through a network, e.g. for composition, ensemble playing or repeating; Compensation of network or internet delays therefor
• CPC Class Codes: G10H 2240/251 Mobile telephone transmission, i.e. transmitting, accessing or controlling music data wirelessly via a wireless or mobile telephone receiver, analog or digital, e.g. DECT GSM, UMTS smartphone, PDA or palmtop used for musical purposes G10H2230/015; mobile ringtone G10H2230/021
• CPC Class Codes: G10H 2240/321 Bluetooth

Abstract

System and methods for an application that allows users to interactively create, transform and play music using cell phones, iPhones™ and other enabled mobile communication devices communicating with a remote host are disclosed. Using an enabled mobile communication device, users are able to strike the mobile device like a drum to create and record rhythms, enter melodies using the keypads, add voice recordings, and manipulate musical tracks by tilting the mobile device continuously in three dimensions. The musical input is sequenced in multiple tracks and the transformative manipulations are applied in real time, allowing users to create their songs in an expressive motion-based manner.

Claims

1. A system for creating, transforming and playing music, comprising:

a personal mobile communication device configured to detect a user gesture and generate music composition data; and

a remote host in communication with the personal mobile communication device, the remote host configured to receive music composition data from the personal mobile communication device and generate music based on the music composition data, wherein the remote host further comprises;

a second memory for storing a second set of executable instructions; and

a second processor configured by the second set of executable instructions to receive the music composition data from the personal mobile communication device, wherein the second set of executable instructions further comprises instructions for presenting a plurality of user interface screens, wherein each of the plurality of screens is assigned one track therein to associate with one sound resource of a plurality of sound resources.

2. A system for creating, transforming and playing music, comprising:

a personal mobile communication device configured to detect a user gesture and generate music composition data; and

a remote host in WIRELESS communication with the personal mobile communication device, the remote host configured to receive music composition data from the personal mobile communication device and generate music based on the music composition data,

wherein the personal mobile communication device further comprises;

a sensor for detecting the user gesture;

a first memory for storing a first set of executable instructions; and

a first processor configured by the first set of executable instructions to detect the user gesture sensed by the sensor, and further configured to generate music composition data based on the detected user gesture, the music composition data comprising compositional, transformational, or control data, and further configured to communicate the music composition data to the remote host,

wherein the first set of executable instructions further comprises instructions for a user interface configured to;

indicate a default percussion track for recording;

receive from the user a plurality of user gestures to establish an initial tempo; and

communicate the plurality of user gestures to the remote host;

wherein a second set of executable instructions comprises instructions for;

responsive to receiving a plurality of taps, recording the plurality of taps sequence to the indicated default percussion track; and

establishing the initial tempo from the plurality of taps; and

generating an audible metronome at the initial tempo.

3. The system of claim 2, wherein the user gesture includes a key press, a button press, a cursor movement, or a touch screen touch.

4. The system of claim 2, wherein the sensor for detecting the user gesture further comprises a microphone.

5. The system of claim 2, wherein the sensor for detecting the user gesture further comprises an accelerometer.

6. The system of claim 2, wherein the first processor is further configured by the first set of executable instructions to communicate the music composition data to the remote host.

7. The system of claim 6, wherein the first processor is further configured by the first set of executable instructions to:

receive a user gesture notification; and

responsive to receiving the user gesture notification, generate a message for communicating music composition data to the host comprising a track identifier and a message type identifier.

8. The system of claim 7, wherein the message for communicating music composition data to the host further comprises a parameter value.

9. The system of claim 6, wherein the first set of executable instructions further comprises instructions for a user interface configured to:

indicate a default percussion track for recording;

receive from the user a plurality of user gestures to establish an initial tempo; and

communicate the plurality of user gestures to the remote host.

10. A system for creating, transforming and playing music, comprising:

a personal mobile communication device configured to detect a user gesture and generate music composition data; and

a remote host in WIRELESS communication with the personal mobile communication device, the remote host configured to receive music composition data from the personal mobile communication device and generate music based on the music composition data,

wherein the personal mobile communication device further comprises;

a sensor for detecting the user gesture;

a first memory for storing a first set of executable instructions; and

a first processor configured by the first set of executable instructions to detect the user gesture sensed by the sensor, and further configured to generate music composition data based on the detected user gesture, the music composition data comprising compositional, transformational, or control data, and further configured to communicate the music composition data to the remote host,

wherein the first set of executable instructions further comprises instructions for a user interface configured to;

indicate a default percussion track for recording;

receive from the user a plurality of user gestures to establish an initial tempo; and

communicate the plurality of user gestures to the remote host;

wherein receiving user input from the personal mobile communication device comprises;

receiving a keydown event;

receiving a keyup event;

responsive to receiving the keyup event, calculating a keypress duration based on the keydown event and the keyup event; and

generating a message comprising a track number, a note number, and a message type corresponding to the event.

11. The system of claim 10, wherein the remote host further comprises:

a second memory for storing a second set of executable instructions; and

a second processor configured by the second set of executable instructions to receive the music composition data from the personal mobile communication device.

12. The system of claim 11, wherein the second processor is further configured by the second set of executable instructions to process the music composition data to identify recordable data therein and to record the recordable data to a plurality of tracks, and further configured to synthesize and to play audio from the data recorded to the plurality of tracks.

13. The system of claim 12, wherein the music composition data further comprises recorded audio.

14. The system of claim 13, wherein the recorded audio further comprises:

audio recorded with a microphone connected to the personal mobile communication device;

audio received from another personal mobile communication device;

or audio of phone calls received by the personal mobile communication device.

15. The system of claim 12, wherein the audio is transformed responsive to the music composition data received by the remote host.

16. The system of claim 12, wherein the second set of executable instructions further comprises instructions for receiving music composition data messages communicated by a plurality of personal mobile communication devices.

17. A method for generating audio, the method comprising the steps of:

recording music event data in an array, wherein the event data is comprised of a time and a track number;

responsive to receiving a melodic note event or a hit;

storing pitch and velocity for each event;

quantizing each note or hit; and

storing the quantized note at a corresponding index in the array;

responsive to receiving a control event;

storing control data for each control event;

responsive to receiving a message to increment a current location pointer, incrementing the current location pointer to the next location in the array; and

sending the contents of the array pointed to by the current location pointer to an output stream buffer;

retrieving from the output stream buffer a current buffer content pointed to by a current output stream buffer pointer;

providing the contents of the location pointed to by the current buffer pointer to an audio synthesizer to generate audio; and

updating the current output stream buffer pointer to point to the next output buffer location.