Gestural Generation, Sequencing and Recording of Music on Mobile Devices

• US 20090027338A1
• Filed: 07/23/2008
• Published: 01/29/2009
• Est. Priority Date: 07/24/2007
• Status: Active Grant

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Beck, Andrew, Loesch, Benedikt, Weinberg, Gil
• Granted Patent: US 8,111,241 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G10H 1/0083 using wireless transmission, e.g. radio, light, infrared
• CPC Class Codes: G10H 2210/191 Tremolo, tremulando, trill or mordent effects, i.e. repeatedly alternating stepwise in pitch between two note pitches or chords, without any portamento between the two notes
• CPC Class Codes: G10H 2210/381 Manual tempo setting or adjustment
• CPC Class Codes: G10H 2220/096 using a touch screen
• CPC Class Codes: G10H 2220/261 Numeric keypad used for musical purposes, e.g. musical input via a telephone or calculator-like keyboard
• CPC Class Codes: G10H 2220/395 Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing
• CPC Class Codes: G10H 2240/175 for jam sessions or musical collaboration through a network, e.g. for composition, ensemble playing or repeating; Compensation of network or internet delays therefor
• CPC Class Codes: G10H 2240/251 Mobile telephone transmission, i.e. transmitting, accessing or controlling music data wirelessly via a wireless or mobile telephone receiver, analogue or digital, e.g. DECT, GSM, UMTS
• CPC Class Codes: G10H 2240/321 Bluetooth

Abstract

System and methods for an application that allows users to interactively create, transform and play music using cell phones, iPhones™ and other enabled mobile communication devices communicating with a remote host are disclosed. Using an enabled mobile communication device, users are able to strike the mobile device like a drum to create and record rhythms, enter melodies using the keypads, add voice recordings, and manipulate musical tracks by tilting the mobile device continuously in three dimensions. The musical input is sequenced in multiple tracks and the transformative manipulations are applied in real time, allowing users to create their songs in an expressive motion-based manner.

Claims

1. A system for creating, transforming and playing music, comprising:

a personal mobile communication device configured to detect a user gesture and generate music composition data; and

a remote host in communication with the personal mobile communication device, the remote host configured to receive music composition data from the personal mobile communication device and generate music based on the music composition data.

2. The system of claim 1, wherein the personal mobile communication device communicates with the remote host wirelessly.

3. The system of claim 2, wherein each of the personal mobile communication devices further comprises:

a sensor for detecting user gestures;

a first memory for storing a first set of executable instructions; and

a first processor configured by the first set of executable instructions to detect a user gesture sensed by the sensor, and further configured to generate music composition data based on the detected user gesture, the music composition data comprising compositional, transformational, or control data.

4. The system of claim 2, wherein the user gesture includes a key press, a button press, a cursor movement, or a touch screen touch.

5. The system of claim 3, wherein the sensor for detecting the user gesture further comprises a microphone.

6. The system of claim 3, wherein the sensor for detecting the user gesture further comprises an accelerometer.

7. The system of claim 3, wherein the first processor is further configured by the first set of executable instructions to communicate the music composition data to the remote host.

8. The system of claim 1, wherein the remote host further comprises:

a second memory for storing a second set of executable instructions; and

a second processor configured by the second set of executable instructions to receive the music composition data from the personal mobile communication device.

9. The system of claim 8, wherein the second processor is further configured by the second set of executable instructions to process the music composition data to identify recordable data therein and to record the recordable data to a plurality of tracks, and further configured to synthesize and to play audio from the data recorded to the plurality of tracks.

10. The system of claim 9, wherein the music composition data further comprises recorded audio.

11. The system of claim 10, wherein the recorded audio further comprises:

audio recorded with a microphone connected to the personal mobile communication device;

audio received from another personal mobile communication device;

or

audio of phone calls received by the personal mobile communication device.

12. The system of claim 9, wherein the audio is transformed responsive to the music composition data received by the remote host.

13. The system of claim 7, further comprising:

receiving a user gesture notification; and

responsive to receiving the user gesture notification, generating a message for communicating music composition data to the host comprising;

a track identifier; and

a message type identifier.

14. The system of claim 13, wherein the message for communicating music composition data to the host further comprises a parameter value.

15. The system of claim 7, wherein the first set of executable instructions further comprises instructions for a user interface configured to:

indicate a default percussion track for recording;

receive from the user a plurality of user gestures to establish an initial tempo; and

communicate the plurality of user gestures to the remote host.

16. The system of claim 9, wherein the second set of executable instructions further comprises instructions for receiving music composition data messages communicated by a plurality of personal mobile communication devices.

17. The system of claim 8, wherein the second set of executable instructions further comprises instructions for presenting a plurality of user interface screens, wherein each of the plurality of screens is assigned one track therein to associate with one sound resource of a plurality of sound resources.

18. The system of claim 15, wherein the second set of executable instructions further comprises instructions for:

responsive to receiving the plurality of taps, recording the plurality of taps sequence to the indicated default track; and

establishing an initial tempo from the plurality of taps; and

generating an audible metronome at the initial tempo.

19. The system of claim 15, wherein receiving user input from the communication device further comprises:

receiving a keydown event;

receiving a keyup event;

responsive to receiving the keyup event, calculating a keypress duration based on the keydown event and the keyup event; and

generating a message comprising a track number, a note number, and a message type corresponding to the event.

20. A method for generating audio, the method comprising the steps of:

recording music event data in an array, wherein the event data is comprised of a time and a track number;

responsive to receiving a melodic note event or a hit;

storing pitch and velocity for each event;

quantizing each note or hit; and

storing the quantized note at a corresponding index in the array;

responsive to receiving a control event;

storing control data for each control event;

responsive to receiving a message to increment a current location pointer, incrementing the current location pointer to the next location in the array; and

sending the contents of the array pointed to by the current location pointer to an output stream buffer;

retrieving from the output stream buffer a current buffer content pointed to by a current output stream buffer pointer;

providing the contents of the location pointed to by the current buffer pointer to an audio synthesizer to generate audio; and

updating the current output stream buffer pointer to point to the next output buffer location.