System and method for capture and rendering of performance on synthetic musical instrument
First Claim
1. A method comprising:
- using a portable computing device as a musical instrument, the portable computing device having a multi-sensor user-machine interface, wherein the musical instrument is a synthetic wind instrument and the multi-sensor user-machine interface includes a microphone and a multi-touch sensitive display;
capturing user gestures from data sampled from plural of the multiple sensors, the user gestures indicative of user manipulation of controls of the musical instrument;
encoding a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors; and
audibly rendering the performance on the portable computing device using the encoded gesture stream as an input to a digital synthesis of the musical instrument executing on the portable computing device.
4 Assignments
0 Petitions
Accused Products
Abstract
Techniques have been developed for capturing and rendering musical performances on handheld or other portable devices using signal processing techniques suitable given the somewhat limited capabilities of such devices and in ways that facilitate efficient encoding and communication of such captured performances via wireless networks. The developed techniques facilitate the capture, encoding and use of gesture streams for rendering of a musical performance. In some embodiments, a gesture stream encoding facilitates audible rendering of the musical performance locally on the portable device on which the musical performance is captured, typically in real time. In some embodiments, a gesture stream efficiently codes the musical performance for transmission from the portable device on which the musical performance is captured to (or toward) a remote device on which the musical performance is (or can be) rendered. Indeed, is some embodiments, a gesture stream so captured and encoded may be rendered both locally and on remote devices using substantially identical or equivalent instances of a digital synthesis of the musical instrument executing on the local and remote devices.
-
Citations
46 Claims
-
1. A method comprising:
-
using a portable computing device as a musical instrument, the portable computing device having a multi-sensor user-machine interface, wherein the musical instrument is a synthetic wind instrument and the multi-sensor user-machine interface includes a microphone and a multi-touch sensitive display; capturing user gestures from data sampled from plural of the multiple sensors, the user gestures indicative of user manipulation of controls of the musical instrument; encoding a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors; and audibly rendering the performance on the portable computing device using the encoded gesture stream as an input to a digital synthesis of the musical instrument executing on the portable computing device. - View Dependent Claims (2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 39)
-
-
3. A method comprising:
-
using a portable computing device as a musical instrument, the portable computing device having a multi-sensor user-machine interface; capturing user gestures from data sampled from plural of the multiple sensors, the user gestures indicative of user manipulation of controls of the musical instrument; encoding a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors; and audibly rendering the performance on the portable computing device using the encoded gesture stream as an input to a digital synthesis of the musical instrument executing on the portable computing device, wherein the encoded gesture stream effectively compresses the sampled data by substantially eliminating duplicative states maintained across multiple samples of user manipulation state and instead coding performance time elapsed between events of the parameterized subset. - View Dependent Claims (4, 40, 41, 42, 43, 44)
-
-
18. A method comprising:
-
using a portable computing device as a musical instrument, the portable computing device having a multi-sensor user-machine interface and a communications interface; capturing user gestures from data sampled from plural of the multiple sensors, the user gestures indicative of user manipulation of controls of the musical instrument; encoding a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors; audibly rendering the performance on the portable computing device using the encoded gesture stream as an input to a digital synthesis of the musical instrument executing on the portable computing device; geocoding and transmitting the encoded gesture stream via the communications interface for rendering of the performance on a remote device; and displaying a geographic origin for, and in correspondence with audible rendering of, a third performance encoded as a third gesture stream received via the communications interface directly or indirectly from a third remote device. - View Dependent Claims (45, 46)
-
-
21. A method of using a portable computing device as a musical instrument, the portable computing device having a multi-sensor user-machine interface, the method comprising:
-
capturing user gestures from data sampled from the sensors, the user gestures indicative of user manipulation of controls of the musical instrument; encoding a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors; and transmitting the encoded gesture stream via a communications interface for rendering of the performance on a remote device using the encoded gesture stream as an input to a digital synthesis of the musical instrument hosted thereon, wherein the encoded gesture stream effectively compresses the sampled data by substantially eliminating duplicative states maintained across multiple samples of user manipulation state and instead coding performance time elapsed between events of the parameterized subset. - View Dependent Claims (22, 29, 30, 31)
-
-
23. A method of using a portable computing device as a musical instrument, the portable computing device having a multi-sensor user-machine interface, the method comprising:
-
capturing user gestures from data sampled from the sensors, the user gestures indicative of user manipulation of controls of the musical instrument, wherein the musical instrument is a synthetic wind instrument and wherein the multi-sensor user-machine interface includes a microphone and a multi-touch sensitive display encoding a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors; and transmitting the encoded gesture stream via a communications interface for rendering of the performance on a remote device using the encoded gesture stream as an input to a digital synthesis of the musical instrument hosted thereon. - View Dependent Claims (24, 25, 26, 27, 28)
-
-
32. An apparatus comprising:
-
a portable computing device having a multi-sensor user-machine interface; and machine readable code executable on the portable computing device to implement the synthetic wind instrument, the machine readable code including instructions executable to capture user gestures from data sampled from plural of the multiple sensors including a microphone and a multi-touch sensitive display, wherein the user gestures are indicative of user manipulation of controls of the wind instrument, and further executable to encode a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors, the machine readable code further executable to audibly render the performance on the portable computing device using the encoded gesture stream as an input to a digital synthesis of the musical instrument. - View Dependent Claims (33, 37, 38)
-
-
34. A computer program product encoded in non-transitory media and including instructions executable to implement a synthetic wind instrument on a portable computing device having a multi-sensor user-machine interface, the computer program product encoding and comprising:
-
instructions executable to capture user gestures from data sampled from plural of the multiple sensors including a microphone and a multi-touch sensitive display, wherein the user gestures are indicative of user manipulation of controls of the wind instrument, and further executable to encode a gesture stream for a performance of the user by parameterizing at least a subset of events captured from the plural sensors, further instructions executable to audibly render the performance on the portable computing device using the encoded gesture stream as an input to a digital synthesis of the musical instrument. - View Dependent Claims (35, 36)
-
Specification