Electronic music controller using inertial navigation-2

• US 9,773,480 B2
• Filed: 04/07/2015
• Issued: 09/26/2017
• Est. Priority Date: 12/14/2011
• Status: Active Grant

• Current Assignee: John W. Rapp
• Original Assignee: John W. Rapp
• Inventors: Rapp, John W.
• Primary Examiner(s): FLETCHER, MARLON T
• Application Number:
  US14/680,591
• Publication Number:
  US 20150287395A1
• Time in Patent Office: 903 Days
• CPC Class Codes: G10D 13/12 Drumsticks; Mallets pedals for beaters G10D13/11
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: G10H 2220/161 with 2D or x/y surface coordinates sensing
• CPC Class Codes: G10H 2220/185 Stick input, e.g. drumsticks with position or contact sensors
• CPC Class Codes: G10H 2220/395 Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing
• CPC Class Codes: G10H 2230/275 Spint drum
• CPC Class Codes: G10H 3/125 Extracting or recognising the pitch or fundamental frequency of the picked up signal
• CPC Class Codes: G10H 3/146 using a membrane, e.g. a drum; Pick-up means for vibrating surfaces, e.g. housing of an instrument
• CPC Class Codes: G10H 7/00 Instruments in which the tones are synthesised from a data store, e.g. computer organs

Abstract

A percussion controller comprises an instrumented striker including devices for obtaining inertial measurements and a wireless transmitter, a sensor-enabled striking surface that receives an impact from the instrumented striker, and a data processing system that receives the inertial measurements and predicts at least one of the force or location of impact of the instrumented striker on the sensor-enabled striking surface before impact actually occurs.

Claims

1. A percussion controller comprising:

an instrumented striker; and

a data processing system, wherein the data processing system;

a) generates a plurality of virtual impact zones, wherein each zone corresponds to a different musical event;

b) receives first signals that convey information pertaining to movement of the instrumented striker;

c) generates a location prediction and a force prediction based on information conveyed by the first signals, wherein;

(i) the location prediction predicts a location of intersection of the instrumented striker and one of the virtual impact zones,

(ii) the force prediction predicts a force with which the instrumented striker would strike the location of intersection if the virtual impact zone were physically manifested;

d) relates the location of intersection to a musical event; and

e) generates a musical event message based on the musical event.

2. The percussion controller of claim 1 and further wherein the location prediction is based, at least in part, on inertial navigation computations.

3. The percussion controller of claim 1 and further comprising a striking surface for striking with the instrumented striker, wherein the striking surface does not include any sensors.

4. The percussion controller of claim 3 and further wherein the data processing system maps at least some of the plurality of virtual impact zones to locations on the striking surface, thereby defining physical impact zones on the striking surface, wherein each physical impact zone corresponds to the musical event associated with virtual impact zone that defined the physical impact zone.

5. The percussion controller of claim 4 wherein the striking surface comprises a resilient surface.

6. The percussion controller of claim 5 and further wherein the striking surface comprises a plurality of lights, wherein the data processing system is operable to selectively illuminate some of the lights to demarcate the physical impact zones.

7. The percussion controller of claim 4 and further comprising an auxiliary instrumented mat that generates second signals, wherein the data processing system uses the second signals to perform at least one of the following tasks:

(i) initialize inertial navigation computations, and (ii) provide on-going corrections to inertial navigation computations.

8. The percussion controller of claim 1 and further comprising a sensor-enabled striking surface including a resilient surface for striking with the instrumented striker and a plurality of sensors disposed beneath the sensor-enabled striking surface, wherein the data processing system:

(f) receives second signals that convey information pertaining to the movement of the instrumented striker toward the sensor-enabled striking surface;

(g) predicts, based on the information conveyed by the second signals, at least one of;

(i) a force of impact of the instrumented striker on the sensor-enabled striking surface, and

(ii) a location at which the instrumented striker will impact the sensor-enabled striking surface;

(h) relates the location of impact to a musical event; and

(i) generates a musical event message based on the musical event.

9. The percussion controller of claim 8 and further comprising an instrumented mat that controls one or more attributes of the sensor-enabled striking surface.

10. The percussion controller of claim 9 wherein striking the instrumented mat at a first location changes the musical event that corresponds to a first location on the sensor-enabled striking surface.

11. The percussion controller of claim 9 wherein striking the instrumented mat at a first location changes an instrument that the sensor-enabled striking surface simulates in conjunction with the data processing system.

12. The percussion controller of claim 10 wherein striking the instrumented mat a second location changes an instrument that the sensor-enabled striking surface simulates in conjunction with the data processing system.

13. The percussion controller of claim 9 wherein the sensor-enabled striking surface simulates a first instrument and the instrument mat simulates a second instrument.

14. The percussion controller of claim 8 and further comprising a foot switch, wherein the foot switch controls one or more attributes of the sensor-enabled striking surface.

15. The percussion controller of claim 1 and further wherein the data processing system alters a number of virtual impact zones in the plurality thereof.

16. The percussion controller of claim 15 and further wherein the data processing system increases the number of virtual impact zones, wherein additional virtual impact zones correspond to additional musical events.

17. The percussion controller of claim 1 and further wherein the data processing system changes the musical events that correspond to particular virtual impact zones.

18. The percussion controller of claim 1 wherein at least one of the virtual impact zones correspond to a cymbal.

19. The percussion controller of claim 1 and further wherein the data processing system:

(f) compares the movement of the instrumented striker, as conveyed by the information in the first signals, to predetermined striker motion patterns that correspond to musical events;

(g) characterizes the movement of the instrumented striker as a non-throwing motion when the striker'"'"'s movement matches one of the predefined striker motion patterns; and

(h) generates a second signal that conveys second information about the musical event corresponding to the matched predefined striker motion pattern.

20. The percussion controller of claim 1 and further wherein the data processing system stores information related to acceleration and position of the instrumented striker, wherein the information is indicative of a user'"'"'s striker-throwing technique.

21. The percussion controller of claim 20 wherein the data processing system:

generates a visual representation of the user'"'"'s striker-throwing technique from the information indicative thereof; and

displays the visual representation for viewing.

22. The percussion controller of claim 20 and further wherein the data processing system assesses the user'"'"'s striker-throwing technique.

23. The percussion controller of claim 22 wherein the data processing system assesses the user'"'"'s striker-throwing technique by comparing the information indicative of the user'"'"'s striker-throwing technique to reference information pertaining to throwing technique.

24. The percussion controller of claim 23 wherein the reference information comprises a prerecorded reference performance.

25. A method comprising:

predicting a location of intersection of an instrumented striker with a virtual impact zone based on signals received from the instrumented striker;

predicting, based on the signals received from the instrumented striker, a force with which the instrumented striker would strike the virtual impact zone if the virtual impact zone were physically manifested;

relating the location of intersection with a musical event;

generating a first signal that conveys first information about the musical event; and

transmitting the first signal to a device that generates a second signal that can be converted to sound that is related to the musical event.

26. The method of claim 25 and further comprising mapping the virtual impact zone onto a striking surface.

27. The method of claim 25 and further comprising:

mapping predefined motion patterns to musical events;

comparing motion of the instrumented striker to the predefined motion patterns;

when the motion matches one of the predefined motion patterns, generating a third signal that conveys second information about the corresponding musical event; and

transmitting the third signal to the device for generating signals that can be converted to a sound that is related to the corresponding musical event.

28. The method of claim 25 and further comprising storing information related to acceleration and position of the instrumented striker, wherein the information is indicative of a user'"'"'s striker-throwing technique.

29. The method of claim 28 and further comprising assessing the user'"'"'s striker-throwing technique.

30. The method of claim 29 wherein assessing the user'"'"'s striker-throwing technique further comprises comparing the information indicative of the user'"'"'s striker-throwing technique to reference information pertaining to throwing technique.

31. The method of claim 30 wherein the reference information comprises a prerecorded reference performance.

32. The method of claim 29 wherein assessing the user'"'"'s throwing technique further comprises:

generating a visual representation of the user'"'"'s technique from the information indicative thereof; and

displaying the visual representation for viewing.

33. The method of claim 25 and further comprising:

generating, at the third device, the signals that can be converted to the sound that is related to the corresponding musical event; and

generating the sound.

34. A method comprising:

monitoring motion of a striker;

predicting, using information obtained from the monitoring, at least one of a location or a force as follows;

(a) a location at which the striker will impact a striking surface,

(b) a location at which the striker will intersect a virtual impact zone,

(c) a force with which the striker will impact the striking surface at the location, or

(d) a force with which the striker would impact the virtual impact zone at the location of intersection, if the virtual impact zone were physically manifested;

generating a visual representation of the monitored motion;

displaying the visual representation for viewing; and

generating a musical event message from the at least one predicted location or force.

35. The method of claim 34 and further comprising assessing a throwing technique of a user that is using the striker.

36. The method of claim 35 and wherein the throwing technique being assessed is selected from the group consisting of a wrist pivot, whether grip is slipping, whether the striker is being rolled, whether a pre-impact release of throwing force occurs, single stroke throw about wrist axis and bounce, and double stroke throw and bounce.

37. The method of claim 35 and further wherein assessing the throwing technique comprises comparing the throwing technique to reference information pertaining to striker throwing technique.

38. The method of claim 35 wherein predicting at least one of a location or a force is based, at least in part, on inertial navigation computations and further wherein assessing the throwing technique is based on at least some of the intertial navigation computations.

39. A percussion controller comprising:

an instrumented striker;

a resilient striking surface for striking with the instrumented striker, wherein the striking surface does not include any sensors; and

a data processing system, wherein the data processing system;

(a) receives first signals that convey information pertaining to kinetics of the instrumented striker; and

(b) processes the first signals using inertial navigation techniques to predict at least one of;

(i) a future location of the instrumented striker;

(ii) a force with which the instrumented striker will impact a surface at the future location;

(iii) a force with which the instrumented striker would impact a virtual impact zone at the future location, if the virtual impact zone were physically manifested.