Floor controller for real-time control of music signal processing, mixing, video and lighting

• US 20020005111A1
• Filed: 03/19/2001
• Published: 01/17/2002
• Est. Priority Date: 05/15/1998
• Status: Active Grant

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: Lester Frank Ludwig
• Inventors: Ludwig, Lester Frank
• Granted Patent: US 6,689,947 B2
• Time in Patent Office: Days
• US Class Current: 84/645
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 1/0058 Transmission between separate instruments or between individual components of a musical system G10H1/0083 takes precedence
• CPC Class Codes: G10H 1/0091 Means for obtaining special acoustic effects combined with modulation G10H1/043
• CPC Class Codes: G10H 2220/521 Hall effect transducers or similar magnetic field sensing semiconductor devices, e.g. for string vibration sensing or key movement sensing
• CPC Class Codes: G10H 2230/095 Spint zither, i.e. mimicking any neckless stringed instrument in which the strings do not extend beyond the sounding board
• CPC Class Codes: G10H 2230/101 Spint koto, i.e. mimicking any traditional asian-style plucked zither with movable bridges
• CPC Class Codes: G10H 2230/115 Spint sitar, i.e. mimicking any long-necked plucked string instrument with a large number of additional non-playable sympathetic resonating strings or an additional gourd-like resonating chamber
• CPC Class Codes: G10H 2230/145 Spint guitar keyboard, i.e. mimicking a combination of a guitar-like instrument, with or without strings, and a piano-like keyboard, e.g. with white and black keys arranged like on a piano
• CPC Class Codes: G10H 2230/271 Spint gong, i.e. mimicking circular flat, nippled or bowl-shaped metallic percussion instruments G10H2230/321 takes precedence
• CPC Class Codes: G10H 3/26 using electric feedback

Abstract

A system and method for real-time controlling of signal processors, synthesizers, musical instruments, MIDI processors, lighting, video, and special effects in performance, recording, and composition environments. The invention provides for combinations of various physical controller elements such as foot switches, rocking foot pedals, null/contact touch-pads, pressure sensor arrays, etc., and well as alpha-numeric displays, graphical displays, and internal control signal processing. Each physical controller element may be assigned specific control message functions, values, names, temporal event sequences, and invocation rules dictated by assignable state-machines. Separate visual display of assigned name, status, and/or value may be provided for each physical controller elements. Rocking foot pedals may be multidimensional through the inclusion of twist plates, pressure sensors, touchpads, levers, wheels, etc. Touch-pads and pressure sensor arrays may be transparent and feature underlying graphical, alphanumeric, or video display. Assignments (of control signals and behaviors to physical controller elements) may be selected, and organized in to pages and sub-pages. Activation of specific pages and subpages may be done with assignable physical controller elements. Physical controller elements and assignments may be organized in hierarchical relationships and in relationships rich in potential geometric metaphors useful for human operation. The invention also provides for MIDI and non-MIDI control signals.

Claims

1. A system for issuing control signals comprising at least one physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touch-pads

pressure sensor arrays,

and further comprising at least one alphanumeric display, said system generating said control signals in response to human operation of said physical controller element according to a selectable assignment, wherein the said alphanumeric display indicates information affiliated with the at least one physical controller element.

2. The system of claim 1 wherein the control signals include signals of MIDI format.

3. The system of claim 1 wherein the alphanumeric display provides at least one of the following information items affiliated with the at least one physical controller element.

currently assigned function

status

current value(s) transmitted or last transmitted

short-hand names affiliated with the currently assigned function

relationships with other controllers.

4. The system of claim 1 wherein at least one physical controller element is either:

a transparent touchpad;

or

a transparent pressure sensor array said physical controller element further comprising an underlying visual display.

5. The system of claim 4 wherein said underlying visual display provides at least one of the following forms of information:

video

graphics

alpha-numeric.

6. The system of claim 1 wherein at least one physical controller element is a transparent touchpad or pressure sensor array further comprising an impact sensor.

7. The system of claim 1 wherein at least one physical controller element is a multi-dimensional rocker pedal.

83. The system of claim 1 wherein said rocking foot pedals include positioning motors.

85. The system of claim 1, said system further comprising a display capable of rendering visual graphical information, wherein information regarding at least one of:

selectable assignments

short-hand names

control value transmitted

geometric metaphors affiliated with selectable assignments

hierarchical organization of selectable assignments

status associated with at least one physical controller element is displayed in graphical format on said display.

8. A system for issuing control signals comprising at least one multidimensional rocker pedal comprising a rocker plate, said multidimensional rocker pedal further comprising at least one physical control component from the collection:

a lever generating continuous-range parameter control signals

a modulation wheel

a null/contact touch-pad

a pressure sensor

wherein said at least one physical control component is mounted on the rocker plate of the rocker pedal.

9. The system of claim 8 wherein said physical control component is a lever or modulation wheel which has a mechanical spring-return.

10. The system of claim 8 wherein said multi-dimensional rocker pedal includes at least two a levers mounted in an opposing configuration.

11. The system of claim 8 wherein the control signals include signals of MIDI format.

12. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment, wherein the said selectable assignments are organized in a hierarchical collection.

13. The system of claim 12 wherein the control signals include signals of MIDI format.

14. The system of claim 12 wherein said hierarchical collection is in the form of a plurality of selectable pages, each said pages comprising a plurality of subpages.

15. The system of claim 14, the system further comprising at least one display with alphanumeric capabilities, wherein information specifying the currently active page and subpage are displayed in said display.

16. The system of claim 14, wherein each said subpage comprises selectable assignments affiliated with only a subset of the plurality of physical controller elements.

17. The system of claim 16 wherein the said subsets are overlapping.

18. The system of claim 16 wherein the said subsets are non-overlapping.

19. The system of claim 16 wherein plurality of physical controller elements are in a geometric arrangement and the said subsets of said subpages are metaphorically affiliated with said geometric arrangement.

20. The system of claim 12 wherein the said selectable assignments are hierarchically organized in a collection of pages, and the selection of active page is made through the use of at least one physical controller element from the said plurality of physical controller elements, the choice of which is itself assignable.

21. The system of claim 20 wherein the control signals include signals of MIDI format.

22. The system of claim 20 wherein said pages further are comprised of a plurality of subpages, and the selection of active subpage is made through the use of at least one physical controller element from the said plurality of physical controller elements, the choice of which is itself assignable.

23. The system of claim 12 wherein the said selectable assignments are organized in a collection of pages, and the change in selection of an active page causes the issuance of control signals.

24. The system of claim 23 wherein the control signals include signals of MIDI format.

25. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements, said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

wherein said plurality of physical controller elements are arranged in a geometric configuration of any of;

producing a physical hierarchical arrangement

allowing the association of geometric metaphors

for the purpose of organizing said selectable assignments with respect to the geometric arrangement of affiliated physical controller elements.

26. The system of claim 25 wherein said physical control elements are arranged in a tiered staircase array.

27. The system of claim 25 wherein said hierarchical arrangement is in correspondence with a hierarchical arrangement of selectable assignment affiliations.

28. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

said control signals comprising at least one control signal event, wherein said selectable assignment includes the issuance of control signals comprising timed sequences of control signal events.

29. The system of claim 28 wherein said timed sequences of control signal events comprises a timed sequence of MIDI events.

30. The system of claim 28 wherein said timed sequences of control signal events include at least one pause of arbitrary duration.

31. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment, said control signals comprising at least one control signal event,

wherein at least two selectable assignments from the said plurality of selectable assignments are invoked according to a state machine driven by operation of the physical controller elements affiliated with the at least two said selectable assignments.

32. The system of claim 31 wherein control signals include signals of MIDI format.

33. The system of claim 31 wherein said state machine is specified by selectable assignment.

34. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

said control signals used to control the pitch-shift of a selected subset of vibrating element audio signals produced from a plurality of vibrating element sense transducers on an external instrument.

35. The system of claim 34 wherein said external instrument is a slide guitar or pedal steel guitar.

36. The system of claim 35 wherein at least one of the physical controller elements is mounted on said external instrument.

37. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment, said control signals comprising at least one control value and further organized into a plurality of control message types.

wherein said control signals may be selectively processed in at least one of the following manners;

generation of complementary mirror-image control signal values

generation of transient envelope control signals

generation of low-frequency oscillator control signals

generation of the multiplicative product of two control signal values

generation of the additive sum of two control signal values

generation of control signals with the same control values but different control message type.

38. The system of claim 37 wherein the control signals include signals of MIDI format.

39. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment, wherein visual indication is provided of the last operated physical controller element.

40. The system of claim 39 wherein visual indication is provided for at least one physical controller element through the use of a bicolor LED.

41. A system for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further comprising a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said system generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

wherein said control signals are used to control the assignment of further control signals produced by any of the following;

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

strum pads.

42. A method for issuing control signals comprising the use of at least one physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touch-pads

pressure sensor arrays,

and further comprising the use of at least one alphanumeric display, said method generating said control signals in response to human operation of said physical controller element according to a selectable assignment,

wherein the said alphanumeric display is used to indicate information affiliated with the at least one physical controller element.

43. The method of claim 42 wherein the control signals employ messages of MIDI format.

44. The method of claim 42 wherein the alphanumeric display is used to provide at least one of the following information items affiliated with the at least one physical controller element.

currently assigned function

status

current value(s) transmitted or last transmitted

short-hand names affiliated with the currently assigned function

relationships with other controllers.

45. The method of claim 42 wherein at least one physical controller element is either:

a transparent touchpad;

or

a transparent pressure sensor array

said physical controller element further utilizing an underlying visual display which is made visible by said transparent nature of said physical controller element.

46. The method of claim 45 wherein said underlying visual display is used to provide at least one of the following forms of information:

video

graphics

alpha-numeric.

47. The method of claim 42 wherein at least one physical controller element utilizes a transparent touchpad or pressure sensor array which further utilizes an impact sensor for measuring percussive impact information.

48. The method of claim 42 wherein at least one physical controller element is a multi-dimensional rocker pedal used to capture additional control parameters from its operation.

84. The method of claim 42 wherein said rocking foot pedals may also be positioned by motors.

86. The method of claim 42, said method further utilizing a display capable of rendering visual graphical information, wherein information regarding at least one of:

selectable assignments

short-hand names

control value transmitted

geometric metaphors affiliated with selectable assignments

hierarchical organization of selectable assignments status associated with at least one physical controller element is displayed in graphical format on said display.

49. A method for issuing control signals comprising the use of at least one multidimensional rocker pedal comprising a rocker plate, said multidimensional rocker pedal further comprising at least one physical control component from the collection:

a lever generating continuous-range parameter control signals

a modulation wheel

a null/contact touch-pad

a pressure sensor

wherein said at least one physical control component is mounted on the rocker plate of the rocker pedal so as to facilitate the capture of additional control parameters.

50. The method of claim 49 wherein said physical control component uses a lever or modulation wheel which has a mechanical spring-return.

51. The method of claim 49 wherein said multi-dimensional rocker pedal uses at least two a levers mounted in an opposing configuration.

52. The method of claim 49 wherein the provided control signals employ messages of MIDI format.

53. A method for issuing control signals utilizing a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human a operation of the physical controller elements according to a selectable assignment,

wherein the said selectable assignments are organized in a hierarchical collection.

54. The method of claim 53 wherein the control signals employ messages of MIDI format.

55. The method of claim 53 wherein said hierarchical collection utilizes a plurality of selectable pages, each said pages utilizing a plurality of subpages.

56. The method of claim 55, the method further utilizing at least one display with alphanumeric capabilities, wherein information specifying the currently active page and subpage are displayed in said display.

57. The method of claim 55, wherein each said subpage utilizes selectable assignments affiliated with only a subset of the plurality of physical controller elements.

58. The method of claim 57 wherein the said subsets are overlapping.

59. The method of claim 57 wherein the said subsets are non-overlapping.

60. The method of claim 57 wherein plurality of physical controller elements are in a geometric arrangement and the said subsets of said subpages are metaphorically affiliated with said geometric arrangement.

61. The method of claim 53 wherein the said selectable assignments are hierarchically organized in a collection of pages, and the selection of active page is made through the use of at least one physical controller element from the said plurality of physical controller elements, the choice of which is itself assignable.

62. The method of claim 61 wherein the control signals employ messages of MIDI format.

63. The method of claim 61 wherein said pages further utilize a plurality of subpages, and the selection of active subpage is made through the use of at least one physical controller element from the said plurality of physical controller elements, the choice of which is itself assignable.

64. The method of claim 53 wherein the said selectable assignments are hierarchically organized in a collection of pages, and the change in selection of an active page causes the issuance of control signals.

65. The method of claim 64 wherein the control signals employ messages of MIDI format.

66. A method for issuing control signals comprising a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

wherein said plurality of physical controller elements are arranged in a geometric configuration of any of;

producing a physical hierarchical arrangement

allowing the association of geometric metaphors for the purpose of organizing said selectable assignments with respect to the geometric arrangement of affiliated physical controller elements.

67. The method of claim 66 wherein said physical control elements are arranged in a tiered staircase array.

68. The method of claim 66 wherein said hierarchical arrangement is in correspondence with a hierarchical arrangement of selectable assignment affiliations.

69. A method for issuing control signals utilizing a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

said control signals comprising at least one control signal event, wherein said selectable assignment includes the issuance of control signals comprising timed sequences of control signal events.

70. The method of claim 69 wherein said timed sequences of control signal events employs a timed sequence of MIDI events.

71. The method of claim 69 wherein said timed sequences of control signal events employs at least one pause of arbitrary duration.

72. A method for issuing control signals utilizing a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

said control signals comprising at least one control signal event, wherein at least two selectable assignments from the said plurality of selectable assignments are invoked according to a state machine driven by operation of the physical controller elements affiliated with the at least two said selectable assignments.

73. The method of claim 72 wherein control signals include signals of MIDI format.

74. The method of claim 72 wherein said state machine is specified by selectable assignment.

75. A method for issuing control signals utilizing a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

said control signals used to control the pitch-shift of a selected subset of vibrating element audio signals produced from a plurality of vibrating element sense transducers on an external instrument.

76. The method of claim 75 wherein said external instrument is a slide guitar or pedal steel guitar.

77. The method of claim 76 wherein at least one of the physical controller elements is mounted on said external instrument.

78. A method for issuing control signals utilizing a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

said control signals comprising at least one control value and further organized into a plurality of control message types.

wherein said control signals may be selectively processed in at least one of the following manners;

generation of complementary mirror-image control signal values

generation of transient envelope control signals

generation of low-frequency oscillator control signals

generation of the multiplicative product of two control signal values

generation of the additive sum of two control signal values

generation of control signals with the same control values but different control message type.

79. The method of claim 78 wherein the control signals employ messages of MIDI format.

80. A method for issuing control signals utilizing a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

wherein visual indication is provided of the last operated physical controller element.

81. A method of claim 80 wherein visual indication is provided for at least one physical controller element through the use of a bicolor LED.

82. A method for issuing control signals utilizing a plurality of physical controller elements from the collection of:

foot switches

rocking foot pedals

null/contact touchpads

pressure sensor arrays,

and further utilizing a plurality of selectable assignments, each selectable assignment affiliating the generation of control signals with at least one physical controller elements,

said method generating said control signals in response to human operation of the physical controller elements according to a selectable assignment,

wherein said control signals are used to control the assignment of further control signals produced by any of the following;

foot switches;

rocking foot pedals;

null/contact touchpads;

pressure sensor arrays;

or

strum pads.