SYSTEM AND METHOD THAT GENERATES OUTPUTS

• US 20090256801A1
• Filed: 06/29/2007
• Published: 10/15/2009
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Superinteractive Pty. Ltd.
• Original Assignee: Commonwealth Scientific & Industrial Research Organisation (Commonwealth Of Australia As Represented By Department Of Industry, Innovation, Climate Change, Science, Research And Tertiary Education)
• Inventors: Helmer, Richard James Neil
• Granted Patent: US 8,830,162 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G10H 1/342 for guitar-like instruments with or without strings and with a neck on which switches or string-fret contacts are used to detect the notes being played electric guitars in which the tones are generated by the vibration of strings G10H3/18
• CPC Class Codes: G10H 1/348 Switches actuated by parts of the body other than fingers
• CPC Class Codes: G10H 2220/005 Non-interactive screen display of musical or status data
• CPC Class Codes: G10H 2220/321 Garment sensors, i.e. musical control means with trigger surfaces or joint angle sensors, worn as a garment by the player, e.g. bracelet, intelligent clothing
• CPC Class Codes: G10H 2230/135 Spint guitar, i.e. guitar-like instruments in which the sound is not generated by vibrating strings, e.g. guitar-shaped game interfaces
• CPC Class Codes: G10H 2230/141 Spint guitar drum, i.e. mimicking a guitar used at least partly as a percussion instrument

Abstract

The present invention relates to a system and method that generates outputs based on the operating position of a sensor which is determined by the biomechanical positions or gestures of individual operators. The system including a garment on which one or more than one sensor is removably attached and the sensors provide a signal based on the biomechanical position, movement, action or gestures of the person wearing the garment, a transmitter receiving signals from the sensors and sends signals to a computer that is calibrated to recognise the signals as representing particular positions that are assigned selected outputs. Suitably the outputs are audio outputs of an instrument, such as a guitar, and the outputs simulate the sound of a guitar that would be played when the biomechanical motion, action, gesture or position of the operator resembles those that would occur when an actual instrument is played.

Claims

1. A system including:

one or more than one sensor and each sensor is operated over two or more than two operating positions; and

an electronic device that produces or generates outputs, wherein the electronic device receives signals from the or each sensor based on the operating positions thereof and the electronic device or the sensors are calibrated so that selected operating positions of the or each sensor and thus the signals received by the electronic device are associated or assigned particular outputs such that, once calibrated, a sequence of outputs can be produced by operating the or each sensor between operating positions.

2. The system according to claim 1, wherein the electronic device or sensors are calibrated based on biomechanical movements of an individual operator.

33. The system according to claim 1, wherein the system includes a transmitter that receives signals from the sensors and wirelessly transmits signal to the electronic device.

34. The system according to any one of claim 1, wherein the system includes one or more textile substrates to which one or each of the sensors are fitted or incorporated and the textile substrates is able to be removably attached to a garment worn by the operator of the system.

35. The system according to claim 33, wherein the system includes a garment to which the sensors are directly or indirectly removably attached, thereby allowing the garment to be refreshed by washing or placed entirely as desired.

36. The system according to claim 35, wherein the transmitter is directly or indirectly removably attached to the garment.

3. A system that is operated by a person to produce desired outputs, the system including:

one or more than one sensor and at least one of the sensors is operated over two or more than two operating positions; and

an electronic device that produces or generates outputs, wherein the electronic device receives signals from the or each sensor based on the operating positions thereof and either one or both of the electronic device and the sensors are calibrated so that selected operating positions of the or each sensor is based on biomechanical movements or positions of the person operating the system, and wherein the signals received by the electronic device are associated or assigned particular outputs such that, once calibrated, a sequence of outputs can be produced by operating the or each sensor between operating positions based on the biomechanical movement of individual operators.

4. The system according to claim 3, wherein the outputs are in the form of either one or a combination of visual outputs and audible outputs.

5. The system according to claim 4, wherein the audible outputs are a pre-recorded sound or a stored sample of sound.

6. The system according to claim 4, wherein the audible outputs are produced or played with accompanying outputs that are unrelated to the operating position of the sensors.

7. The system according to claim 3, wherein the sensors includes an electrical resistor that provides variable electrical resistance depending on the various operating positions of the sensors.

8. The system according to claim 3, wherein the electronic device or sensors are calibrated such that the operating positions of the sensors are defined by signals that vary over a range of values.

9. The system according to claim 3, wherein the operating positions of the sensors are defined by a range of angular orientations of one or more body joint of the operator such as but by no means limited to finger, wrist, elbow, shoulder, neck, back, hip, knee or ankle joints.

10. The system according to claim 8, wherein the values or the range of values of the signals that are associated with selected operating positions are adjustable as desired or is able to be recalibrated to suit different individual operators.

11. The system according to claim 3, wherein the electronic device or sensors stores the configuration in which the system is calibrated for each individual operator and the stored configuration can be recalled or re-activated after use of the system by other operators.

12. The system according to claim 3, wherein the system includes a feedback means that communicates to the operator which operating position the sensors are located.

13. The system according to claim 12, wherein the feedback means includes a means for visually communicating to the operator the operating position of the sensors.

14. The system according to claim 12, wherein the feedback means is in the form of a monitor that displays which operating position at least one of the sensors is located to the operator relative to other operating positions of the respective sensor.

15. The system according to claim 14, wherein the monitor includes a picture or schematic illustration of a musical instrument or a part thereof, and each operating position represents or simulates biomechanical positions of the operator playing the instrument which is shown on the monitor.

16. The system according to claim 12, wherein the feedback means also communicates to the operator information identifying which particular output that has been assigned to each operating position.

17. The system according to claim 3, wherein the delay between the sensor being located in an operative position and the electronic device producing or play the desired output is equal to or less than 50 milliseconds.

18. The system according to claim 17, wherein the delay period is less than or equal to 10 milliseconds.

19. The system according to claim 3, wherein the electronic device has a memory that stores data on any one or a combination of:

the operating positions, including ranges of values of the signals of the sensor that define operating positions;

the outputs, including audible outputs; and

the assignment or allocation of particular outputs to selected operating positions.

20. The system according to claim 3, wherein the electronic device has a data process unit that processes data on any one or a combination of the following:

carrying out algorithms analysing the signal from the sensor;

carrying out algorithms that assess whether the signals equals or falls within the ranges of the values of the signals that defines calibrated operation positions; and

carrying out algorithms that calculate the speed at which the value of a signal of a sensor changes, and thereby determines the volume at which an audible output should be played based on the rate of change of the signal.

21. The system according to any one of claim 3, wherein the system includes two or more sensors, and each sensor having two or more than two operating positions.

22. The system according to claim 21, wherein two of the sensors are calibrated such that at least one of the operating positions of one of the sensors is associated with multiple outputs, or allocate a number of outputs, and the number of outputs equals or is less than the number of operating positions of the other sensor such that the total maximum number of possible outputs is the number of calibrated operating positions of the sensors multiplied together and the output generated is determined by the operating position of both of the sensors.

23. The system according to any one of claim 3, wherein the system includes two sensors for monitoring the angular orientation of the elbows of the operator.

24. The system according to claim 23, wherein the sensors and/or electrical device are calibrated such that the operating positions have an angular orientation ranging from approximately 5 to 45 degrees.

25. The system according to claim 24, wherein the sensor and/or electrical device are calibrated such that the operating positions are defined over angular orientations ranging from 5 to 20 degrees.

26. The system according to claim 23, wherein one of the sensors, namely a fret sensor, monitors the angular orientation of one of the operator'"'"'s elbow and is calibrated into at least six ranges, each representing or simulating biomechanical positions of an operator'"'"'s elbow on the fret board of a guitar.

27. The system according to claim 26, wherein the six operating positions of the fret sensor are associated with a pair of outputs, each pair representing a chord of a guitar that produces the sound of the chord played by an upstrum or a downstrum of a guitar.

28. The system according to claim 26, wherein another sensor, namely a strumming sensor, monitors the angular orientation of the other elbow such that the signals from the strumming sensor are calibrated into at least two different operating positions.

29. The system according to claim 28, wherein one of the operating positions represents when the elbow is oriented in a curled position and the other when the elbow is oriented in an extended position and the two operating positions are separated by a further range of angular orientations that is not assigned an output.

30. The system according to claim 29, wherein when the electronic device receives a signal from the strumming sensor indicating that the elbow has moved from the curled position to an extended position, the electronic device produces or plays an output that corresponds to down strumming a particular chord determined by the signal from the fret sensor, and when the electronic device receives a signal indicating that the strumming sensor has moved from an extended position to the curled position, the electronic device produces or plays a sound that corresponds to an upward strumming of the guitar of a particular chord that is determined by the signal of the fret sensor.

31. The system according to claim 28, wherein the rate of change of the strumming sensor determines, at least in part, the volume at which outputs are played.

32. The system according to claim 28, wherein the strumming sensor is calibrated into 3 or more operating positions and each operating position is associated with an output that reflects a single string of a guitar being plucked.

37. A method of operating a system including one or more sensors having a plurality of operating positions and an electronic device for producing or playing outputs, wherein the electronic device receives signals from the or each sensor that is based on the operating positions of the sensors, and wherein the method includes the steps of:

a) adjusting the sensor(s) between different operating positions; and

b) assigning at least one output to the operating positions of the or each sensor such that when adjusting the sensor(s) between different operating positions in accordance with step a), the electronic device produces or plays an output.

38. A method of operating a system including one or more sensors having a plurality of operating positions and an electronic device for producing or playing outputs based on biomechanical positions or movement of an individual operator, the method including the steps of:

a) transmitting a signal from the sensors to the electronic device, wherein the sensors are adapted to transmit signals based on biomechanical movement and positions of the operator; and

b) assigning at least one output to the operating positions of the or each sensor such that when adjusting the sensors between different operating positions, the electronic device produces or plays the desired output.

39. The method according to claim 38, wherein the outputs are audible outputs including notes, chords, riffs, bars of music, or any section of actual music.

40. The method according to claim 38, wherein the method includes moving the sensor between known operating positions having known outputs assigned thereto so as produce a known sequence of outputs.

41. The method according to claim 38, wherein the method includes step c) which involves calibrating the sensors or electrical device so as to define the operating positions of the sensor based on selected biomechanical positions of individual operators.

42. The method according to claim 41, wherein step c) involves defining particular operating positions over a range of select biomechanical positions and thus, calibrate the operating positions relative to a range of values of the signals of the sensors received by the electronic device.

43. The method according to claim 42, wherein the biomechanical movement or position is a function of the angular orientation of one or more body joints such as finger, wrist, elbow, shoulder, neck, back, hips, knees or ankles.

44. The method according to claim 43, wherein step b) involves assigning outputs to the operating positions based on the operating positions resembling the angular orientation or gestures that would be required when operating an actual instrument.

45. The method according to claim 41, wherein any one or a combination of steps a), b) and c) are carried out while the electronic device communicates or presents to the operator information relating to either one or both of:

the operating position of the sensors and whether this falls within the calibrated operating positions; and

/or

the type of output assigned to the calibrated operating positions.

46. The method according to claim 45, wherein the information is presented or communicated to the operator by visual and/or audio sources.

47. The method according to claim 46, wherein the visual source displays to the operator all of the possible operating positions of the sensors and, at any point in time, the operating positions in which each sensor is located.

48. The method according to claim 47, wherein the visual source is a monitor that displays to the operator a picture or schematic illustration of an instrument, or a part thereof, and each operating position that has been calibrated approximately simulates the biomechanical position that an operator would assume when playing the instrument depicted in the monitor.

49. The method according to claim 46, wherein the visual source displays information identifying the type of output assigned to each operating position.

50. The method according to claim 48, wherein the type of output that has been assigned to the operating positions is based on or matches outputs of an actual instrument that resembles the biomechanical position of an operator playing the instrument depicted.

51. The method according to claim 41, wherein the method further includes step d) that involves storing data on the signals or the range of values of the signals that have been calibrated as defining the operating positions.

52. The method according to claim 51, wherein step d) also involves storing data on the outputs assigned to the operating positions.

53. The method according to claim 51, wherein step d) stores data profiles relating to either one or a combination of operating positions and outputs assigned to each operating position with reference to individual operators, and the electronic device can be switched between different stored profiles to facilitate use by more than one operator without requiring recalibration.

54. The method according to claim 41, wherein the method includes analysing the signals from the sensors using an algorithm that calculates whether the signals equal or falls within the ranges of the values of the signals that have been selected as defining or calibrated with the operating positions in accordance with step c).

55. The method according to claim 41, wherein the method includes analysing the signals from the sensors using an algorithm that calculates the rate of change of the value of the signals of the sensors, and in turn, determines the volume at which an audible output is played.

56. The method according to claim 41, wherein the method includes playing a partially complete piece of music at the same time as carrying out steps a) and b) to accompany the music being played.

57. The method according to claim 41, wherein the system includes two or more sensors having multiple operating positions and step b) involves assigning to at least one of the operating positions of the one of the sensors a number of outputs that equals or is less than the number of operating positions of the other sensor.

58. The method according to claim 41, wherein the system includes two sensors for monitoring the angular orientation of the elbows of the operator.

59. The method according to claim 58, wherein step c) involves calibrating the sensors or electrical device such that the operating positions of the elbows of the operator have an angular orientation ranging over from approximately 5 to 45 degrees.

60. The method according to claim 59, wherein the sensor and/or electrical device are calibrated such that the operating positions are defined over angular orientations ranging from 5 to 20 degrees.

61. The method according to claim 58, wherein one of the sensors, namely a fret sensor, monitors the angular orientation of one of the operator'"'"'s elbow and is calibrated into at least six ranges, each representing or simulating biomechanical positions of an operator'"'"'s elbow on the fret board of a guitar.

62. The method according to claim 61, wherein the six operating positions of the fret sensor are associated with a pair of outputs, each pair representing a chord of a guitar that produces the sound of the chord played by an up-strum or a down-strum of a guitar.

63. The method according to claim 61, wherein another sensor, namely a strumming sensor, monitors the angular orientation of the other elbow such that the signals from the strumming sensor are calibrated into at least two different operating positions.

64. The method according to claim 63, wherein one of the operating positions represents when the elbow is oriented in a curled position and the other when the elbow is oriented in an extended position and the two operating positions are separated by a further range of angular orientations that is not assigned an output.

65. The method according to claim 64, wherein when the electronic device receives a signal from the strumming sensor indicating that the elbow has moved from the curled position to an extended position, the electronic device produces or plays an output that corresponds to down strumming a particular chord determined by the signal from the fret sensor, and when the electronic device receives a signal indicating that the strumming sensor has moved from an extended position to the curled position, the electronic device produces or plays a sound that corresponds to an upward strumming of the guitar of a particular chord that is determined by the signal of the fret sensor.

66. The method according to claim 63, wherein the strumming sensor is calibrated into 3 or more operating positions and each operating position is associated with an output that reflects a single string of a guitar being plucked.

67. A garment that is worn by a person that is operating a system that produces or plays outputs, the garment including:

one or more than one sensor that is removably attached to the garment and at least one of the sensors is operable over two or more than two operating position; and

a transmitter that is removably attached to the garment and receives signals from the sensors and wirelessly transmits signals to an electronic device,

wherein the sensors are adapted to transmit signals base on biomechanical movement and positions of the person wearing the garment.

68. The garment according to claim 67, wherein the sensors and transmitter are attached to different sections of the garment and are interconnected by a conductive medium such as a yarn or thread that conveys signals from the sensor to the transmitter.