System and method that generates outputs

• US 8,830,162 B2
• Filed: 06/29/2007
• Issued: 09/09/2014
• Est. Priority Date: 06/29/2006
• Status: Expired due to Fees

• Current Assignee: Superinteractive Pty. Ltd.
• Original Assignee: Commonwealth Scientific & Industrial Research Organisation (Commonwealth Of Australia As Represented By Department Of Industry, Innovation, Climate Change, Science, Research And Tertiary Education)
• Inventors: Helmer, Richard James Neil
• Primary Examiner(s): Hicks, Charles V
• Application Number:
  US12/306,643
• Publication Number:
  US 20090256801A1
• Time in Patent Office: 2,629 Days
• Field of Search: 345/156
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G10H 1/342 for guitar-like instruments with or without strings and with a neck on which switches or string-fret contacts are used to detect the notes being played electric guitars in which the tones are generated by the vibration of strings G10H3/18
• CPC Class Codes: G10H 1/348 Switches actuated by parts of the body other than fingers
• CPC Class Codes: G10H 2220/005 Non-interactive screen display of musical or status data graphical user interfaces specifically adapted for electrophonic musical instruments G10H2220/091; fingering displays G10H2220/041
• CPC Class Codes: G10H 2220/321 Garment sensors, i.e. musical control means with trigger surfaces or joint angle sensors, worn as a garment by the player, e.g. bracelet, intelligent clothing vital parameter sensing G10H2220/371; wearable interfaces in general G06F3/00
• CPC Class Codes: G10H 2230/135 Spint guitar, i.e. guitar-like instruments in which the sound is not generated by vibrating strings, e.g. guitar-shaped game interfaces
• CPC Class Codes: G10H 2230/141 Spint guitar drum, i.e. mimicking a guitar used at least partly as a percussion instrument

Abstract

The present invention relates to a system and method that generates outputs based on the operating position of a sensor which is determined by the biomechanical positions or gestures of individual operators. The system including a garment on which one or more than one sensor is removably attached and the sensors provide a signal based on the biomechanical position, movement, action or gestures of the person wearing the garment, a transmitter receiving signals from the sensors and sends signals to a computer that is calibrated to recognize the signals as representing particular positions that are assigned selected outputs. Suitably the outputs are audio outputs of an instrument, such as a guitar, and the outputs simulate the sound of a guitar that would be played when the biomechanical motion, action, gesture or position of the operator resembles those that would occur when an actual instrument is played.

Claims

1. A system that is operated by a user to produce outputs, the system including:

a garment that is worn by the user having at least two sensors attached to the garment for sensing the biomechanical position of two different joints of the user'"'"'s body, each sensor being operated over two or more operating positions that each represent a different biomechanical position of the user, wherein the sensors are removably attached to the garment, thereby allowing the garment to be refreshed by washing or placed entirely as desired;

an electronic device that receives output signals from the sensors and has a display that displays i) graphically representations of the operating positions, ii) assignable outputs of the system, and the electronic device also producing the outputs of the system;

a transmitter that receives output signals from the sensors and wirelessly transmits the output signals to the electronic device;

an input device that can be operated to calibrate the system by assigning each graphical representation of each operating position to at least one of the outputs of the system, and assigning ranges of the output signals of each sensor to each of the graphical representations of the operating positions, and wherein the input device comprises a plurality of selectable buttons corresponding to the graphical representations of each operating position of the sensors, and operating the buttons allows the user to assign ranges of the output signals of each sensor to the graphical representations of the operating positions of the sensors, and allows the outputs of the system to be assigned to the graphical representations of the operating positions of at least one of the sensors from a range of possible outputs of the system displayed on the display;

a feedback means that communicates to the user the operating positions of the sensors by indicating on a display the graphical representation of the operating position for the output signal being received by the electronic at any one time, and the feedback means also communicates to the user information identifying which particular output of the system that has been assigned to each graphical representation of the operating positions; and

wherein a set of outputs can be reproduced by the user by repeating biomechanical movements of the joints for which the garment has sensors.

2. The system according to claim 1, wherein the sensors include first and second sensors, and the input device can be operated to assign each graphical representation of the operating positions of the first sensor to at least two outputs of the system, and the input device can be operated to associate the graphical representation of the operating positions of the second sensor to each of the two outputs of the graphical operating positions of the first sensor, such that the output of the system is determined by the operating positions of the first and second sensors.

3. The system according to claim 1, wherein the outputs of the system are in the form of either one or a combination of visual outputs and audible outputs.

4. The system according to claim 3, wherein the audible outputs are a pre-recorded sound or a stored sample of sound.

5. The system according to claim 3, wherein the audible outputs are produced or played with accompanying outputs that are unrelated to the operating position of the sensors.

6. The system according to claim 1, wherein the sensors includes an electrical resistor that provides variable electrical resistance depending on the various operating positions of the sensors and thereby provide the output signals of the resistors.

7. The system according to claim 1, wherein the operating positions of the sensors are defined by a range of angular orientations of one or more body joint of the user such as but by no means limited to finger, wrist, elbow, shoulder, neck, back, hip, knee or ankle joints.

8. The system according to claim 1, wherein the range of output signals that are assigned to graphical representations of the operating positions are adjustable by operating the input device or are adjustable so as to recalibrate the system to suit different individual users.

9. The system according to claim 1, wherein the system is calibrated for each individual operator and the stored configuration can be recalled or re-activated after use of the system by other operators.

10. The system according to claim 1, wherein the display includes a picture or schematic illustration of a musical instrument or a part thereof, and each operating position represents or simulates biomechanical positions of the operator playing the instrument which is shown on the display.

11. The system according to claim 1, wherein the delay between the sensors being located in an operative position and the electronic device producing or playing the desired output is equal to or less than 50 milliseconds.

12. The system according to claim 11, wherein the delay period is less than or equal to 10 milliseconds.

13. The system according to claim 1, wherein the electronic device has a memory that stores data on any one or a combination of:

the operating positions, including ranges of values of the output signals of the sensors that define operating positions;

the outputs of the system, including audible outputs; and

the assignment or allocation of particular outputs to selected operating positions.

14. The system according to claim 1, wherein the electronic device has a data process unit that processes data on any one or a combination of the following:

carrying out algorithms analyzing the signal from the sensor;

carrying out algorithms that assess whether the signals equals or falls within the ranges of the values of the signals that defines calibrated operation positions; and

carrying out algorithms that calculate the speed at which the value of a signal of a sensor changes, and thereby determines the volume at which an audible output should be played based on the rate of change of the signal.

15. The system according to claim 1, wherein the first and second sensors sense displaying the angular orientation of the elbows of the user.

16. The system according to claim 15, wherein the system is calibrated such that the operating positions have an angular orientation ranging from approximately 5 to 45 degrees.

17. The system according to claim 16, wherein the sensor and/or electrical device are calibrated such that the operating positions are defined over angular orientations ranging from 5 to 20 degrees.

18. The system according to claim 15, wherein the first sensor represents a fret sensor which displays the angular orientation of one of the operator'"'"'s elbows and is divided into at least six ranges, each being assigned one of the graphical representations of the operating position and each simulating biomechanical positions of an operator'"'"'s elbow with respect to the fret board of a guitar.

19. The system according to claim 18, wherein the six operating positions of the fret sensor are associated with a pair of outputs, each pair representing a chord of a guitar that produces the sound of the chord played by an up-strum or a down-strum of a guitar.

20. The system according to claim 15, wherein the second sensor represents a strumming sensor and, displays the angular orientation of the other elbow such that the output signals from the strumming sensor are assigned to at least two different graphical representations of the operating positions.

21. The system according to claim 20, wherein a first graphical representation of a first operating position represents when the elbow is oriented in a curled position and a second graphical representation of a second operating position represents when the elbow is oriented in an extended position, and the two operating positions are separated by a further range of angular orientations of the elbow that is not assigned an output of the system.

22. The system according to claim 21, wherein when the electronic device receives the output signal from the strumming sensor indicating that the elbow has moved from the first graphical representation of the operation position to the second graphical representation of the operating position, i.e. the elbow at the strumming sensor has moved from a curled position to an extended position, the electronic device produces or plays an output that corresponds to down strumming a particular chord determined by the signal from the fret sensor, and when the electronic device receives a signal indicating that the strumming sensor has moved from an extended position to the curled position, the electronic device produces or plays a sound

that corresponds to an upward strumming of the guitar of a particular chord that is determined by the signal of the fret sensor.

23. The system according to claim 20, wherein the electronic device includes an algorithm that calculates the rate of change of the output signal of the strumming sensor which determines, at least in part, the volume at which outputs of the system are played.

24. A method of operating a system including:

a garment that is worn by a user having at least two sensors removably attached to the garment for sensing the biomechanical position of two different joints of a user'"'"'s body, each sensor being operated over two or more operating positions that each represent a different biomechanical position of the user;

an electronic device that has a display and comprises a plurality of selectable buttons corresponding to the graphical representations of each operating position of the sensors, the electronic device also producing the outputs of the system; and

an input device that can be operated to calibrate the system, the method including the steps of;

a) transmitting output signals wirelessly from the sensors to the electronic device, wherein the sensors are adapted to transmit signals based on biomechanical movement and positions of the operator;

b) displaying on the display i) graphical representations of two or more operating positions and ii) the outputs of the system that can be assigned to the graphical representations,

c) assigning each graphical representation of the operating positions to at least one of the outputs of the system, wherein step b) includes operating buttons of the input device so as to assign outputs from the range of assignable outputs of the system to the graphical representations of the operating positions of at least on the sensors;

d) assigning ranges of the output signals of each sensor to each of the graphical representations of the operating positions, wherein step c) includes operating buttons of the input device so as to assign ranges of the output signals of each sensor to the graphical representations of the operating positions of the sensors;

e) communicating to the user the operating positions of the sensors by indicating on the display the graphical representation of the operating position for the output signal being received by the electronic device at any one point of time;

f) communicating to the user information identifying which particular output of the system that has been assigned to each graphical representation of the operating positions,

g) reproducing a set of outputs by the user repeating biomechanical movements of the joints for which the garment has sensors, and

h) removing the sensors from the garment, thereby allowing the garment to be refreshed by washing or placed entirely as desired.

25. The method according to claim 24, wherein the sensors of the system include first and second sensors, and the input device comprises a plurality of selectable buttons for each sensor, wherein the method includes assigning each graphical representation of the operating positions of the first sensor to at least two outputs of the system, and associating the graphical representation of the operating positions of the second sensor to each of the two outputs of the graphical operating positions of the first sensor, and determining the output of the system based on the operating positions of the first and second sensors.

26. The method according to claim 24, wherein the outputs are audible outputs including notes, chords, riffs, bars of music, or any section of actual music.

27. The method according to claim 24, wherein the method includes moving the sensors between known operating positions that have been graphically represented and to which known outputs have been assigned so as produce a known sequence of outputs.

28. The method according to claim 24, wherein step d) involves defining particular operating positions over a range of selectable biomechanical positions and thus to a range of values of the signals of the sensors received by the electronic device.

29. The method according to claim 28, wherein the biomechanical movement or position is a function of the angular orientation of one or more body joints such as finger, wrist, elbow, shoulder, neck, back, hips, knees or ankles.

30. The method according to claim 29, wherein step c) involves assigning outputs to the graphical representations of the operating positions based on operating positions resembling the angular orientation or gestures that would be required when operating an actual instrument.

31. The method according to claim 24, wherein the information is presented or communicated to the operator by visual and/or audio sources.

32. The method according to claim 31, wherein the visual source displays to the user all of the possible operating positions of the sensors and, at any point in time, the operating positions in which each sensor is located.

33. The method according to claim 32, wherein the visual source is a display that displays to the user a picture or schematic illustration of an instrument, or a part thereof, and each graphical representation of the operating position simulates the biomechanical position that the user would assume when playing the instrument depicted in the display to produce the output assigned to the graphical representation of the operating position.

34. The method according to claim 31, wherein the visual source displays information identifying the type of output assigned to each graphical representation of the operating position.

35. The method according to claim 24, wherein the method further includes step h) that involves storing data on the signals or the range of values of the output signals that have been assigned to the graphical representation of the operating positions for each user.

36. The method according to claim 35, wherein step h) stores data profiles relating to either one or a combination of operating positions and outputs assigned to each operating position with reference to individual operators, and the electronic device can be switched between different stored profiles to facilitate use by more than one operator without requiring recalibration.

37. The method according to claim 24, wherein the method includes analyzing the signals from the sensors using an algorithm that calculates whether the output signals equal or falls within the ranges of the values of the signals that have been selected as defining or calibrated with the operating positions in accordance with step d).

38. The method according to claim 24, wherein the method includes analyzing the output signals from the sensors using an algorithm that calculates the rate of change of the value of the output signals of the sensors, and in turn, determines the volume at which an audible output of the system is played.

39. The method according to claim 24, wherein the method includes playing a partially complete piece of music at the same time as carrying out steps a) and b) to accompany the music being played.

40. The method according to claim 24, wherein the system includes two sensors for displaying the angular orientation of the elbows of the operator.

41. The method according to claim 40, wherein step d) involves assigning the output signals of the sensors in which the elbows of the operator have an angular orientation ranging over from approximately 5 to 45 degrees.

42. The method according to claim 25, wherein the first sensor is a fret sensor which, displays the angular orientation of one of the operator'"'"'s elbows and is divided into at least six ranges, each being assigned one of the graphical representations of the operating position and each simulating biomechanical positions of an operator'"'"'s elbow with respect to the fret board of a guitar.

43. The method according to claim 42, wherein the six operating positions of the fret sensor are associated with a pair of outputs, each pair representing a chord of a guitar that produces the sound of the chord played by an up-strum or a down-strum of a guitar.

44. The method according to claim 42, wherein the second sensor represents a strumming sensor and displays the angular orientation of the other elbow such that the output signals from the strumming sensor are assigned to at least two different graphical representations of the operating positions.

45. The method according to claim 44, wherein a first graphical representation of a first operating position represents when the elbow is oriented in a curled position and a second graphical representation of a second operating position represents when the elbow is oriented in an extended position and the two operating positions are separated by a further range of angular orientations that is not assigned an output.

46. The method according to claim 45, wherein when the electronic device receives the output signal from the strumming sensor indicating that the elbow has moved from the first graphical representation of the first operating position to the second graphical representation of the operating position, i.e. the elbow at the strumming sensor has moved from the curled position to an extended position, the electronic device produces or plays an output that corresponds to down strumming a particular chord determined by the signal from the fret sensor, and when the electronic device receives a signal indicating that the strumming sensor has moved from an extended position to the curled position, the electronic device produces or plays a sound that corresponds to an upward strumming of the guitar of a particular chord that is determined by the signal of the fret sensor.