Codification and cueing system for sport and vocational activities

• US 10,668,353 B2
• Filed: 08/11/2015
• Issued: 06/02/2020
• Est. Priority Date: 08/11/2014
• Status: Active Grant

• Current Assignee: icuemotion, LLC
• Original Assignee: icuemotion, LLC
• Inventors: Mettler May, Berenice
• Primary Examiner(s): Lewis, David L
• Assistant Examiner(s): Thomas, Eric M
• Application Number:
  US15/502,612
• Publication Number:
  US 20170232324A1
• Time in Patent Office: 1,757 Days
• CPC Class Codes: A63B 69/38 for tennis A63B61/006, A63B69/0073 and A63B69/0097 take precedence
• CPC Class Codes: A63B 71/06 Indicating or scoring devices for games or players , or for other sports activities
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising

Abstract

A codification and cueing system for adversarial or cooperative interactions between actors engaged in sports and other activities uses measurements from motion sensors to identify events in the activity and participant'"'"'s behavior to provide real-time cues to help the actors perform as well as train. The codification describes the interactions based on behavioral events and movements. The cueing method can be implemented using different subsets of such events. Following detection of these events, a signal is transmitted to generate auditory, visual and/or haptic stimuli. These stimuli encode information and hence the cues help the participants select, prepare for and execute relevant aspects of the performance.

Claims

1. A cueing system comprising:

one or more sensors configured to take measurement data from an environment and one or more of first and second participants engaged in an activity in the environment, wherein the one or more sensors are selected from an inertial sensor, a magnetometer, an acoustic sensor, a strain gauge sensor, an optical sensor, and a light sensor;

a processor in communication with the one or more sensors, wherein the processor is configured to;

analyze the measurement data from the one or more sensors to detect events associated with the environment and the one or more participants;

extract attributes associated with the events from the measurement data;

predict a movement outcome associated with one of the events, based on the extracted attributes; and

generate one or more instructions based on the predicted movement outcome;

a cue administrator configured to generate one or more anticipatory cues for the first participant with a first cueing device, based on the one or more instructions generated from the processor; and

a wireless transmitter in communication with a second cueing device, wherein one or more anticipatory cues are further generated for the second participant and the one or more anticipatory cues for the second participant are administered as cue stimuli encoding timing and magnitude information selected to reduce a response time between the one event and recognition of the one event by the second participant.

2. The cueing system of claim 1, further comprising a racket, wherein the first cueing device is wearable or formed in a frame of the racket, or fitted on the frame or string bed of the racket using a securement device for securing the first cueing device to the racket.

3. The cueing system of claim 1, wherein the cue administrator is incorporated into the first cueing device and comprises one or more of a speaker, a light emitter, or a vibration generator.

4. The cueing system of claim 1, further comprising the wireless transmitter in communication with a smart phone or tablet configured for access to the measurement data collected during the activity.

5. The cueing system of claim 1, wherein the cue administrator generates the one or more anticipatory cues based on detection of a movement, extraction of movement phase information for the one or more participants, prediction of the movement outcome, and processing of the movement and phase information, wherein the detected movement and extracted movement phase information are analyzed to predict the movement outcome.

6. The cueing system of claim 1, wherein the cue administrator comprises a separate unit that is in communication with the first cueing device.

24. The cueing system of claim 1, wherein the processor is further configured for detection and extraction of features in the measurement data for generation of the one or more anticipatory cues, the features corresponding to kinematics or dynamics of a behavior of the one or more participants, and wherein the predicted movement outcome is described by a behavior model of the behavior.

25. The cueing system of claim 24, wherein the movement outcome comprises ball trajectory.

26. The cueing system of claim 24, wherein the one or more anticipatory cues encode ball velocity, spin type, or ball depth.

27. The cueing system of claim 1, wherein the extracted attributes comprise state information selected from position of the one or more participants, orientations of a racket, ball speed and spin.

7. A cueing apparatus comprising:

a housing for a first cueing device;

with;

at least one sensor configured to collect measurement data from an environment and one or more of first and second participants engaged in an activity, wherein the at least one sensor is selected from an inertial sensor, a magnetometer, an acoustic sensor, a strain gauge sensor, an optical sensor, and a light sensor; and

a processor in communication with the at least one sensor, wherein the processor is configured to;

analyze the collected measurement data from the at least one sensor to detect events associated with the environment and the one or more participants;

extract attributes associated with the events from the collected measurement data;

predict a movement outcome associated with one of the events, based on the extracted attributes; and

generate one or more instructions based on the predicted movement outcome;

a cue administrator configured to generate one or more anticipatory cues for the first participant based on the one or more instructions generated from the processor;

a power supply; and

a wireless transmitter in communication with a second cueing device, wherein one or more anticipatory cues are further generated for the second participant and the one or more anticipatory cues for the second participant are administered as cue stimuli encoding timing and magnitude information selected to reduce a response time between the one event and recognition of the one event by the second participant.

8. The cueing apparatus of claim 7, further comprising a racket, wherein the cueing apparatus is wearable or formed in a frame of the racket or fitted on the frame or string bed of the racket using a securement device for securing the cueing apparatus to the racket.

9. The cueing apparatus of claim 7, wherein the cue administrator comprises one or more of a speaker, a light emitter, or a vibration generator.

10. The cueing apparatus of claim 7, further comprising the wireless transmitter in communication with a smart phone or tablet configured for access to the measurement data collected during the activity.

11. The cueing apparatus of claim 7, wherein the cue administrator generates the one or more anticipatory cues based on detection of a movement, extraction of movement phase information for the one or more participants, prediction of the movement outcome, and processing of the movement and phase information, wherein the detected movement and extracted movement phase information are analyzed to predict the movement outcome.

28. The cueing apparatus of claim 7, wherein the one or more anticipatory cues comprise a rhythmic pattern of signals selected from auditory, visual, and haptic signals.

29. The cueing apparatus of claim 28, wherein the rhythmic pattern is adjusted based on a desired pace of the activity or adapted based on performance of the activity by the one or more participants.

30. The cueing apparatus of claim 7, wherein the processor is further configured for detection and extraction of features in the measurement data for generation of the one or more anticipatory cues to provide an objective timing reference signaling an impact to establish a rhythm for the one or more participants, wherein the features correspond to kinematics or dynamics of a behavior of the one or more participants and the predicted movement outcome is described by a behavior model of the behavior.

31. The cueing apparatus of claim 7, wherein the one or more anticipatory cues are modulated in magnitude or frequency to provide anticipatory information about the predicted movement outcome.

32. The cueing apparatus of claim 7, wherein the one or more anticipatory cues are communicated to the second participant with a nominal delay that varies based on the magnitude information to provide anticipatory information about the predicted movement outcome.

33. The cueing apparatus of claim 32, wherein the predicted movement outcome comprises ball speed and the nominal delay varies based on stroke intensity to provide information about the ball speed.

12. A cueing system comprising:

a first cueing device having one or more first sensors configured to collect data from an environment and a first participant engaged in an activity, wherein the first sensors are selected from an inertial sensor, a magnetometer, an acoustic sensor, a strain gauge sensor, an optical sensor, and a light sensor, with;

a first cueing processor in communication with the first sensors, wherein the first cueing processor is configured to analyze the data collected from the first sensors to detect first events associated with the first participant, to extract first attributes associated with the first events, to predict first movement outcomes associated with the first events based on the first extracted attributes, and to generate one or more first instructions based thereon, and

a first cue administrator in communication with the first cueing processor, configured to generate one or more anticipatory cues based on the first instructions from the first cueing processor; and

a wireless transmitter in communication with a second cueing device having one or more second sensors configured to collect data from the environment and a second participant engaged in the activity, wherein the second sensors are selected from an inertial sensor, a magnetometer, an acoustic sensor, a strain gauge sensor, an optical sensor, and a light sensor, with;

a second cueing processor in communication with the second sensors, wherein the second cueing processor is configured to analyze the data collected from the second sensors to detect second events associated with the second participant, to extract second attributes associated with the second events, to predict second movement outcomes associated with the second events based on the second extracted attributes, and to generate one or more second instructions based thereon, and

a second cue administrator in communication with the second cueing processor, configured to generate one or more anticipatory cues based on the second instructions from the second cueing processor;

wherein the one or more anticipatory cues are administered as cue stimuli encoding timing and magnitude information selected to reduce a response time between the first and second events and recognition of the first and second events by the participants.

13. The cueing system of claim 12, further comprising at least one of a first securement device for securing the first cueing device to a first racket and a second securement device for securing the second cueing device to a second racket, wherein measurement of stroke and racket state of the first participant at impact is used to predict a ball trajectory and to cue the second participant by producing a cue stimulus that encodes at least one of expected ball velocity, spin type, and ball depth.

14. The cueing system of claim 12, further comprising the wireless transmitter in communication with one or more of a smart phone or tablet configured for access to the data collected during the activity.

15. The cueing system of claim 12, wherein the cue administrators generate the anticipatory cues based on detection of a movement, extraction of movement phase information for the participants, prediction of the movement outcomes, and processing of the movement and phase information, wherein the detected movement and extracted movement phase information are analyzed to predict the movement outcomes.

34. The cueing system of claim 12, wherein the first cueing device transmits the first extracted and estimated attributes to the second cueing device and the second cueing device generates the one or more respective anticipatory cues based on the first extracted and estimated attributes.

35. The cueing system of claim 12, wherein the anticipatory cues are configured to inform the second participant of an action of the first participant.

36. The cueing system of claim 12, wherein cueing processors are further configured for detection and extraction of features in the data collected from the environment and the first participant engaged in the activity for generation of the anticipatory cues to elicit an action from the second participant, wherein the features correspond to kinematics or dynamics of a behavior of the first participant and the first predicted movement outcome is described by a behavior model of the behavior.

37. The cueing system of claim 12, further comprising the wireless transmitter in communication with one or more of a smart phone or tablet configured for access to the data collected during the activity.

38. The cueing system of claim 12, wherein the anticipatory cues are communicated to the second participant with a nominal delay that varies based on the magnitude information to provide anticipatory information about the predicted movement outcomes.

39. The cueing system of claim 38, wherein the predicted movement outcomes comprise ball speed and the nominal delay varies based on stroke intensity to provide information about the ball speed.

40. The cueing system of claim 38, wherein the anticipatory cues are based on a combination of signals from the first and second participants, and wherein a phase difference between the two signals provides a measure of advance or delay used to modulate the cues.

16. A method for cueing a participant in an environment, the method comprising:

obtaining sensed data from a cueing device having one or more sensors selected from an inertial sensor, a magnetometer, an acoustic sensor, a strain gauge sensor, an optical sensor, and a light sensor, wherein the sensed data are taken from an environment and one or more participants engaged in an activity in the environment;

analyzing the obtained data to detect an event associated with the environment and the one or more participants;

extracting attributes associated with the event from the obtained data;

predicting a movement outcome associated with the event in the environment based on the attributes extracted from the analyzed data;

generating an instruction to issue an anticipatory cue based on the predicted movement outcome; and

communicating the anticipatory cue to one or more of the participants via a wireless transmitter, wherein the anticipatory cue is administered as a cue stimulus encoding timing and magnitude information selected to reduce a response time between the event and recognition of the event by the one or more participants.

17. The method of claim 16, further comprising securing the cueing device to a racket, wherein measurement of stroke and racket state at impact is used to predict a ball trajectory, and cueing a second participant by producing a cue stimulus that encodes at least one of expected ball velocity, spin type, and ball depth.

18. The method of claim 16, further comprising communicating with at least one of a smart phone and a tablet configured for access to the sensed data obtained during the activity.

19. The method of claim 16, further comprising generating the anticipatory cue based on detection of a movement, extraction of movement phase information for the one or more participants, prediction of the movement outcome, and processing of the movement and phase information, wherein the detected movement and extracted movement phase information are analyzed to predict the movement outcome.

20. The method of claim 16, further comprising encoding the timing and magnitude information in the anticipatory cue, wherein the timing and magnitude information describes one or more of ball trajectory, court event, player behavior or racket motion to help the one or more participants to prepare a stroke or anticipate an adversary shot type.

21. The method of claim 16, further comprising administering the anticipatory cue at a time selected to reduce the response time between an opponent'"'"'s strike or return and the recognition of the event, wherein the anticipatory cue enables earlier detection of the event than absence of the anticipatory cue.

22. The method of claim 16, further comprising encoding event type information in the anticipatory cue, wherein the anticipatory cue triggers selection of a specific type of stroke by the participant.

23. The method of claim 16, further comprising encoding action type information in the anticipatory cue, wherein the anticipatory cue triggers the participant to assume a body posture for initiating and execution of a selected stroke type.