Accelerated Learning, Entertainment and Cognitive Therapy Using Augmented Reality Comprising Combined Haptic, Auditory, and Visual Stimulation

• US 20150317910A1
• Filed: 05/03/2014
• Published: 11/05/2015
• Est. Priority Date: 05/03/2013
• Status: Active Grant

• Current Assignee: John James Daniels
• Original Assignee: John James Daniels
• Inventors: Daniels, John James
• Granted Patent: US 9,390,630 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61M 2021/0016 by the smell sense
• CPC Class Codes: A61M 2021/0022 by the tactile sense, e.g. vibrations
• CPC Class Codes: A61M 2021/0027 by the hearing sense
• CPC Class Codes: A61M 2021/005 images, e.g. video
• CPC Class Codes: A61M 2021/0072 with application of electrical currents
• CPC Class Codes: A61M 2021/0077 with application of chemical or pharmacological stimulus
• CPC Class Codes: A61M 21/00 Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
• CPC Class Codes: A61M 21/0094 Isolation chambers used therewith, i.e. for isolating individuals from external stimuli other treatment rooms or enclosures A61G10/00
• CPC Class Codes: A61M 2205/507 Head Mounted Displays [HMD]
• CPC Class Codes: A61M 2210/0612 Eyes
• CPC Class Codes: A61M 2210/0662 Ears
• CPC Class Codes: A61M 2210/083 Arms
• CPC Class Codes: A61M 2210/086 Legs
• CPC Class Codes: G09B 15/00 Teaching music metronomes G04F5/02
• CPC Class Codes: G09B 15/003 with indication of the keys or strings to be played on instruments
• CPC Class Codes: G09B 15/06 Devices for exercising or strengthening fingers or arms; Devices for holding fingers or arms in a proper position for playing for teaching typing G09B13/00; exercising apparatus for developing or strengthening the muscles for physical training A63B21/00, A63B23/00
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G09B 19/22 Games, e.g. card games
• CPC Class Codes: G09B 21/00 : Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 5/065 : Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G09B 9/00 : Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: G09B 9/003 : for military purposes and tactics
• CPC Class Codes: G09B 9/06 : for teaching control of ships, boats, or other waterborne vehicles
• CPC Class Codes: G09B 9/063 : by using visual displays
• CPC Class Codes: G09B 9/08 : for teaching control of aircraft, e.g. Link trainer
• CPC Class Codes: G09B 9/52 : for teaching control of an outer space vehicle
• CPC Class Codes: G10H 1/18 : Selecting circuits
• CPC Class Codes: G16H 20/30 : relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising

Abstract

An accelerated learning and rehabilitation system for teaching the performance of a musical instrument, a remotely operated system, a sport, a weapon, and for brain rehabilitation and other uses includes generating sensory cues including auditory, haptic and visual sensory cues capable of being perceived by a user. The generated sensory cues are applied to the user and are dependent on a position of at least one body member of a performer relative to a performance element of a performance object with which an event is performed. The sensory cues are effective for stimulating a various processing center of a brain of the user so that user learns how to position his body member corresponding to the position of the performer of the event. The sensory cues can include visual sensory cues effective for stimulating the visual processing center of the brain of the user. The visual sensory cues are synchronized with the other applied sensory cues so that the position of the body member of the performer is virtually visually indicated in synchronization with the other sensory cues so that the visual processing center is stimulated with a visual sensory cue in synchronization with the stimulation of other processing centers corresponding to the other sensory cues for teaching the user to perform a version of the event.

Claims

1. A method, comprising:

generating a plurality of first sensory cues capable of being perceived by a user, each first sensory cue of the plurality of first sensory cues being dependent on a position of at least one body member of a performer relative to a performance element of a performance object with which an event is performed, the plurality of first sensory cues being effective for stimulating a first processing center of a brain of the user; and

generating a plurality of visual sensory cues capable of being displayed to the user on a video display device, the visual sensory cues providing a virtual visual indication to the user of the position of the at least one body member, the visual sensory cues being effective for stimulating the visual processing center of the brain of the user, the visual sensory cues being synchronized with the first sensory cues so that the position of the at least one body member is virtually visually indicated in synchronization with the first sensory cue and so that the visual processing center is stimulated with a visual sensory cue in synchronization with a first sensory cue stimulating the first processing center, wherein the synchronized stimulation of the first processing center and the visual processing center is effective for teaching the user to perform a version of the event.

2. A method according to claim 1;

further comprising generating a second plurality of sensory cues capable of being perceived by the user, each second sensory cue of the plurality of sensory cues being dependent on at least one of the position of the at least one body member and an action of the event, the action dependent on the position of the at least one body member, the second sensory cues being effective for stimulating at least a second processing center of the brain of the user, the second sensory cues being synchronized with the first sensory cues so that the second processing center is stimulated with a second sensory cue in synchronization with a first sensory cue stimulating the first processing center, wherein the synchronized stimulation of the first processing center, the visual processing center and the second processing center is effective for teaching the user to perform a version of the event.

3. A method according to claim 1;

wherein the video display device comprises at least one of pair of augmented or virtual eyeglasses, a computer monitor, a television, a smart phone display or a personal information device display.

4. A method according to claim 1;

wherein the event comprises controlling at least one of a sports related object, a weapon, a video gaming controller, a remotely controllable system including a space probe, a drone aircraft, an underwater probe, a robot.

5. A method according to claim 1;

wherein at least one of the first and the second plurality of sensory cues are remotely determined from corresponding to the event that is performed, the event being remote in at least one of time and location relative to the user; and

wherein at least one of the first and the second plurality of sensory cues stimulates a brain processing center for at least one of the five senses of hearing, seeing, smelling, feeling and taste.

6. A method, comprising:

generating auditory sensory cues capable of being heard a user, the auditory sensory cues comprising a plurality of musical tones played in series and making up a piece of music to be played on keys of a musical instrument, the musical tones comprising one or more individual notes, each individual note corresponding to a respective key of the musical instrument;

generating haptic sensory cues capable of being felt by the user, the haptic sensory cues providing a haptic indication to one or more fingers of the users to indicate which finger of said one or more fingers is to be used to play said each respective key corresponding to the one or more individual notes of the musical tones played on the instrument, the haptic sensory cues being synchronized with the auditory sensory cues so that a predetermined finger that is to play the respective key is stimulated with a haptic sensory cue in synchronization with an auditory sensory cue comprised of said individual note; and

generating visual sensory cues capable of being displayed to the user, the visual sensory cues providing a virtual visual indication to the user of which finger of said one or more fingers is to be used to play said each respective key corresponding to the individual note of the musical tone played on the instrument, the visual sensory cues being synchronized with the auditory sensory cues so that the predetermined finger that is to play the respective key is virtually visually indicated in synchronization with the auditory sensory cue comprised of said individual note.

7. A method according to claim 6;

wherein the haptic sensory cue is a vibration applied to the predetermined finger.

8. A method according to claim 7;

wherein the vibration has an intensity corresponding to an audio volume of the individual note.

9. A method according to claim 6;

wherein the haptic sensory cue is an electrical impulse applied to a nerve corresponding to the predetermined finger.

10. A method according to claim 6;

wherein the virtual visual indication is displayed on a display comprising at least one of a pair of augmented or virtual eyeglasses, a computer monitor, a television, a smart phone display or a personal information device display.

11. A method according to claim 6;

wherein the virtual visual indication comprises at least one of a color image, light intensity or light position displayed on a display.

12. A method according to claim 6;

wherein the virtual visual indication comprises at least one of an image displayed on a display of at least one hand indicating the predetermined finger.

13. A method according to claim 6;

wherein the virtual visual indication comprises a light located in proximity to the predetermined finger.

14. A method according to claim 6;

wherein the virtual visual information comprises a virtual image displayed on augmented reality eyeglasses as a virtual visual overlay to a real-world scene.

15. A method according to claim 6;

further comprising providing a learning enhancing chemical effective to induce a learning-enhanced state in the brain of the user; and

administering the learning enhancing chemical to the user in conjunction with the generation of the auditory sensory cues and at least one of the generation of the haptic sensory cues and the visual sensory cues.

16. A method, comprising:

determining auditory information usable for providing auditory sensory cues capable of being heard by a user, the auditory sensory cues comprising a plurality of musical tones played in series and making up a piece of music to be played on keys of a musical instrument, the musical tones comprising one or more individual notes, each individual note corresponding to a respective key of the musical instrument; and

at least one of

determining digit position information, the digit position information for providing haptic sensory cues capable of being felt by the user, the haptic sensory cues providing an indication of one or more fingers of the user to indicate which finger of said one or more fingers is to be used to play said each respective key corresponding to the one or more individual notes of the musical tones played on the instrument, the determined digit position information being usable for generating haptic sensory cues synchronizable with the auditory sensory cues so that a predetermined finger that is to play the respective key can be stimulated with a haptic sensory cue in synchronization with an auditory sensory cue comprised of said individual note; and

determining visual information usable for providing visual sensory cues capable of being displayed to the user, the visual sensory cues providing a visual indication to the user of which finger of said one or more fingers is to be used to play said each respective key corresponding to the individual note of the musical tone played on the instrument, the determined visual information being usable for generating visual sensory cues synchronizable with the auditory sensory cues so that the predetermined finger that is to play the respective key is visually indicated in synchronization with the auditory sensory cue comprised of said individual note.

17. A method according to claim 16;

wherein the determined auditory information is determined by recording a performance by a performer playing the piece of music on the musical instrument

18. A method according to claim 16;

wherein the determined digit position information is determined by (digit/key detecting keyboard), pattern recognition software.

19-28. -28. (canceled)