Systems and methods for computerized interactive skill training

• US 10,127,831 B2
• Filed: 10/26/2016
• Issued: 11/13/2018
• Est. Priority Date: 07/28/2008
• Status: Active Grant

• Current Assignee: Breakthrough Performancetech LLC
• Original Assignee: Breakthrough Performancetech LLC
• Inventors: Cohen, Martin L., Brown, Edward G.
• Primary Examiner(s): Gishnock, Nikolai A
• Application Number:
  US15/335,182
• Publication Number:
  US 20170116881A1
• Time in Patent Office: 748 Days
• Field of Search: 434118, 434350, 434362
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/06398 Performance of employee with respect to a job function
• CPC Class Codes: G06Q 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G09B 15/00 Teaching music metronomes G04F5/02
• CPC Class Codes: G09B 17/003 electrically operated apparatus or devices
• CPC Class Codes: G09B 17/006 with audible presentation of the material to be studied audio and audio-visual teaching apparatus in general G09B5/04, G09B5/06
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 19/0046 History
• CPC Class Codes: G09B 19/0053 Computers, e.g. programming
• CPC Class Codes: G09B 19/025 with electrically operated apparatus or devices
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 19/18 Book-keeping or economics
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G09B 5/10 : all student stations being capable of presenting the same information simultaneously G09B5/14 takes precedence
• CPC Class Codes: G09B 7/00 : Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 : of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: G09B 7/04 : characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation
• CPC Class Codes: G09B 7/06 : of the multiple-choice answer-type, i.e. where a given question is provided with a series of answers and a choice has to be made from the answers
• CPC Class Codes: G09B 7/063 : with hand-placed probe or plug
• CPC Class Codes: G09B 7/08 : characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying further information
• CPC Class Codes: G09B 9/00 : Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04M 2201/38 : Displays
• CPC Class Codes: H04M 2201/40 : using speech recognition
• CPC Class Codes: H04M 2201/42 : Graphical user interfaces
• CPC Class Codes: H04M 2250/62 : user interface aspects of conference calls
• CPC Class Codes: H04M 2250/74 : with voice recognition means

Abstract

The present invention is directed to interactive training, and in particular, to methods and systems for computerized interactive skill training. An example embodiment provides a method and system for providing skill training using a computerized system. The computerized system receives a selection of a first training subject. Several related training components can be invoked, such as reading, watching, performing, and/or reviewing components. In addition, a scored challenge session is provided, wherein a training challenge is provided to a user via a terminal, optionally in video form.

Claims

1. A training system configured to train a user, comprising:

a server comprising a hardware computing device;

a network interface coupled to the server;

a tangible, non-transitory, computer-readable medium having computer-executable instructions stored thereon that, when executed by the server, cause the training system to perform operations comprising;

receiving an identification of a training module, the module including at least a first set of challenges including a plurality of challenges that include a statement or question regarding a subject that a user is to be trained to verbally respond to;

for a respective challenge, in response to a user action, transmitting, for presentation on a user terminal having a display, user interfaces associated with a learning session and user interfaces associated with a practice session, wherein;

the learning session includes a reading section configured to train the user in how to respond to the respective challenge using text, and a watching section, configured to train the user to respond to the respective challenge using an audio video presentation,

wherein the reading section includes a reading user interface configured to present via text;

the respective challenge;

a plurality of guideline language constructs that provide a model answer to the respective challenge, wherein the guideline constructs are in the form of text positioned so that each guideline language construct is spaced apart from at least one other guideline language construct, and wherein a given guideline language construct includes;

a key element; and

contextual language in which the key element is embedded, wherein computer-executable instructions are configured to cause the key element to be visually distinguished via an attribute not present in the contextual language;

wherein the watching section includes a watching user interface configured to present;

a textual representation of the key elements previously presented via the reading user interface, wherein the key elements are not embedded in the guideline language constructs;

an audio video presentation of a first animated avatar audibly presenting the guideline language constructs, including the key elements, presented via the reading user interface and/or audibly presenting the key elements without the guideline language constructs, wherein the first animated avatar'"'"'s lips, facial expression, and gestures are animated to provide a simulation of a human while being visually distinguishable as an animation rather than a real person and wherein the first avatar has lip motions at least substantially synchronized with the audible key elements;

wherein the practice session user interfaces include;

a performing user interface, wherein the performing user interface includes an audio video presentation of a second avatar, wherein the second avatar has a different appearance then the first avatar, wherein the second avatar audibly presents the respective challenge presented via the reading and watching user interfaces, wherein the user is to verbally provide a response to the respective challenge, the response including each of the key elements presented via the watching user interface and the reading user interface for that respective challenge;

a review user interface configured and arranged to include at least a textual representation of key elements previously presented via the watching user interface;

in response to at least one user action, transmitting for presentation on the user terminal user interfaces associated with a scored challenge session configured to test the user with respect to the challenges included in the module, wherein for a given challenge on which the user is to be tested the scored challenge user interface includes;

an avatar audibly presenting a selected challenge from the plurality of challenges, wherein the avatar has lip motions at least substantially synchronized with the audible selected challenge, and wherein the user is to audibly respond to the selected challenge by at least presenting corresponding key elements;

a scoring interface for the selected challenge, the scoring interface configured to receive and/or provide at least the following scoring information;

 

how accurately the user audibly, verbally presented the key elements corresponding to the selected challenge audibly presented by the avatar;

 

how confident the user seemed when audibly, verbally responding to the selected challenge audibly presented by the avatar.

2. The system as defined in claim 1, wherein the system is configured to provide an avatar design interface that enables the user to provide an avatar specification, the avatar specification specifying an appearance of at least one avatar, avatar clothing, skin color, a background to be displayed behind the avatar, and audio content to be audibly presented by the avatar, and in response to receiving the avatar specification via the avatar design interface, generating a scene including the specified avatar, avatar clothing, skin color, background, and audio content.

3. The system as defined in claim 1, wherein the system is further configured to use a formula to generate a cumulative score using at least the following for a given challenge response:

a score related to the correctness and/or completeness of the challenge response;

a score related to how quickly the user initiated the challenge response; and

a score related to the confidence and/or style with which the user verbalized the challenge response.

4. The system as defined in claim 1, wherein the system is configured to inhibit scoring when the performing user interface is displayed.

5. The system as defined in claim 1, wherein the system is configured to size and position the plurality of guideline language constructs displayed via the reading user interface so that they are displayed at the same time.

6. The system as defined in claim 1, wherein the system further comprises a voice recognition system configured to detect when the user begins responding to a given challenge, halt a response timer in response to detecting that the user has begun responding, and perform voice recognition of user audible, verbal responses and to bold, illuminate, and/or emphasize respective key elements the user correctly stated and/or respective key elements the user incorrectly stated or did not state at all.

7. The system as defined in claim 1, wherein the system further comprises a voice recognition system configured to detect when the user begins responding and convert the user'"'"'s verbal challenge responses to machine readable characters, compare the machine readable characters to text corresponding to a known correct, reference version of the key elements, and determine which key elements the user correctly recited.

8. A computer implemented method, the method comprising:

receiving an identification of a training module, the module including at least a first set of challenges including a plurality of challenges that include a statement or question regarding a subject that a user is to be trained to verbally respond to;

for a respective challenge, in response to a user action,

transmitting, for presentation on a user terminal having a display, user interfaces associated with a learning session and user interfaces associated with a practice session, wherein;

the learning session includes a reading section configured to train the user, using text, in how to respond to the respective challenge, and a watching section, configured to train the user to respond to the respective challenge using an audio video presentation,

wherein the reading section includes a reading user interface configured to present via text;

the respective challenge;

a plurality of guideline language constructs that provide a model answer to the respective challenge, wherein a given guideline language construct includes;

a key element; and

contextual language in which the key element is embedded, wherein the key element are visually distinguished via an attribute not present in the contextual language;

wherein the watching section includes a watching user interface configured to present;

a textual representation of the key elements previously presented via the reading user interface, wherein the key elements are not embedded in the guideline language constructs;

an audio video presentation of a first animated avatar audibly presenting the guideline language constructs, including the key elements, presented via the reading user interface and/or audibly presenting the key elements without the guideline language constructs, wherein the first animated avatar'"'"'s lips, facial expression, and gestures are animated to provide a simulation of a human while being visually distinguishable as an animation rather than a real person and wherein the first avatar has lip motions at least substantially synchronized with the audible key elements;

wherein the practice session user interfaces include;

a performing user interface, wherein the performing user interface includes an audio video presentation of a second avatar, wherein the second avatar has a different appearance then the first avatar, wherein the second avatar audibly presents the respective challenge presented via the reading and watching user interfaces, wherein the user is to verbally provide a response to the respective challenge, the response including each of the key elements presented via the watching user interface and the reading user interface for that respective challenge;

a review user interface configured and arranged to include at least a textual representation of key elements previously presented via the watching user interface;

in response to at least one user action, transmitting for presentation on the user terminal user interfaces associated with a scored challenge session configured to test the user with respect to the challenges included in the module, wherein for a given challenge on which the user is to be tested the scored challenge user interface includes;

an avatar audibly presenting a selected challenge from the plurality of challenges, wherein the avatar has lip motions at least substantially synchronized with the audible selected challenge, and wherein the user is to audibly respond to the selected challenge by at least presenting corresponding key elements;

a scoring interface for the selected challenge, the scoring interface configured to receive and/or provide at least the following scoring information;

how accurately the user audibly, verbally presented the key elements corresponding to the selected challenge audibly presented by the avatar;

how confident the user seemed when audibly, verbally responding to the selected challenge audibly presented by the avatar.

9. The method as defined in claim 8, the method further comprising providing an avatar design interface that enables the user to provide an avatar specification, the avatar specification specifying an appearance of at least one avatar, avatar clothing, skin color, a background to be displayed behind the avatar, and audio content to be audibly presented by the avatar, and in response to receiving the avatar specification via the avatar design interface, generating a scene including the specified avatar, avatar clothing, skin color, background, and audio content.

10. The method as defined in claim 8, the method further comprising using a formula to generate a cumulative score using at least the following for a given challenge response:

a score related to the correctness and/or completeness of the challenge response;

a score related to how quickly the user initiated the challenge response; and

a score related to the confidence and/or style with which the user verbalized the challenge response.

11. The method as defined in claim 8, the method further comprising inhibiting scoring when the performing user interface is displayed.

12. The method as defined in claim 8, the method further comprising causing the plurality of guideline language constructs displayed via the reading user interface to be sized and positioned so that they are displayed at the same time.

13. The method as defined in claim 8, the method further comprising:

detecting when the user begins responding to a given challenge;

halting a response timer in response to detecting that the user has begun responding;

performing voice recognition of user audible, verbal responses; and

bolding, illuminating, and/or emphasizing respective key elements the user correctly stated and/or respective key elements the user incorrectly stated or did not state at all.

14. The method as defined in claim 8, the method further comprising:

detecting when the user begins responding to a given challenge;

converting the user'"'"'s verbal challenge responses to machine readable characters;

comparing the machine readable characters to text corresponding to a known correct, reference version of corresponding key elements;

determining which key elements the user correctly recited; and

generating at least one score based at least in part on the determination as to which key elements the user correctly recited.

15. A tangible, non-transitory, computer-readable medium having computer-executable instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:

receiving an identification of a training module, the module including at least a first set of challenges including a plurality of challenges that include a statement or question regarding a subject that a user is to be trained to verbally respond to;

for a respective challenge, in response to a user action,

transmitting, for presentation on a user terminal having a display, user interfaces associated with a learning session and user interfaces associated with a practice session, wherein;

the learning session includes a reading section configured to train the user, using text, in how to respond to the respective challenge, and/or a watching section, configured to train the user to respond to the respective challenge using an audio video presentation,

wherein the reading section includes a reading user interface configured to present via text;

the respective challenge;

a plurality of guideline language constructs that provide a model answer to the respective challenge, wherein a given guideline language construct includes;

a key element; and

contextual language in which the key element is embedded, wherein the key element are visually distinguished via an attribute not present in the contextual language;

wherein the watching section includes a watching user interface configured to present;

a textual representation of the key elements previously presented via the reading user interface, wherein the key elements are not embedded in the guideline language constructs;

an audio video presentation of a first animated avatar audibly presenting the guideline language constructs, including the key elements, presented via the reading user interface and/or audibly presenting the key elements without the guideline language constructs, wherein the first animated avatar'"'"'s lips, facial expression, and gestures are animated to provide a simulation of a human while being visually distinguishable as an animation rather than a real person and wherein the first avatar has lip motions at least substantially synchronized with the audible key elements;

wherein the practice session user interfaces include;

a performing user interface, wherein the performing user interface includes an audio video presentation of the respective challenge presented via the reading and watching user interfaces, wherein the user is to verbally provide a response to the respective challenge, the response including each of the key elements presented via the watching user interface and the reading user interface for that respective challenge;

a review user interface configured and arranged to include at least a textual representation of key elements previously presented via the watching user interface;

in response to at least one user action, transmitting for presentation on the user terminal user interfaces associated with a scored challenge session configured to test the user with respect to the challenges included in the module, wherein for a given challenge on which the user is to be tested the scored challenge user interface includes;

an avatar audibly presenting a selected challenge from the plurality of challenges, wherein the avatar has lip motions at least substantially synchronized with the audible selected challenge, and wherein the user is to audibly respond to the selected challenge by at least presenting corresponding key elements;

a scoring interface for the selected challenge, the scoring interface configured to receive and/or provide at least the following scoring information;

how accurately the user audibly presented the key elements corresponding to the selected challenge audibly presented by the avatar;

how confident the user seemed when verbally responding to

the selected challenge audibly presented by the avatar.

16. The computer-readable medium as defined in claim 15, the operations further comprising providing an avatar design interface that enables the user to provide an avatar specification, the avatar specification specifying an appearance of at least one avatar, avatar clothing, skin color, and audio content to be audibly presented by the avatar, and in response to receiving the avatar specification via the avatar design interface, generating a scene including the specified avatar, avatar clothing, skin color, and audio content.

17. The computer-readable medium as defined in claim 15, the operations further comprising using a formula to generate a cumulative score using at least the following for a given challenge response:

a score related to the correctness and/or completeness of the challenge response;

a score related to how quickly the user initiated the challenge response; and

a score related to the confidence and/or style with which the user verbalized the challenge response.

18. The computer-readable medium as defined in claim 15, the operations further comprising:

detecting when the user begins responding to a given challenge;

halting a response timer in response to detecting that the user has begun responding;

performing voice recognition of user audible, verbal responses; and

bolding, illuminating, and/or emphasizing respective key elements the user correctly audibly verbalized.