System and Method for Dynamic Response to User Interaction

• US 20170076626A1
• Filed: 09/14/2015
• Published: 03/16/2017
• Est. Priority Date: 09/14/2015
• Status: Abandoned Application

• Current Assignee: Seashells Education Software, Inc.
• Original Assignee: Seashells Education Software, Inc.
• Inventors: Lam, Melanie Jing Yee, Aist, Gregory, Cano, Rodrigo, Gupta, Umang
• Application Number:
  US14/853,054
• Publication Number:
  US 20170076626A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G09B 17/003 electrically operated apparatus or devices
• CPC Class Codes: G09B 17/006 with audible presentation of the material to be studied audio and audio-visual teaching apparatus in general G09B5/04, G09B5/06
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A computing device, having a user input interface including an input button, processes a sentence array including component words of a target sentence string to determine at least one target word to be read by the user. The computing device detects press and hold by the user of the input button, and in direct response, receives and processes user speech input to recognize at least one spoken word, and upon recognizing the at least one spoken word, determines whether the user has correctly read the at least one target word. The computing device also detects release by the user of the input button, and in direct response, identifies a context position relative to the target sentence string, and processes at least one predefined action based on the identified context position.

Claims

1. A method for providing a dynamic response to user interaction, comprising:

at a computing device with a user input interface including an input button;

processing a sentence array including component words of a target sentence string to determine at least one target word to be read by the user, the determined at least one target word defining a context position relative to the target sentence string;

detecting press and hold by the user of the input button, and in direct response to detecting the press and hold by the user of the input button;

receiving user speech input;

processing the user speech input to recognize in the user speech input at least one spoken word; and

upon recognizing the at least one spoken word, determining whether the user has correctly read the at least one target word; and

detecting release by the user of the input button, and in direct response to detecting the release by the user of the input button;

identifying the context position relative to the target sentence string; and

processing at least one predefined action based on the identified context position.

2. The method of claim 1, further comprising outputting, to a display of the computing device, the target sentence string and an indication of the at least one target words to be read by the user.

3. The method of claim 2, wherein the at least one predefined action further comprises outputting audio, visual and/or tactile indication associated with the one or more target words to be read by the user.

4. The method of claim 3, wherein the at least one predefined action comprises retrieving and outputting an audible representation of said target word.

5. The method of claim 4, wherein the audible representation is retrieved from a database.

6. The method of claim 3, wherein the at least one predefined action further comprises processing the sentence array to determine a subsequent at least one target word to be read by the user.

7. The method of claim 3, wherein the at least one predefined action further comprises sending a notification to a processing module of the computing device.

8. The method of claim 1, further comprising detecting a plurality of releases by the user of the input button while at the same context position, and in direct response to each subsequent release by the user of the input button, processing a respective predefined action to output one of a series of escalating assistance.

9. The method of claim 1, wherein the at least one predefined action is further based on the user'"'"'s age or experience level.

10. The method of claim 8 or 9, wherein at least one predefined action comprises retrieving and outputting one of a set of audible representations of said target word.

11. The method of claim 8 or 9, wherein at least one predefined action comprises retrieving and outputting an audible and/or visual version of at least one target word.

12. The method of claim 3, wherein determining whether the user has correctly read the at least one target word includes calculating a match score associated with the determination, and wherein the output indication is based on the calculated match score.

13. The method of claim 1, wherein a different action is predefined for respective ones of a plurality of context positions.

14. The method of claim 1, wherein the context position is defined relative to one of the start and end of the target sentence string.

15. The method of claim 14, wherein the context position defined relative to the end of the target sentence string is associated with a predefined action to retrieve a subsequent sentence array including component words of another target sentence string.

16. The method of claim 14, wherein the context position defined relative to the end of the target sentence string is further associated with a predefined action to calculate and generate dynamic feedback based on the processing of user speech input to recognize the component words of the target sentence string.

17. The method of claim 1, wherein the user input interface is a touch screen display.

18. A system for providing a dynamic response to user interaction, comprising:

a user input interface including an input button;

one or more processors configured to;

detect press and hold by the user of the input button, and in direct response to detecting the press and hold by the user of the input button;

receive user speech input, process the user speech input to recognize in the user speech input at least one spoken word, and determine based on the recognized at least one spoken word that the user has correctly read at least one target word of a target sentence string; and

detect subsequent release by the user of the input button, and in direct response to detecting the release by the user of the input button;

identify a context position relative to the target sentence string defined by the at least one target word, and process a predefined action based on the identified context position.

19. A non-transitory computer-readable medium comprising computer-executable instructions, that when executed by a computing device perform the method of:

receiving a sentence array including component words of a target sentence string;

processing the sentence array to determine at least one target word to be read by the user, defining a context position relative to the target sentence string;

detecting press and hold by the user of the input button, and in direct response to detecting the press and hold by the user of the input button;

receiving user speech input;

processing the user speech input to recognize in the user speech input at least one spoken word; and

upon recognizing the at least one spoken word, determining whether the user has correctly read the at least one target word; and

detecting release by the user of the input button, and in direct response to detecting the release by the user of the input button;

identifying the context position relative to the target sentence string; and

processing at least one predefined action based on the identified context position.