INTERACTIVE STORY SYSTEM USING FOUR-VALUED LOGIC

• US 20170243517A1
• Filed: 05/05/2017
• Published: 08/24/2017
• Est. Priority Date: 09/03/2013
• Status: Active Grant

• Current Assignee: Roger Midmore
• Original Assignee: Roger Midmore
• Inventors: Midmore, Roger
• Granted Patent: US 10,706,741 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems

Abstract

A language learning system that teaches an individualized set of vocabulary words to users through an interactive story. The interactive story is modeled through probabilistic rules in a semantic network having objects and relations. Dialog and narration is generated dynamically based on the state of the interactive story model using phrasal rewrite rules evaluated with a four-valued logic system in which truth values of the objects and relations are encoded as true, false, defined, and undefined in parallel memory structures.

Claims

1. A language teaching process, comprising:

initializing a semantic network modeling an interactive story using triples having one object and one relation, two objects and one relation, or one object and two relations, wherein truth values associated with each object and relation are expressed using a four-valued logic system allowing true values, false values, defined values, and undefined values,

defining a set of phrasal rewrite rules that accept an input triple as an argument;

applying a particular phrasal rewrite rule to a particular argument when;

elements expected to be true by said phrasal rewrite rule are set to true in said argument,

elements expected to be false by said phrasal rewrite rule are set to false in said argument, and

elements associated with horizontal features are encoded as expected in said argument,

substituting syntactic units from said particular phrasal rewrite rule'"'"'s right hand side for its left hand side when said phrasal rewrite rule is applied, inheriting elements of its argument to the syntactic units of the right hand side, and distributing features of inherited arguments from designated syntactic units to other syntactic units on the right hand side;

generating a sentence in a target language by selecting words from a vocabulary list to express syntactic units generated by applied phrasal rewrite rules; and

presenting said sentence audibly and/or visually to a user.

2. The language teaching process of claim 1, further comprising displaying visuals representing the state of said interactive story via a 3D projector.

3. The language teaching process of claim 1, further comprising updating the state of said objects and/or relations in said semantic network over time through probabilistic rules.

4. The language teaching process of claim 1, wherein truth values associated with each object and/or relation are encoded using two memory structures, with values in corresponding bit positions in both memory structures indicating an encoded true value, an encoded false value, an encoded defined value, or an encoded undefined value.

5. The language teaching process of claim 4, wherein whether or not a phrasal rewrite rule is to be applied is determined in constant O(c) time by testing conditionals through truth values encoded in the two memory structures associated with the elements of the argument.

6. The language teaching process of claim 1, further comprising:

generating a vocabulary graph based one or more source texts using the Good-Turing Frequency Estimation and random partitioning;

subdividing said vocabulary graph into a target vocabulary area;

generating said vocabulary list based on words in said target vocabulary area; and

defining or modifying said set of phrasal rewrite rules based on grammar forms in said target vocabulary area.

7. The language teaching process of claim 1, further comprising:

receiving a voice recording of a user captured by a microphone;

recognizing spoken words in said recorded user input according to a Markov model;

determining a state of said interactive story relevant to the recognized spoken words;

generating a response to said spoken words with said phrasal rewrite rules; and

presenting said response to the user.

8. The language teaching process of claim 7, further comprising:

displaying visuals representing the state of said interactive story;

receiving an image of a user captured by a camera;

recognizing a gesture performed by the user in said image;

determining a designated area of said visuals pointed to by the user'"'"'s gesture, and using visuals at said designated area to determine what object the user'"'"'s spoken words referred to in said semantic network when generating a response to said spoken words.

9. A language learning system, comprising:

a blackboard memory area for central control and command, the blackboard memory area storing data representing;

a semantic network that models an interactive story;

a four-valued logic system allowing true values, false values, defined values, and undefined values; and

a list of phrasal rewrite rules;

an audio generation module linked to said blackboard memory area, said audio generation module being configured to generate story audio based on said phrasal rewrite rules that match the state of said interactive story and add said story audio to said blackboard memory area;

a speaker linked to said blackboard memory area, said speaker being configured to play said story audio from said blackboard memory area for a user;

a visual generation module linked to said blackboard memory area, said visual generation module being configured to render story visuals representing the state of said interactive story and add said story visuals to said blackboard memory area;

a display component linked to said blackboard memory area, said projector being configured to present said story visuals from said blackboard memory area for the user,

wherein said semantic network models said interactive story through objects and relations, with associations between objects and relations being represented with 2-element triples having one object and one relation or 3-element triples having either two objects and one relation or one object and two relations,

wherein the state of said interactive story is modeled over time through probabilistic rules that change the state of said objects and/or said relations, and

wherein whether or not particular phrasal rewrite rules apply based on a particular input triple is determined using said four-valued logic system.

10. The language learning system of claim 9, wherein said display is a 3D projector.

11. The language learning system of claim 9, wherein truth values associated with each object and/or relation are encoded using two memory structures, with values in corresponding bit positions in both memory structures indicating an encoded true value, an encoded false value, an encoded defined value, or an encoded undefined value.

12. The language learning system of claim 9, wherein one or more of said phrasal rewrite rules are applied to said input triple to generate a sentence in a target language that expresses the state of said input triple in said semantic network.

13. The language learning system of claim 12, wherein said phrasal rewrite rules and words selected for said sentence are based on a target vocabulary area subdivided from a vocabulary graph generated from one or more source texts using the Good-Turing Frequency Estimation and random partitioning.

14. The language learning system of claim 9, further comprising:

a microphone linked to said blackboard memory area, said microphone being configured to add recorded user input to said blackboard memory area; and

an audio recognition module linked to said blackboard memory area, said audio recognition module being configured to recognize spoken words in said recorded user input according to a Markov model and adds recognized words to said blackboard memory area.

15. The language learning system of claim 14, wherein said audio generation module uses said phrasal rewrite rules to generate a sentence in response to spoken words recognized in said recorded user input by said audio recognition module, and adds the generated sentence to said blackboard memory area to played with said speaker.

16. The language learning system of claim 14, further comprising:

a camera linked to said blackboard memory area, said camera being configured to add images of the user to said blackboard memory area; and

a visual recognition module linked to said blackboard memory area, said visual recognition module being configured to recognize gestures pointing to portions of said story visuals being displayed by said display component.