Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling
First Claim
1. An electronic device, comprising:
- one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;
receiving a current word;
determining a context of the current word based on the current word and a context of a previous word;
determining, using a morpheme-based language model, a first representation indicating a likelihood of each prefix of a predetermined set of prefixes, wherein the likelihood of each prefix is determined based on the context of the current word;
determining, using the morpheme-based language model, a second representation indicating a likelihood of each stem of a predetermined set of stems, wherein the likelihood of each stem is determined based on the context of the current word;
determining, using the morpheme-based language model, a third representation indicating a likelihood of each suffix of a predetermined set of suffixes, wherein the likelihood of each suffix is determined based on the context of the current word;
determining a next word based on the first representation, the second representation, and the third representation; and
providing an output including the next word.
1 Assignment
0 Petitions
Accused Products
Abstract
The present disclosure generally relates to systems and processes for morpheme-based word prediction. An example method includes receiving a current word; determining a context of the current word based on the current word and a context of a previous word; determining, using a morpheme-based language model, a likelihood of a prefix based on the context of the current word; determining, using the morpheme-based language model, a likelihood of a stem based on the context of the current word; determining, using the morpheme-based language model, a likelihood of a suffix based on the context of the current word; determining a next word based on the likelihood of the prefix, the likelihood of the stem, and the likelihood of the suffix; and providing an output including the next word.
2408 Citations
20 Claims
-
1. An electronic device, comprising:
-
one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for; receiving a current word; determining a context of the current word based on the current word and a context of a previous word; determining, using a morpheme-based language model, a first representation indicating a likelihood of each prefix of a predetermined set of prefixes, wherein the likelihood of each prefix is determined based on the context of the current word; determining, using the morpheme-based language model, a second representation indicating a likelihood of each stem of a predetermined set of stems, wherein the likelihood of each stem is determined based on the context of the current word; determining, using the morpheme-based language model, a third representation indicating a likelihood of each suffix of a predetermined set of suffixes, wherein the likelihood of each suffix is determined based on the context of the current word; determining a next word based on the first representation, the second representation, and the third representation; and providing an output including the next word. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:
-
receive a current word; determine a context of the current word based on the current word and a context of a previous word; determine, using a morpheme-based language model, a first representation indicating a likelihood of each prefix of a predetermined set of prefixes, wherein the likelihood of each prefix is determined based on the context of the current word; determine, using the morpheme-based language model, a second representation indicating a likelihood of each stem of a predetermined set of stems, wherein the likelihood of each stem is determined based on the context of the current word; determine, using the morpheme-based language model, a third representation indicating a likelihood of each suffix of a predetermined set of suffixes, wherein the likelihood of each suffix is determined based on the context of the current word; determine a next word based on the first representation, the second representation, and the third representation; and providing an output including the next word. - View Dependent Claims (12, 13, 14, 15)
-
-
16. A method, comprising:
at an electronic device having one or more processors; receiving a current word; determining a context of the current word based on the current word and a context of a previous word; determining, using a morpheme-based language model, a first representation indicating a likelihood of each prefix of a predetermined set of prefixes, wherein the likelihood of each prefix is determined based on the context of the current word; determining, using the morpheme-based language model, a second representation indicating a likelihood of each stem of a predetermined set of stems, wherein the likelihood of each stem is determined based on the context of the current word; determining, using the morpheme-based language model, a third representation indicating a likelihood of each suffix of a predetermined set of suffixes, wherein the likelihood of each suffix is determined based on the context of the current word; determining a next word based on the first representation, the second representation, and the third representation; and providing an output including the next word. - View Dependent Claims (17, 18, 19, 20)
Specification