Adaptive learning system using automatically-rated problems and pupils

• US 10,720,072 B2
• Filed: 02/17/2017
• Issued: 07/21/2020
• Est. Priority Date: 02/19/2016
• Status: Active Grant

• Current Assignee: Expii, Inc.
• Original Assignee: Expii, Inc.
• Inventors: Loh, Po-Shen, Hamilton, Linus Ulysses, Li, Ray Yu
• Primary Examiner(s): Utama, Robert J
• Application Number:
  US15/435,930
• Publication Number:
  US 20170243502A1
• Time in Patent Office: 1,250 Days
• CPC Class Codes: G09B 17/003 electrically operated apparatus or devices
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Abstract

Implementations for adaptive learning include actions of: outputting a practice problem having a problem difficulty rating, in which the practice problem is identified by one or more concept tags, each concept tag relating to a different subject-matter area that forms a basis of the practice problem; receiving an answer from a user to the practice problem; responsive to receiving the answer, determining whether the answer includes a correct or incorrect response to the practice problem; responsive to determining whether the answer includes a correct or incorrect response, updating, for each of the different subject-matter areas, a corresponding user specific comprehension rating and the problem difficulty rating of the practice problem, based on whether the answer includes a correct or incorrect response; and outputting, for at least one of the different subject areas that forms the basis of the practice problem, the updated user specific comprehension rating.

Claims

1. A computer-implemented method for adaptive learning, the method being executed using one or more processors and comprising:

outputting, from the one or more processors executing an interactive learning program of an online learning system, a practice problem having a current problem difficulty rating to an interactive graphical user interface, wherein the practice problem is identified by one or more concept tags, each concept tag relating to a different subject-matter area that forms a basis of the practice problem, wherein the current problem difficulty rating is derived based on whether one or more users previously answered the practice problem correctly or incorrectly;

receiving, by the one or more processors and from the interactive graphical user interface, an answer from a first user to the practice problem;

responsive to receiving the answer, determining whether the answer includes a correct or incorrect response to the practice problem;

responsive to determining whether the answer includes a correct or incorrect response,

automatically updating in real-time, for each of the different subject-matter areas that forms a basis of the practice problem, a corresponding user specific comprehension rating for the first user based on the current problem difficulty rating to provide an updated user specific comprehension ratings, such that the updated user specific comprehension rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem, and

automatically updating in real-time, for each of the different subject-matter areas that forms the basis of the practice problem, the current problem difficulty rating of the practice problem to provide an updated problem difficulty rating, such that the updated problem difficulty rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem,

wherein a magnitude of the changes to the user specific comprehension rating and to the current problem difficulty rating depends on the user specific comprehension rating prior to receiving the answer, the current problem difficulty rating prior to receiving the answer, and whether the answer includes a correct or incorrect response;

outputting, for at least one of the different subject-matter areas that forms the basis of the practice problem, the updated user specific comprehension rating to the interactive graphical user interface; and

subsequent to automatically updating the corresponding user specific comprehension rating for each of the different subject-matter areas,

automatically outputting a new practice problem to the interactive graphical user interface based on matching at least one of the updated user specific comprehension ratings to a problem difficulty rating of the new practice problem or to within a predetermined range of the problem difficulty rating of the new practice problem.

2. The method of claim 1, further comprising:

prior to outputting the practice problem, identifying a potential practice problem having a problem difficulty rating within a predetermined range of at least one user specific comprehension rating for a specified user; and

selecting the identified potential practice problem as the practice problem to output.

3. The method of claim 2, wherein identifying the potential practice problem comprises identifying a problem that the specified user has not previously attempted.

4. The method of claim 2, wherein identifying the potential practice problem comprises identifying a problem associated with at least one concept tag that corresponds to a subject-matter area selected by the specified user.

5. The method of claim 1, wherein each user specific comprehension rating is unbounded, and wherein each user specific comprehension rating is an indicator of the specified user'"'"'s comprehension of the corresponding subject-matter area relative to the comprehension of the corresponding subject-matter area by one or more other users.

6. The method of claim 1, wherein determining whether the answer includes a correct or incorrect response to the practice problem comprises determining whether the answer includes a correct response on the user'"'"'s first attempt to solve the problem.

7. The method of claim 1, wherein updating the corresponding user specific comprehension rating and updating the current problem difficulty rating of the practice problem comprise determining whether the user previously attempted to solve more or less than a predetermined number of practice problems, each of which is associated with at least one of the one or more concept tags.

8. The method of claim 1, wherein each practice problem is identified by a plurality of concept tags, and wherein updating the corresponding user specific comprehension rating and updating the current problem difficulty rating of the practice problem comprise determining whether the user previously attempted to solve more or less than a predetermined number of practice problems, each of which is associated with the plurality of concept tags.

28. The method of claim 1, wherein updating the current problem difficulty rating comprises:

deriving an expected result value for each concept tag to provide a set of expected results; and

modifying the current problem difficulty rating based, in part, on all of the set of expected results to provide the updated problem difficulty rating.

29. The method of claim 1, wherein updating the user specific comprehension rating for each of the different subject-matter areas comprises:

deriving an expected result value for each concept tag to provide a set of expected results; and

for each of the different subject-matter areas, modifying the corresponding user specific comprehension rating based, in part, on the set of expected results.

30. The method of claim 1, wherein updating, for each of the different subject-matter areas that forms the basis of the practice problem, the corresponding user specific comprehension rating and the current problem difficulty rating of the practice problem is performed in real-time.

9. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for adaptive learning, the operations comprising:

outputting, from the one or more processors executing an interactive learning program of an online learning system, a practice problem having a current problem difficulty rating to an interactive graphical user interface, wherein the practice problem is identified by one or more concept tags, each concept tag relating to a different subject-matter area that forms a basis of the practice problem, wherein the current problem difficulty rating is derived based on whether one or more users previously answered the practice problem correctly or incorrectly;

receiving, by the one or more processors and from the interactive graphical user interface, an answer from a first user to the practice problem;

responsive to receiving the answer, determining whether the answer includes a correct or incorrect response to the practice problem;

responsive to determining whether the answer includes a correct or incorrect response, automatically updating in real-time, for each of the different subject-matter areas that forms a basis of the practice problem, a corresponding user specific comprehension rating for the first user based on the current problem difficulty rating to provide an updated user specific comprehension rating, such that the updated user specific comprehension rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem, and

automatically updating in real-time, for each of the different subject-matter areas that forms the basis of the practice problem, the current problem difficulty rating of the practice problem to provide an updated problem difficulty rating, such that the updated problem difficulty rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem,

wherein a magnitude of the changes to the user specific comprehension rating and to the current problem difficulty rating depends on the user specific comprehension rating prior to receiving the answer, the current problem difficulty rating prior to receiving the answer, and whether the answer includes a correct or incorrect response;

outputting, for at least one of the different subject-matter areas that forms the basis of the practice problem, the updated user specific comprehension rating to the interactive graphical user interface; and

subsequent to automatically updating the corresponding user specific comprehension rating for each of the different subject-matter areas,

automatically outputting a new practice problem to the interactive graphical user interface based on matching at least one of the updated user specific comprehension ratings to a problem difficulty rating of the new practice problem or to within a predetermined range of the problem difficulty rating of the new practice problem.

10. The computer-readable storage medium of claim 9, wherein operations further comprise:

prior to outputting the practice problem, identifying a potential practice problem having a problem difficulty rating within a predetermined range of at least one user specific comprehension rating for a specified user; and

selecting the identified potential practice problem as the practice problem to output.

11. The computer-readable storage medium of claim 10, wherein identifying the potential practice problem comprises identifying a problem that the specified user has not previously attempted.

12. The computer-readable storage medium of claim 10, wherein identifying the potential practice problem comprises identifying a problem associated with at least one concept tag that corresponds to a subject-matter area selected by the specified user.

13. The computer-readable storage medium of claim 10, wherein each user specific comprehension rating is unbounded, and wherein each user specific comprehension rating is indicative of the specified user'"'"'s comprehension of the corresponding concept relative to a plurality of other users.

14. The computer-readable storage medium of claim 9, wherein determining whether the answer includes a correct or incorrect response to the practice problem comprises determining whether the answer includes a correct response on the user'"'"'s first attempt to solve the problem.

15. The computer-readable storage medium of claim 9, wherein updating the corresponding user specific comprehension rating and updating the current problem difficulty rating of the practice problem comprise determining whether the user previously attempted to solve more or less than a predetermined number of practice problems, each of which is associated with at least one of the one or more concept tags.

16. The computer-readable storage medium of claim 9, wherein each practice problem is identified by a plurality of concept tags, and wherein updating the corresponding user specific comprehension rating and updating the current problem difficulty rating of the practice problem comprise determining whether the user previously attempted to solve more or less than a predetermined number of practice problems, each of which is associated with the plurality of concept tags.

17. A system, comprising:

a computing device; and

a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by one or more processors of the computing device, cause the computing device to perform operations for adaptive learning, the operations comprising;

executing an interactive learning program of an online learning system;

outputting, to the interactive learning program, a practice problem having a current problem difficulty rating to an interactive graphical user interface, wherein the practice problem is identified by one or more concept tags, each concept tag relating to a different subject-matter area that forms a basis of the practice problem, wherein the current problem difficulty rating is derived based on whether one or more users previously answered the practice problem correctly or incorrectly;

receiving from the interactive graphical user interface an answer from a first user to the practice problem;

responsive to receiving the answer, determining whether the answer includes a correct or incorrect response to the practice problem;

responsive to determining whether the answer includes a correct or incorrect response, automatically updating in real-time, for each of the different subject-matter areas that forms a basis of the practice problem, a corresponding user specific comprehension rating for the first user based on the current problem difficulty rating to provide an updated user specific comprehension rating, such that the updated user specific comprehension rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem, and

automatically updating in real-time, for each of the different subject-matter areas that forms the basis of the practice problem, the current problem difficulty rating of the practice problem to provide an updated problem difficulty rating, such that the updated problem difficulty rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem,

wherein a magnitude of the changes to the user specific comprehension rating and to the current problem difficulty rating depends on the user specific comprehension rating prior to receiving the answer, the problem difficulty rating prior to receiving the answer, and whether the answer includes a correct or incorrect response;

outputting, for at least one of the different subject-matter areas that forms the basis of the practice problem, the updated user specific comprehension rating to the interactive graphical user interface; and

subsequent to automatically updating the corresponding user specific comprehension rating for each of the different subject-matter areas,

automatically outputting a new practice problem to the interactive graphical user interface based on matching at least one of the updated user specific comprehension ratings to a problem difficulty rating of the new practice problem or to within a predetermined range of the problem difficulty rating of the new practice problem.

18. The system of claim 17, wherein operations further comprise:

prior to outputting the practice problem, identifying a potential practice problem having a problem difficulty rating within a predetermined range of at least one user specific comprehension rating for a specified user; and

selecting the identified potential practice problem as the practice problem to output.

19. The system of claim 18, wherein identifying the potential practice problem comprises identifying a problem that the specified user has not previously attempted.

20. The system of claim 18, wherein identifying the potential practice problem comprises identifying a problem associated with at least one concept tag that corresponds to a subject-matter area selected by the specified user.

21. The system of claim 17, wherein each user specific comprehension rating is unbounded, and wherein each user specific comprehension rating is indicative of the specified user'"'"'s comprehension of the corresponding concept relative to a plurality of other users.

22. The system of claim 17, wherein determining whether the answer includes a correct or incorrect response to the practice problem comprises determining whether the answer includes a correct response on the user'"'"'s first attempt to solve the problem.

23. The system of claim 17, wherein updating the corresponding user specific comprehension rating and updating the current problem difficulty rating of the practice problem comprise determining whether the user previously attempted to solve more or less than a predetermined number of practice problems, each of which is associated with at least one of the one or more concept tags.

24. The system of claim 17, wherein each practice problem is identified by a plurality of concept tags, and wherein updating the corresponding user specific comprehension rating and updating the current problem difficulty rating of the practice problem comprise determining whether the user previously attempted to solve more or less than a predetermined number of practice problems, each of which is associated with the plurality of concept tags.

25. A computer-implemented method for adaptive learning, the method being executed using one or more processors and comprising:

receiving, over a network, from a client device, an identification by a first user of at least one subject-matter area to review;

responsive to receiving the identification, identifying, by the one or more processors, a potential practice problem having a current problem difficulty rating within a predetermined range of at least one user specific comprehension rating for the first user, wherein the practice problem is identified by one or more concept tags, each concept tag relating to a different subject-matter area that forms a basis of the practice problem, wherein the current problem difficulty rating is derived based on whether one or more users previously answered the practice problem correctly or incorrectly;

transmitting, from the one or more processors, the identified practice problem to the client device over the network;

receiving, over the network, from the client device, an answer to the practice problem;

responsive to receiving the answer, determining, by the one or more processors, whether the answer includes a correct or incorrect response to the practice problem;

responsive to determining whether the answer includes a correct or incorrect response, automatically updating, for each of the different subject-matter areas that forms a basis of the practice problem, a corresponding user specific comprehension rating to provide an updated user specific comprehension rating that is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem, and automatically updating the current problem difficulty rating of the practice problem to provide an updated problem difficulty rating, such that the updated problem difficulty rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem,

wherein a magnitude of the changes to the user specific comprehension rating and the current problem difficulty rating depends on the user specific comprehension rating prior to receiving the answer, the current problem difficulty rating prior to receiving the answer, and whether the answer includes a correct or incorrect response;

outputting, for at least one of the different subject-matter areas that forms the basis of the practice problem, the updated user specific comprehension rating to an interactive graphical user interface; and

subsequent to automatically updating the corresponding user specific comprehension rating for each of the different subject-matter areas,

automatically outputting a new practice problem to the interactive graphical user interface based on matching at least one of the updated user specific comprehension ratings to a problem difficulty rating of the new practice problem or to within a predetermined range of the problem difficulty rating of the new practice problem.

26. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for adaptive learning, the operations comprising:

receiving, over a network, from a client device, an identification by a first user of at least one subject-matter area to review;

responsive to receiving the identification, identifying, by the one or more processors, a potential practice problem having a current problem difficulty rating within a predetermined range of at least one user specific comprehension rating for the first user, wherein the practice problem is identified by one or more concept tags, each concept tag relating to a different subject-matter area that forms a basis of the practice problem, wherein the current problem difficulty rating is derived based on whether one or more users previously answered the practice problem correctly or incorrectly;

transmitting, from the one or more processors, the identified practice problem to the client device over the network;

receiving, over the network, from the client device, an answer to the practice problem;

responsive to receiving the answer, determining, by the one or more processors, whether the answer includes a correct or incorrect response to the practice problem;

responsive to determining whether the answer includes a correct or incorrect response, automatically updating, for each of the different subject-matter areas that forms a basis of the practice problem, a corresponding user specific comprehension rating to provide an updated user specific comprehension rating that is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem, and automatically updating the current problem difficulty rating of the practice problem to provide an updated problem difficulty rating, such that the updated problem difficulty rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem,

wherein a magnitude of the changes to the user specific comprehension rating and the current problem difficulty rating depends on the user specific comprehension rating prior to receiving the answer, the current problem difficulty rating prior to receiving the answer, and whether the answer includes a correct or incorrect response;

outputting, for at least one of the different subject-matter areas that forms the basis of the practice problem, the updated user specific comprehension rating to an interactive graphical user interface; and

subsequent to automatically updating the corresponding user specific comprehension rating for each of the different subject-matter areas,

automatically outputting a new practice problem to the interactive graphical user interface based on matching at least one of the updated user specific comprehension ratings to a problem difficulty rating of the new practice problem or to within a predetermined range of the problem difficulty rating of the new practice problem.

27. A system, comprising:

a computing device; and

a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by one or more processors of the computing device, cause the computing device to perform operations for adaptive learning, the operations comprising;

receiving, over a network, from a client device, an identification by a first user of at least one subject-matter area to review;

responsive to receiving the identification, identifying a potential practice problem having a current problem difficulty rating within a predetermined range of at least one user specific comprehension rating for the first user, wherein the practice problem is identified by one or more concept tags, each concept tag relating to a different subject-matter area that forms a basis of the practice problem, wherein the current problem difficulty rating is derived based on whether one or more users previously answered the practice problem correctly or incorrectly;

transmitting, from the computing device, the identified practice problem to the client device over the network;

receiving, over the network, from the client device, an answer to the practice problem;

responsive to receiving the answer, determining whether the answer includes a correct or incorrect response to the practice problem;

responsive to determining whether the answer includes a correct or incorrect response, automatically updating, for each of the different subject-matter areas that forms a basis of the practice problem, a corresponding user specific comprehension rating to provide an updated user specific comprehension rating that is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem, and automatically updating the current problem difficulty rating of the practice problem to provide an updated problem difficulty rating, such that the updated problem difficulty rating is indirectly based on other user responses to the practice problem and directly based on the first user'"'"'s response to the practice problem,

wherein a magnitude of the changes to the user specific comprehension rating and the current problem difficulty rating depends on the user specific comprehension rating prior to receiving the answer, the problem difficulty rating prior to receiving the answer, and whether the answer includes a correct or incorrect response;

outputting, for at least one of the different subject-matter areas that forms the basis of the practice problem, the updated user specific comprehension rating to an interactive graphical user interface; and

subsequent to automatically updating the corresponding user specific comprehension rating for each of the different subject-matter areas,

automatically outputting a new practice problem to the interactive graphical user interface based on matching at least one of the updated user specific comprehension ratings to a problem difficulty rating of the new practice problem or to within a predetermined range of the problem difficulty rating of the new practice problem.