Systems and methods for providing information incorporating reinforcement-based learning and feedback

• US 10,249,206 B2
• Filed: 06/16/2015
• Issued: 04/02/2019
• Est. Priority Date: 06/01/2011
• Status: Active Grant

• Current Assignee: D2L Corporation
• Original Assignee: D2L Corporation
• Inventors: Bilic, Jugoslav, Bilic, Ivana, Callison, Justin, Aggarwal, Manuj, Michaud, Stephen, Ogg, Ryan, Ramirez, Juan Camilo, Sarkaria, Sarbjit, Zhou, Jian
• Primary Examiner(s): Al-Hashemi, Sana A
• Application Number:
  US14/740,733
• Publication Number:
  US 20150310754A1
• Time in Patent Office: 1,386 Days
• Field of Search: 707687, 707693, 707797, 707704, 707713, 707736, 707758, 707821
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F

Abstract

Systems and methods are provided for feedback-driven provision of information in relation to a plurality of information resources. Some such methods involve: accessing information resources from one or more information resource repositories to establish a mapping between each information resource and each of the other information resources; maintaining a state table comprising a plurality of state-table records, each state-table record attributing a value to a state-action pair, the state-action pair comprising an indication of a series of one or more actions in relation to the information resources; monitoring actions of a first user in relation to the information resources; receiving a feedback metric related to interaction of the first user with the information resources; using the feedback metric as a basis for updating the values of a subset of the state-table records, the subset of the state-table records comprising state-action pairs which correspond to the monitored actions of the first user in relation to the information resources; and providing output information related to the information resources based at least in part on the updated values of the state-table records.

Claims

1. A method, the method comprising:

storing a first mapping among a plurality of information resources stored on one or more repositories;

monitoring an action of a first user in relation to at least a subset of the plurality of information resources;

obtaining a feedback metric associated with the action of the first user in relation to the at least the subset of the plurality of information resources;

determining a second mapping of one or more actions to the at least the subset of the plurality of information resources based at least in part on feedback metric; and

providing output information related to the plurality of information resources based at least in part on the second mapping.

3. A method according to claim 1 wherein the output information comprises one or more recommended actions for a particular user in relation to the plurality of information resources, the one or more recommended actions relating to corresponding ones of the plurality of information resources.

4. A method according to claim 1 wherein the one or more information resource repositories comprise a plurality of information resource repositories, at least two of which are maintained by different entities.

5. A method according to claim 1 wherein the one or more information resource repositories comprise a plurality of information resource repositories, at least two of which are different types of information resource repositories.

6. A method according to claim 1 wherein one of the one or more information resource repositories comprises the internet generally.

7. A method according to claim 1 wherein one of the one or more resources comprises a portion of the internet, the portion of the internet maintained by one or more entities.

8. A method according to claim 1 wherein storing the first mapping among a plurality of information resources stored on one or more repositories comprises determining mappings between individual information resources based on semantic analysis of the individual information resources.

9. A method according to claim 3 wherein the particular user is different than the first user.

10. A method according to claim 3 comprising:

monitoring actions of each of a plurality of first users in relation to the at least the subset of the plurality of information resources;

obtaining a plurality of feedback metrics, each the plurality of feedback metrics relating to the interaction of a corresponding one of the plurality of first users with the at least the subset of information resources; and

determining the second mapping based at least in part on one or more of the plurality of feedback metrics.

11. A method according to claim 8 wherein determining the mappings between individual information resources based on semantic analysis comprise determining a metric that provides an indication of similarity between each information resource and each of the other information resources.

12. A method according to claim 8 wherein determining the mappings between each information resource and each of the other information resources comprises crawling at least one of the one or more information repositories to access the information resources.

13. A method according to claim 8 wherein determining the mappings between each information resource and each of the other information resources comprises receiving information pushed from at least one of the one or more repositories.

14. A method according to claim 9 wherein obtaining the feedback metric comprises receiving a grade resulting from the first user taking an assessment, the assessment comprising one or more questions related to information included in the information resources.

15. A method according claim 9 wherein receiving the feedback metric comprises receiving the feedback metric from an instructor based on the instructor'"'"'s assessment of a level of understanding of the first user of information included in the information resources.

16. A method according to claim 9 wherein monitoring the action of the first user comprises determining that the first user has interacted with a particular set of information resources and wherein obtaining the feedback metric comprises obtaining an assigned feedback metric in the event that the first user has interacted with the particular set of information resources.

17. A method according to claim 9 wherein monitoring the action of the first user comprises determining that the first user has interacted with one or more particular information resources a threshold number of times and wherein obtaining the feedback metric comprises obtaining an assigned feedback metric in the event that the first user has interacted with the one or more particular information resources the threshold number of times.

18. A method according to claim 9 wherein obtaining the feedback metric comprises obtaining user-evaluation feedback from the first user, the user-evaluation feedback relating to the first user'"'"'s perception of the first user'"'"'s interaction with the information resources.

19. A method according to claim 9 wherein determining the second mapping comprises using reinforcement learning to determine the updated values.

20. A method according to claim 9 wherein the determining the second mapping comprises updating a previous mapping corresponding to the monitored actions of the first user since a previous time that a feedback metric was received in relation to the action of the first user with the at least the subset of the plurality of information resources.

2. A system, comprising

one or more processors configured to:

store a first mapping among a plurality of information resources stored on one or more repositories;

monitor an action of a first user in relation to at least a subset of the plurality of information resources;

obtain a feedback metric associated with the action of the first user in relation to the at least the subset of the plurality of information resources;

determine a second mapping of one or more actions to the at least the subset of the plurality of information resources based at least in part on feedback metric; and

provide output information related to the plurality of information resources based at least in part on the second mapping; and

a memory coupled to the at least one processor and configured to provide the at least one processor with instructions.