ENGAGEMENT LEVEL DETERMINATION AND DISSEMINATION

• US 20200135045A1
• Filed: 12/27/2018
• Published: 04/30/2020
• Est. Priority Date: 12/27/2018
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Alyuz Civitci, Nese, Tanriover, Cagri Cagatay, Aslan, Sinem, Okur Kavil, Eda, Arslan Esme, Asli, Genc, Ergin
• Granted Patent: US 11,908,345 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 50/20 Education
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 5/08 providing for individual presentation of information to a plurality of student stations

Abstract

Various systems and methods for engagement dissemination. A face detector detects a face in video data. A context filterer determines a student is on-platform and a section type. An appearance monitor selects an emotional and a behavioral classifiers. Emotional and behavioral components are classified based on the detected face. A context-performance monitor selects an emotional and a behavioral classifiers specific to the section type, Emotional and behavioral components are classified based on the log data. A fuser combines the emotional components into an emotional state of the student based on confidence values of the emotional components. The fuser combines the behavioral components a behavioral state of the student based on confidence values of the behavioral components. The user determine an engagement level of the student based on the emotional state and the behavioral state of the student.

Claims

1. An apparatus for engagement dissemination, the apparatus comprising:

a face detector configured to detect a face in video data;

a context filterer configured to determine a student is on-platform based on log data and to determine a section type for the student based on the log data;

an appearance monitor configured to;

select a first emotional classifier specific to the section type;

classify, using the first emotional classifier, a first emotional component based on the detected face;

select a first behavioral classifier specific to the section type; and

classify, using the first behavioral classifier, a first behavioral component based on the detected face;

a context-performance monitor configured to;

select a second emotional classifier specific to the section type;

classify, using the second emotional classifier, a second emotional component based on the log data and the section type;

select a second behavioral classifier specific to the section type; and

classify, using the second behavioral classifier_;

a second behavioral component based on the log data and the section type;

a fuser configured to;

combine the classified first emotional component and the second emotional component into an emotional state of the student based on confidence values of the first emotional component and the second emotional component;

combine the classified first behavioral component and the second behavioral component into a behavioral state of the student based on confidence values of the first behavioral component and the second behavioral component; and

determine an engagement level of the student based on the emotional state of the student and the behavioral state of the student.

2. The apparatus of claim 1, wherein the engagement level is one of engaged, may be engaged, or not engaged.

3. The apparatus of claim 2, further comprising a dashboard configured to:

determine an engagement color based on the engagement level of the student; and

display a student avatar, wherein the student avatar is colored the engagement color.

4. The apparatus of claim 3, wherein the dashboard is further configured to display an intervention alert for the student based on the engagement level of the student.

5. The apparatus of claim 3, wherein to determine the engagement color, the dashboard is further configured to:

determine a number of engagement levels for a previous time period;

determine a number of engagement level for each possible engagement level of the student for a previous time period;

calculate a ratio using the number of engagement levels and the number of engagement level for each possible engagement level;

determine a largest ratio from the calculated ratios; and

determine the engagement color based on the largest ratio.

6. The apparatus of claim 1, wherein the behavior state is one of on-task or off-task.

7. The apparatus of claim 4, wherein the emotional state is one of satisfied, bored, or confused.

8. The apparatus of claim 1, wherein the appearance monitor is further configured to:

calculate first features from detected face based on a proper subset of face animation parameters; and

calculate second features from the detected face based on movement of the detected face from previous video data, wherein the first features and the second features are used by the first emotional classifier and the second behavioral classifier.

9. The apparatus of claim 1, wherein the context-performance monitor is further configured to:

determine a number of questions accessed in a previous context time period; and

determine a number of questions where a hint associated with a question was accessed in the previous context time period, wherein the number of questions and the number of questions where a hint associated with a question was accessed are used by the first behavioral classifier and the second behavioral classifier.

10. A method for engagement dissemination, the method comprising operations performed using an electronic processor, the operations comprising:

detecting a face in video data;

determining a student is on-platform based on log data;

determining a section type for the student based on the log data;

selecting, using an appearance monitor (APM), a first emotional classifier specific to the section type;

classifying, using the first emotional classifier, a first emotional component based on the detected face;

selecting, using the APM, a first behavioral classifier specific to the section type;

classifying, using the first behavioral classifier, a first behavioral component based on the detected face;

selecting, using a context-performance monitor (CPM), a second emotional classifier specific to the section type;

classifying, using the second emotional classifier, a second emotional component based on the log data and the section type;

selecting, using the CPM, a second behavioral classifier specific to the section type;

classifying, using the second behavioral classifier, a second behavioral component based on the log data and the section type;

combining the classified first emotional component and the second emotional component into an emotional state of the student based on confidence values of the first emotional component and the second emotional component;

combining the classified first behavioral component and the second behavioral component into a behavioral state of the student based on confidence values of the first behavioral component and the second behavioral component; and

determining an engagement level of the student based on the emotional state of the student and the behavioral state of the student.

11. The method of claim 10, wherein the engagement level is one of engaged, may be engaged, or not engaged.

12. The method of claim 11, further comprising:

determining an engagement color based on the engagement level of the student; and

displaying a student avatar, wherein the student avatar is colored the engagement color.

13. The method of claim 12, further comprising displaying an intervention alert for the student based on the engagement level of the student.

14. The method of claim 12, wherein determining the engagement color comprises:

determining a number of engagement levels for a previous time period;

determining a number of engagement level for each possible engagement level of the student for a previous time period;

calculating a ratio using the number of engagement levels and the number of engagement level for each possible engagement level;

determining a largest ratio from the calculated ratios; and

determining the engagement color based on the largest ratio.

15. The method of claim 10, wherein the behavior state is one of on-task or off-task.

16. The method of claim 13, wherein the emotional state is one of satisfied, bored, or confused.

17. The method of claim 10, further comprising:

calculating, using the APM, first features from the detected face based on a proper subset of face animation parameters; and

calculating, using the APM, second features from the detected face based on movement of the detected face from previous video data, wherein the first features and the second features are used by the first emotional classifier and the second behavioral classifier.

18. The method of claim 10, further comprising:

determining, using the CPM, a number of questions accessed in a previous context time period; and

determining, using the CPM, a number of questions where a hint associated with a question was accessed in the previous context time period, wherein the number of questions and the number of questions where a hint associated with a question was accessed are used by the first behavioral classifier and the second behavioral classifier.

19. At least one non-transitory computer-readable medium including instructions for engagement dissemination, the computer-readable medium comprising instructions that, when executed by a machine, cause the machine to perform operations comprising:

detecting a face in video data;

determining a student is on-platform based on log data;

determining a section type for the student based on the log data;

selecting, using an appearance monitor (APM), a first emotional classifier specific to the section type;

classifying, using the first emotional classifier, a first emotional component based on the detected face;

selecting, using the APM, a first behavioral classifier specific to the section type;

classifying, using the first behavioral classifier, a first behavioral component based on the detected face;

selecting, using a context-performance monitor (CPM), a second emotional classifier specific to the section type;

classifying, using the second emotional classifier, a second emotional component based on the log data and the section type;

selecting, using the CPM, a second behavioral classifier specific to the section type;

classifying, using the second behavioral classifier, a second behavioral component based on the log data and the section type;

combining the classified first emotional component and the second emotional component into an emotional state of the student based on confidence values of the first emotional component and the second emotional component;

combining the classified first behavioral component and the second behavioral component into a behavioral state of the student based on confidence values of the first behavioral component and the second behavioral component; and

determining an engagement level of the student based on the emotional state of the student and the behavioral state of the student.

20. The at least one non-transitory computer-readable medium of claim 19, wherein the engagement level is one of engaged, may be engaged, or not engaged.

21. The at least one non-transitory computer-readable medium of claim 20, further comprising:

determining an engagement color based on the engagement level of the student; and

displaying a student avatar, wherein the student avatar is colored the engagement color.

22. The at least one non-transitory computer-readable medium of claim 21, further comprising displaying an intervention alert for the student based on the engagement level of the student.

23. The at least one non-transitory computer-readable medium of claim 21, wherein determining the engagement color comprises:

determining a number of engagement levels for a previous time period;

determining a number of engagement level for each possible engagement level of the student for a previous time period;

calculating a ratio using the number of engagement levels and the number of engagement level for each possible engagement level;

determining a largest ratio from the calculated ratios; and

determining the engagement color based on the largest ratio.

24. The at least one non-transitory computer-readable medium of claim 19, wherein the behavior state is one of on-task or off-task.

25. The at least one non-transitory computer-readable medium of claim 22, wherein the emotional state is one of satisfied, bored, or confused.