LEARNING VALIDATION USING GESTURE RECOGNITION

• US 20130097565A1
• Filed: 10/17/2011
• Published: 04/18/2013
• Est. Priority Date: 10/17/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Stone, Brian, Atkins, Joshua Whitney Samuels, White, Matthew, Freeding, Aimee
• Application Number:
  US13/275,134
• Publication Number:
  US 20130097565A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

Embodiments are disclosed that relate to assessing a user'"'"'s ability to recognize a target item by reacting to the target item and performing a target gesture. For example, one disclosed embodiment provides a method of assessing a user'"'"'s ability to recognize a target item from a collection of learning items that includes the target item. The method includes providing to a display device the learning items in a sequence and receiving input from a sensor to recognize a user gesture made by the user. If the user gesture is received within a target timeframe corresponding to the target item, then the method includes determining whether the user gesture matches a target gesture. If the user gesture matches the target gesture, then the method includes providing to the display device a reward image for the user.

Claims

1. In a computing device, a method of assessing a user'"'"'s ability to recognize a target item from a collection of learning items that includes the target item, the method comprising:

providing to a display device the learning items in a sequence;

while providing to the display device the learning items in the sequence, receiving input from a sensor to recognize a user gesture made by the user;

determining whether the user gesture is received within a target timeframe corresponding to the target item;

if the user gesture is received within the target timeframe corresponding to the target item, then determining whether the user gesture matches a target gesture; and

if the user gesture matches the target gesture, then providing to the display device a reward image for the user.

2. The method of claim 1, further comprising:

if the user gesture is not received within the target timeframe corresponding to the target item, then;

storing an incorrect answer instance;

selecting a hint from a hint structure; and

providing the hint to the display device for display; and

if the user gesture is received within the target timeframe corresponding to the target item, then storing a correct answer instance.

3. The method of claim 2, further comprising:

if the incorrect answer instance is a first incorrect answer instance, then the hint is a first hint; and

if the incorrect answer instance is a second incorrect answer instance, then the hint is a second hint that provides different support than the first hint previously provided to the display device.

4. The method of claim 3, further comprising presenting the first hint to the user via one or more of audio feedback and visual feedback.

5. The method of claim 2, further comprising:

providing to the display device multiple instances of the target item; and

detecting whether the user gesture is received within the target timeframe corresponding to each instance of the target item; and

providing to the display device a performance measure that relates a number of the correct answer instances to a number of the multiple instances of the target item provided to the display device.

6. The method of claim 1, wherein if the user gesture does not match the target gesture, then providing a target gesture reminder to the display device.

7. The method of claim 1, further comprising:

using the input received from the sensor to determine that the user has not reacted to a threshold number of instances of the target item; and

providing to the display device a reaction reminder that encourages the user to react when the user sees the target item.

8. The method of claim 7, wherein the threshold number is a first threshold number and the reaction reminder is a first reaction reminder that includes audio feedback, and the method further comprising:

using the input received from the sensor to determine that the user has not reacted to a second threshold number of instances of the target item, where the second threshold number is greater than the first threshold number; and

providing to the display device a second reaction reminder that includes audio and visual feedback that encourages the user to react when the user sees the target item.

9. The method of claim 6, wherein the target gesture reminder comprises a character performing the target gesture.

10. The method of claim 1, further comprising customizing the reward image according to the target gesture.

11. The method of claim 1, wherein the sensor comprises a depth camera.

12. The method of claim 1, wherein providing to a display device the learning items in a sequence and providing to the display device the reward image for the user further comprise one or more of synchronously streaming multiple layers of streamed video content to the display device and branching between a first buffered video content portion and a second buffered video content portion.

13. In a computing device, a method of assessing a user'"'"'s ability to recognize a target item from a collection of learning items that includes the target item, the method comprising:

providing to a display device the learning items in a sequence;

while displaying the learning items in a sequence, receiving from a depth camera image data capturing a user gesture made by the user;

determining whether the user gesture is received within a target timeframe corresponding to the target item;

if the user gesture is not received within the target timeframe corresponding to the target item, then;

storing an incorrect answer instance,

selecting a hint from a hint structure, and

providing the hint to the display device for display;

if the user gesture was performed within the target timeframe, then determining whether the user gesture matches a target gesture;

if the user gesture does not match the target gesture, then providing to the display device a target gesture reminder; and

if the user gesture matches the target gesture, then providing to the display device a reward image for the user.

14. The method of claim 13, wherein the target item is presented as at least a first item and a last item provided to the display device in the sequence.

15. The method of claim 14, wherein the target item comprises one or more of a number or a letter, and the target gesture comprises one or more of a limb motion, a jumping motion or a clapping motion.

16. The method of claim 13, further comprising:

if the incorrect answer instance is a first incorrect answer instance, then the hint is a first hint; and

if the incorrect answer instance is a second incorrect answer instance, then the hint is a second hint that provides different support than the first hint.

17. The method of claim 13, further comprising using image data received from the depth camera to determine that the user has not reacted to a threshold number of instances of the target item;

and

providing a reaction reminder to the user that encourages the user to react when the user sees the target item.

18. The method of claim 17, wherein the threshold number is a first threshold number and the reaction reminder is a first reaction reminder that includes audio feedback, further comprising:

using image data received from the depth camera to determine that the user has not reacted to a second threshold number of instances of the target item, where the second threshold number is greater than the first threshold number; and

providing a second reaction reminder to the user including audio feedback and visual feedback on the display device that encourages the user to react when the user sees the target item.

19. A computer readable storage medium excluding signals per se and comprising instructions executable by a computing device to assess a user'"'"'s ability to recognize a target item from a collection of learning items that includes the target item, the instructions being executable to:

provide to a display device the learning items in a sequence;

while providing the learning items to the display device, receive image data from a sensor and recognize via the image data a user gesture made by the user;

determine whether the user gesture is received within a target timeframe corresponding to the target item;

if the user gesture is received within the target timeframe corresponding to the target item, then determine whether the user gesture matches a target gesture; and

if the user gesture matches the target gesture, then provide to the display device a reward image for the user.

20. The computer readable storage medium of claim 19, wherein the instructions are executable by the computing device to one or more of branch between a first buffered video content and a second buffered video content, and synchronously stream multiple layers of streamed video content to the display device.