Learning method and system using detached sensor

• US 8,475,174 B2
• Filed: 05/26/2012
• Issued: 07/02/2013
• Est. Priority Date: 08/13/1996
• Status: Expired due to Fees

• Current Assignee: IpLearn-Focus LLC (IpLearn LLC)
• Original Assignee: IpLearn-Focus LLC (IpLearn LLC)
• Inventors: Ho, Chi Fai, Tong, Peter
• Primary Examiner(s): Utama, Robert J
• Application Number:
  US13/481,821
• Publication Number:
  US 20120237920A1
• Time in Patent Office: 402 Days
• Field of Search: 434/322, 434/323, 434/350
• US Class Current: 434/322
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G09B 5/14 with provision for individual teacher-student communication
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Abstract

One embodiment includes a computer-implemented method using a window environment of a display, with a detached sensor, to teach a user. Another embodiment includes a computer system helping teach a user using a detached imaging sensor. In yet another embodiment, a computer-aided-educational system considers a user'"'"'s concentration level when teaching the user. The system monitors automatically more than once a user'"'"'s concentration-sensitive behavior while the user is working on study materials. Through monitoring the user'"'"'s volitional or involuntary behavior, the system provides an indication on the user'"'"'s concentration level. Based on the indication, the system could react such as by providing rewards, punishments, and stimulation; or changing the study materials. The system can also react by asking the user a question to stimulate the user or to assess the user'"'"'s understanding. Based on the user'"'"'s response, the system may change to more appropriate study materials, or different presentation styles.

Claims

1. A computer-implemented method for a user using at least a first window of a display, with at least some of the space outside of the first window being viewable via the display, with the space outside of the first window including at least a part of another window at least partially viewable via the display, the method comprising:

acquiring data, by a computing device, regarding a volitional input of the user, the data from a sensor sensing a physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to sense at least the physical attribute;

analyzing the data by the computing device, which is coupled to the sensor and the display;

determining by the computing device from the analyzing of the data whether the volitional input suggests the user paying attention to materials in the first window or not to materials in the first window, whereby the volitional input suggesting the user not paying attention to materials in the first window indicates a change in situation from the volitional input suggesting the user paying attention to materials in the first window; and

adjusting, by the computing device, materials for presenting to the user if the determining determines that the volitional input suggests the user not paying attention to materials in the first window,

wherein the sensor includes an imaging sensor.

2. A computer-implemented method as recited in claim 1 further comprising waiting for a period of time before adjusting,

wherein the adjusting comprises switching to alternative materials that are different from materials that would have been presented,

wherein the alternative materials include materials on a product, and

wherein at least a portion of the alternative materials are configured to be retrieved by the computing device, via a network, from another computing device.

6. A computer-implemented method as recited in claim 1 wherein the method further comprises determining understanding of the user in materials presented to the user.

7. A computer-implemented method as recited in claim 1 further comprising, in view of the analyzing, having materials retrieved to be presented to the user to help the user.

9. A computer-implemented method as recited in claim 1 further comprising asking the user a question in view of the analyzing to help the user.

13. A computer-implemented method as recited in claim 1, wherein the method is configured to electronically sense and analyze another volitional input of the user using another sensor to help determine what to present to the user, the another volitional input being of a type different from the volitional input of the user.

3. A computer-implemented method for a user using at least a first window of a display, with at least some of the space outside of the first window being viewable via the display, with the space outside of the first window including at least a part of another window at least partially viewable via the display, the method comprising:

acquiring data, by a computing device, regarding a volitional input of the user, the data from a sensor sensing a physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to sense at least the physical attribute;

analyzing the data by the computing device, which is coupled to the sensor and the display; and

determining by the computing device from the analyzing of the data whether the volitional input suggests the user paying attention to materials in the first window or not to materials in the first window, whereby the volitional input suggesting the user not paying attention to materials in the first window indicates a change in situation from the volitional input suggesting the user paying attention to materials in the first window,

wherein the volitional input depends on at least facial information of the user, and

wherein the sensor includes an imaging sensor.

4. A computer-implemented method as recited in claim 3 wherein the facial information includes the user frowning.

8. A computer-implemented method as recited in claim 3, wherein the facial information is related to an eye behavior of the user.

10. A computer-implemented method as recited in claim 3,

wherein the sensor includes a camera, and

wherein the data includes an image of the user.

11. A computer-implemented method as recited in claim 10 further comprising analyzing more than one image of the user produced by the camera to determine whether the volitional input suggests the user paying attention to materials in the first window or not to materials in the first window.

12. A computer-implemented method as recited in claim 10 further comprising asking the user to be in front of the camera for producing the data.

5. A computer-implemented method for enabling a user to learn via a first window of a display, with at least some of the space outside of the first window being viewable via the display, with the space outside of the first window including at least a part of another window at least partially viewable via the display, the method comprising:

acquiring data, by a computing device, regarding a volitional input of the user, the data from a sensor sensing a physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to sense at least the physical attribute;

analyzing the data by the computing device, which is coupled to the sensor and the display;

determining by the computing device from the analyzing of the data whether the volitional input suggests the user paying attention to materials in the first window or not to materials in the first window, whereby the volitional input suggesting the user not paying attention to materials in the first window indicates a change in situation from the volitional input suggesting the user paying attention to materials in the first window; and

using a rule regarding a speed of the user to analyze a volitional input of the user to help the user learn.

14. A computer-implemented method for a user using at least a first window of a display, with at least some of the space outside of the first window being viewable via the display, with the space outside of the first window including at least a part of another window at least partially viewable via the display, the method comprising:

acquiring data, by a computing device, regarding a volitional input of the user, the data from a sensor sensing a physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to sense at least the physical attribute;

analyzing the data by the computing device, which is coupled to the sensor and the display; and

determining by the computing device from the analyzing of the data whether the volitional input suggests the user paying attention to materials in the first window or not to materials in the first window, whereby the volitional input suggesting the user not paying attention to materials in the first window indicates a change in situation from the volitional input suggesting the user paying attention to materials in the first window,

wherein the sensor includes an imaging sensor configured to produce data regarding volitional inputs of the user at different times, and

wherein the method further comprises having materials retrieved for the user, based upon changes in the volitional inputs of the user from the sensor sensing at least the physical attribute associated with the user.

15. A computer-implemented method for enabling a user to learn via a first window of a display, with at least some of the space outside of the first window being viewable via the display, with the space outside of the first window including at least a part of another window at least partially viewable via the display, the method comprising:

acquiring data, by a computing device, regarding a volitional input of the user, the data from a sensor sensing a physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to sense at least the physical attribute;

analyzing the data by the computing device, which is coupled to the sensor and the display;

determining by the computing device from the analyzing of the data whether the volitional input suggests the user paying attention to materials in the first window or not to materials in the first window, whereby the volitional input suggesting the user not paying attention to materials in the first window indicates a change in situation from the volitional input suggesting the user paying attention to materials in the first window;

acquiring, in a session, another piece of data regarding another volitional input of the user, the another piece of data being produced by the sensor sensing the physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to sense at least the physical attribute; and

comparing the another piece of data with the data to enable the user to learn materials during the session.

16. A computer system for a user comprising:

an imaging sensor configured to produce data regarding volitional behaviors of the user at different times;

computer memory having materials for the user to view; and

a controller associated with the computer memory, the controller configured to communicate with the sensor and analyze the data to help determine what to present to the user based upon changes in the volitional behaviors,

wherein the sensor is configured to be detached from an area of the user to sense at least the area for producing the data,

wherein the volitional behaviors depend on at least facial information of the user, and

wherein the system is configured to sense and analyze another volitional behavior of the user using another sensor to help determine what to present to the user, the another volitional behavior being of a type different from the volitional behaviors of the user.

17. A computer system as recited in claim 16,

wherein the sensor includes a camera, and

wherein the data includes an image of the user.

18. A computer system as recited in claim 16, wherein the facial information is related to an eye behavior of the user.

21. A computer system as recited in claim 16, wherein the system includes a display with a window, and the controller is configured to determine if the volitional behaviors of the user suggest the user paying attention to materials in the window.

19. A computer-implemented method to enable a user to learn materials comprising:

capturing in a session, a first piece of information regarding a volitional input of the user, by a sensor monitoring a physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to monitor at least the physical attribute;

linking by a computing device, which includes the sensor, the first piece of information to an identity of the user;

storing the first piece of information with the identity of the user in a memory device;

capturing in another session, a second piece of information regarding another volitional input of the user, by the sensor monitoring the physical attribute associated with the user, the sensor being configured to be detached from the physical attribute associated with the user to monitor at least the physical attribute;

retrieving, by the computing device, the first piece of information from the memory device; and

comparing, by the computing device, the second piece of information with the first piece of information to help the user learn materials,

wherein the first piece of information includes at least an image of the user.

20. A computer-implemented method as recited in claim 19 further comprising adjusting materials for presenting to the user in view of the comparing.

22. A computer system for a user comprising:

an imaging sensor configured to produce data regarding volitional behaviors of the user at different times;

computer memory having materials for the user; and

a controller associated with the computer memory, the controller configured to communicate with the imaging sensor and analyze the data to have materials retrieved for the user, based upon changes in the volitional behaviors,

wherein the imaging sensor is configured to be detached from an area of the user to sense at least the area for producing the data, and

wherein the controller is further configured to use a rule regarding a speed of the user to analyze the volitional behaviors of the user to help determine what to present to the user.

23. A computing system for a user comprising:

a display having a first window, with at least some of the space outside of the first window being viewable via the display, and with the space outside of the first window including at least a part of another window at least partially viewable via the display;

a sensor configured to sense an area of the user to produce data regarding a volitional behavior of the user, the sensor being configured to be detached from the area of the user to sense the area, wherein the sensor includes a camera and the data includes an image of the user; and

a processor coupled to the sensor and the display, the processor configured to;

analyze, relative to the display, at least the data;

determine from the analysis whether the volitional behavior to suggest the user paying attention to materials in the first window or not to materials in the first window, whereby the volitional behavior to suggest the user not paying attention to materials in the first window to indicate a change in situation from the volitional behavior to suggest the user paying attention to materials in the first window; and

adjust what is to be presented by the display in view of the determination that the volitional behavior to suggest the user not paying attention to materials in the first window, and resume what is to be presented by the display in view of a subsequent volitional behavior to suggest the user paying attention to materials in the first window,

wherein the subsequent volitional behavior to suggest the user paying attention is based on analysis, relative to the display, of data from the sensor.

24. A computing system as recited in claim 23, wherein the sensor is configured to sense the user periodically to help determine if the user is paying attention to materials in the first window or not to materials in the first window.

25. A computing system as recited in claim 23 wherein to adjust includes to adjust a visual effect.

26. A computing system as recited in claim 23, wherein the volitional behavior to depend on whether the user has drooped.

27. A computing system as recited in claim 23, wherein the volitional behavior to depend on a head orientation of the user.

28. A computing system as recited in claim 23,

wherein the volitional behavior to depend on at least facial information of the user, and

wherein the facial information to include information regarding an eye of the user.

29. A computing system as recited in claim 23,

wherein the volitional behavior to depend on at least facial information of the user, and

wherein the facial information to include information regarding an eyelid of the user.

30. A computing system as recited in claim 23, wherein in view of the subsequent volitional behavior to suggest the user paying attention to materials in the first window, the processor is configured to restore the display back to a previous screen.

31. A computing system as recited in claim 23,

wherein the sensor is configured to sense the area to produce the data in a session,

wherein the sensor, in another session, is configured to sense an area of the user to produce another piece of data regarding another volitional behavior of the user, the sensor being configured to be detached from the area of the user where the sensor is sensing, and

wherein the another piece of data is to be electronically linked to an identity of the user.

32. A computing system as recited in claim 31, wherein the identity is to be provided by the user upon a request by the computing system.

33. A computing system as recited in claim 24, wherein to adjust includes to adjust a visual effect.

34. A computing system as recited in claim 33,

wherein the volitional behavior to depend on a head orientation of the user, and

wherein to resume what is to be presented by the display includes to restore the display back to a previous screen.

35. A computing system as recited in claim 34,

wherein the sensor is configured to sense the area to produce the data in a session,

wherein the sensor, in another session, is configured to sense an area of the user to produce another piece of data regarding another volitional behavior of the user, the sensor being configured to be detached from the area of the user where the sensor is sensing,

wherein the another piece of data is to be electronically linked to an identity of the user,

wherein the identity is to be provided by the user upon a request by the computing system, and

wherein the computing system is configured to direct the user to face the display to sense the area of the user in the another session.

36. A computing system as recited in claim 32,

wherein the sensor is configured to sense the user periodically in the session to help determine if the user is paying attention to materials in the first window or not to materials in the first window,

wherein the processor is configured to identify materials to present to the user at least based on the another piece of data, and

wherein to produce the another piece of data, the computing system is configured to direct the user to face the display.

37. A computing system for a user using a display that is configured to display a first window, with at least some of the space outside of the first window being viewable via the display, and with the space outside of the first window including at least a part of another window at least partially viewable via the display, the computing system comprising:

a sensor configured to;

sense, in a session, an area of the user to produce a first piece of data regarding a volitional behavior of the user, the first piece of data being electronically linked to an identity of the user, with the sensor being configured to be detached from the area of the user to sense the area; and

sense, in another session, an area of the user to produce a second piece of data regarding another volitional behavior of the user, with the sensor being configured to be detached from the area of the user to sense the area; and

a processor coupled to the sensor and the display, the processor configured to compare at least the first piece of data with the second piece of data to help identify materials for presenting to the user via the display,

wherein the sensor includes a camera, and

wherein the first piece of data includes an image of the user.

38. A computing system as recited in claim 37 wherein the sensor is configured to sense the user periodically to help identify materials for presenting to the user via the display.

39. A computing system as recited in claim 37 wherein the identified materials are retrieved via a network external to the computing system for presenting to the user via the display.

40. A computing system as recited in claim 37, wherein the processor is configured to direct the user to be in front of the computing system to help the sensor sense the area of the user to produce the piece of data regarding the volitional behavior of the user in the session.

41. A computing system as recited in claim 37, wherein the materials include materials on a product.

42. A computing system as recited in claim 37, wherein certain materials are identified for presenting to the user via the display based not only on the piece of data and the another piece of data from the two sessions, but also on an additional piece of data produced by the sensor regarding an additional volitional behavior of the user in an additional session.

43. A computing system as recited in claim 37, wherein the processor is configured to use a rule regarding a speed of the user to help determine a progress of the user in learning.

44. A computing system as recited in claim 43, wherein the rule depends on comparing the speed of the user with another speed of the user.

45. A computing system as recited in claim 37, wherein the processor is configured to:

analyzing a different piece of data from the sensor;

determining whether a result from the analyzing suggests the user paying attention to materials in the first window or not to materials in the first window; and

adjusting what is to be presented by the display in view of the determining determines that the result suggests the user not paying attention to materials in the first window.

46. A computing system as recited in claim 45, wherein the processor is configured to resuming what is to be presented by the display in view of a piece of data produced by the sensor subsequent to the different piece of data, the piece of data suggesting the user paying attention to materials in the first window.

47. A computing system as recited in claim 37, wherein the system includes the display.

48. A computing system as recited in claim 38, wherein the processor is configured to direct the user to be in front of the computing system to help the sensor sense the area of the user in the session.

49. A computing system as recited in claim 48,

wherein the processor is configured to use a rule regarding a speed of the user to help determine a progress of the user in learning, and

wherein the rule depends on comparing the speed of the user with another speed of the user.

50. A computing system as recited in claim 49, wherein certain materials are identified for presenting to the user via the display based not only on the piece of data and the another piece of data from the two sessions, but also on an additional piece of data produced by the sensor regarding an additional volitional behavior of the user in an additional session.

51. A computing system as recited in claim 50, wherein the processor is configured to:

analyzing a different piece of data from the sensor;

determining whether a result from the analyzing suggests the user paying attention to materials in the first window or not to materials in the first window; and

adjusting what is to be presented by the display in view of the determining determines that the result suggests the user not paying attention to materials in the first window.

52. A computing system as recited in claim 51, wherein the processor is configured to resuming what is to be presented by the display in view of a piece of data produced by the sensor subsequent to the different piece of data, the piece of data suggesting the user paying attention to materials in the first window.

53. A computing system as recited in claim 52, wherein the processor is configured to resuming what is to be presented after receiving a response from the user to an inquiry from the computing system.

54. A computing system as recited in claim 52, wherein the system includes the display.

55. A computing system as recited in claim 48, wherein the processor is configured to:

analyzing a different piece of data from the sensor;

determining whether a result from the analyzing suggests the user paying attention to materials in the first window or not to materials in the first window; and

adjusting what is to be presented by the display in view of the determining determines that the result suggests the user not paying attention to materials in the first window.

56. A computing system as recited in claim 55, wherein the processor is configured to resuming what is to be presented by the display in view of a piece of data produced by the sensor subsequent to the different piece of data, the piece of data suggesting the user paying attention to materials in the first window.

57. A computing system as recited in claim 56, wherein the system includes the display.

58. A computing system for a user using a display that is configured to display a first window, with at least some of the space outside of the first window being viewable via the display, and with the space outside of the first window including at least a part of another window at least partially viewable via the display, the computing system comprising:

a sensor configured to sense an area of the user to produce data regarding a volitional behavior of the user, the sensor being configured to be detached from the area of the user to sense the area, wherein the sensor includes a camera and the data includes an image of the user; and

a processor coupled to the sensor and the display, the processor configured to;

analyzing, relative to the display, the data;

determining from analyzing the data whether the volitional behavior suggests the user paying attention to materials in the first window or not to materials in the first window, whereby the volitional behavior suggesting the user not paying attention to materials in the first window indicates a change in situation from the volitional behavior suggesting the user paying attention to materials in the first window; and

adjusting what is to be presented by the display if the determining determines that the volitional behavior suggests the user not paying attention to materials in the first window, wherein the adjusting includes asking the user to respond to an inquiry.