Computing method and system with detached sensor in a network environment

• US 9,472,116 B2
• Filed: 12/21/2015
• Issued: 10/18/2016
• Est. Priority Date: 08/13/1996
• Status: Expired due to Fees

• Current Assignee: IpLearn-Focus LLC (IpLearn LLC)
• Original Assignee: IpLearn-Focus LLC (IpLearn LLC)
• Inventors: Ho, Chi Fai, Tong, Peter P.
• Primary Examiner(s): Utama, Robert J
• Application Number:
  US14/976,930
• Publication Number:
  US 20160111012A1
• Time in Patent Office: 302 Days
• US Class Current: 1/1
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G09B 5/14 with provision for individual teacher-student communication
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Abstract

One embodiment includes a computer-implemented method and system in a network environment using at least a display and a detached sensor. Another embodiment includes a computer-implemented system helping a user learn using a detached imaging sensor. In yet another embodiment, a computer-implemented system monitors automatically more than once a user'"'"'s behavior while the user is working on materials. Through monitoring the user'"'"'s voluntary or involuntary behavior, the system determines what to present by the display. The presentation could include providing rewards, punishments, stimulation, and other materials. The system can also react by asking the user a question. Based on the user'"'"'s response, the system may change to more appropriate materials, or different presentation styles.

Claims

1. A computing apparatus comprising:

a storage unit;

an imaging sensor; and

a processor operatively connected to the imaging sensor and the storage unit,

wherein the storage unit storing a plurality of instructions that, when executed by the processor, cause the processor to;

acquire, in a first session, a first set of measurements produced from the imaging sensor sensing a first feature of a user in the first session, regarding a first behavior of the user, the first feature relating to the head of the user, the imaging sensor detached from the first feature when sensing the first feature, and the first set of measurements to include a plurality of images of the first feature of the user;

store data based on the first set of measurements in the first session, with the store including to link the stored data to an identity of the user;

present, in the first session, materials in a multi-window environment, by a first window in a display;

change what is to be presented by the first window in the display, with the change including to pause presenting the materials by the first window in the display;

resume to present the materials in the display in view of another set of measurements produced from the imaging sensor sensing the user, with the resume including to re-start presenting the materials paused;

acquire, in a second session, which is after the first session, a second set of measurements produced from the imaging sensor sensing the user;

access, in the second session, the stored data based on the stored data being linked to the identity of the user; and

compare the accessed stored data and data based on the second set of measurements to identify materials for presentation in the second session by a window in the display,

wherein the plurality of instructions, when executed by the processor, further cause the processor to perform, before the first session, a calibration process on at least an image of the user, with the calibration process using the imaging sensor, and with the present, in the first session, depending on the calibration process.

2. A computing apparatus as recited in claim 1, wherein at least some materials presented by the display in the first session depends on analyzing data based on measurements produced from the imaging sensor sensing the user, together with data based on measurements produced from another sensor in the computing apparatus regarding another behavior of the user.

3. A computing apparatus as recited in claim 2, wherein the another behavior of the user is an involuntary behavior of the user.

4. A computing apparatus as recited in claim 1, wherein the change what is to be presented depends on identifying, by the processor, the user not paying attention to materials presented by the display.

5. A computing apparatus as recited in claim 4, wherein the resume to present depends on the another set of measurements suggesting the user paying attention to materials presented by the display.

6. A computing apparatus as recited in claim 1, wherein the first feature includes a facial orientation of the user.

7. A computing apparatus as recited in claim 1, wherein the first feature includes at least one eye of the user.

8. A computing apparatus as recited in claim 1, wherein at least some of the materials presented by the display in the first session are retrieved, using a network interface unit in the computing apparatus, via a network external to the computing apparatus.

9. A computing apparatus as recited in claim 1, wherein the plurality of instructions, when executed by the processor, further cause the processor to determine a position on the display based on measurements produced from the imaging sensor sensing the user in the first session.

10. A computing apparatus as recited in claim 1, wherein the plurality of instructions, when executed by the processor, further cause the processor to control movement of some materials on the display based on measurements produced from the imaging sensor sensing the user in the first session.

11. A non-transitory computer readable storage medium comprising a plurality of instructions, the plurality of instructions executable by a computing apparatus to result in the computing apparatus:

acquiring, in a first session, a first set of measurements produced from an imaging sensor in the computing apparatus sensing a first feature of a user in the first session, regarding a first behavior of the user, the first feature relating to the head of the user, the imaging sensor detached from the first feature when sensing the first feature, and the first set of measurements to include a plurality of images of the first feature of the user;

storing data based on the first set of measurements in the first session, with the storing including linking the stored data to an identity of the user;

presenting, in the first session, materials in a multi-window environment, by a first window in a display;

changing what is to be presented by the first window in the display, with the changing including pausing presenting the materials by the first window in the display;

resuming presenting the materials in the display in view of another set of measurements produced from the imaging sensor sensing the user, with the resuming including re-starting presenting the materials paused;

acquiring, in a second session, which is after the first session, a second set of measurements produced from the imaging sensor sensing the user;

accessing, in the second session, the stored data based on the stored data being linked to the identity of the user; and

comparing the accessed stored data and data based on the second set of measurements to identify materials for presentation in the second session by a window in the display,

wherein the plurality of instructions executable by the computing apparatus to further result in the computing apparatus performing, before the first session, a calibration process on at least an image of the user, with the calibration process using the imaging sensor, and with the presenting, in the first session, depending on the calibration process.

12. A non-transitory computer readable storage medium as recited in claim 11, wherein at least some materials presented by the display in the first session depends on analyzing data based on measurements produced from the imaging sensor sensing the user, together with data based on measurements produced from another sensor in the computing apparatus regarding another behavior of the user.

13. A non-transitory computer readable storage medium as recited in claim 12, wherein the another behavior of the user is an involuntary behavior of the user.

14. A non-transitory computer readable storage medium as recited in claim 11, wherein the changing what is to be presented depends on identifying, by the computing apparatus, the user not paying attention to materials presented by the display.

15. A non-transitory computer readable storage medium as recited in claim 14, wherein the resuming presenting depends on the another set of measurements suggesting the user paying attention to materials presented by the display.

16. A non-transitory computer readable storage medium as recited in claim 11, wherein the first feature includes a facial orientation of the user.

17. A non-transitory computer readable storage medium as recited in claim 11, wherein the first feature includes at least one eye of the user.

18. A non-transitory computer readable storage medium as recited in claim 11, wherein at least some of the materials presented by the display in the first session are retrieved, using a network interface unit in the computing apparatus, via a network external to the computing apparatus.

19. A non-transitory computer readable storage medium as recited in claim 11, wherein the plurality of instructions executable by the computing apparatus to further result in the computing apparatus determining a position on the display based on measurements produced from the imaging sensor sensing the user in the first session.

20. A non-transitory computer readable storage medium as recited in claim 11, wherein the plurality of instructions executable by the computing apparatus to further result in the computing apparatus controlling movement of some materials on the display based on measurements produced from the imaging sensor sensing the user in the first session.