System and method for automated capture and compaction of instructional performances

• US 9,164,590 B2
• Filed: 12/22/2011
• Issued: 10/20/2015
• Est. Priority Date: 12/24/2010
• Status: Active Grant

• Current Assignee: Kevadiya, Inc.
• Original Assignee: Kevadiya, Inc.
• Inventors: Patel, Nilesh, Kroll, Scott, Schwartz, Mark
• Primary Examiner(s): LEE, NICHOLAS J
• Application Number:
  US13/995,266
• Publication Number:
  US 20130285909A1
• Time in Patent Office: 1,398 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

The system comprises functionality for instructors to record their lessons in an easy method while allowing them to capture their teaching techniques with a tool which improves the effectiveness of the playback, by reverse scripting of the teachers motions and use of educational tools including blackboards, whiteboards, and tablet computers among others and automating highlight of the relevant multiple media channels for best emphasis during presentation. Access to the centralized lesson database will permit students to learn from the best teachers and instructors and can execute the system right on their desktops or portable computers or access it through a dedicated website. Playback may be personalized to the needs and preferences of each student and the conceptual content is essentially highlighted in video and audio to maximize didactic effectiveness of presentation.

Claims

1. A method of capturing and rendering educational information, comprising:

using at least one processor associated with a recording computer;

(a) capturing gesture information from a surface digitizing apparatus coupled to the recording computer, and storing in a recording computer memory gestural data that associates the gesture information to a reference frame associated with the surface digitizing apparatus

(b) extracting image information from a camera coupled to the recording computer and storing in a recording computer memory image data that associates the image information to a reference frame associated with the camera'"'"'s field of view,

(c) generating recording space calibration data that associates spatial coordinates within the field of view of the camera with spatial coordinates of the surface digitizing apparatus;

(d) sending the gestural data, image data and calibration data to a rendering computer;

using at least one processor associated with a rendering computer coupled to a display device;

(e) receiving the gestural data, image data and calibration data sent by the recording computer;

(f) generating a calibration transform based on the calibration data that converts the gestural data and image data into a form usable by the display device to generate calibrated gestural data and calibrated image data;

(g) using the calibrated gestural data to render a first visual image corresponding to the gestural information captured by the surface digitizing apparatus;

(h) using the calibrated image data to generate a second visual image corresponding to the image information extracted from the camera;

(i) blending the first and second visual images to define a recomposed image made up of combined gesture information and image information that is then displayed on the display device.

2. The method of claim 1 wherein the (b) extracting image information step includes separating moving object information from stationary background information and using only the moving object information to define said image data.

3. The method of claim 1 wherein the blending step further comprises representing the first and second visual images as first and second layers where one of said layers is positioned on top of the other of said layers.

4. The method of claim 3 further comprising receiving playback mode selection input from a user and using said selection input to control which of the first and second layers is positioned on top.

5. The method of claim 1 further comprising using at least one processor associated with a recording computer to capture application-specific event data from an computer program generating educational content concurrently with at least one of steps (a) and (b), and sending the captured application-specific event data to the rendering computer.

6. The method of claim 5 further comprising using at least one processor associated with the rendering computer to run a computer program using the captured application-specific event data and to thereby generate an application-specific image for display.

7. The method of claim 1 wherein the sending and receiving steps (d) and (e) are performed using a networked computer system or telecommunication system.

8. The method of claim 1 wherein the sending and receiving steps (d) and (e) are performed using an intermediary computer-readable storage medium.

9. The method of claim 1 further comprising displaying the recomposed image so that the image information displays in front of the gestural information to simulate a view where a lecturer stands in front of a chalkboard.

10. The method of claim 1 further comprising displaying the recomposed image so that the image information displays behind the gestural information to simulate a view where a lecturer draws on a glass pane positioned between lecturer and audience.

11. The method of claim 6 further comprising displaying the recomposed image so that the gestural information displays in front of the application-specific image to simulate a view where a lecturer draws on a glass pane positioned in front of a displayed computer program.

12. The method of claim 1 wherein at least one of the captured gesture information and the extracted image information is three-dimensional information.

13. The method of claim 5 wherein the application-specific event data represents three-dimensional information.