Iterative video teaching aid with recordable commentary and indexing

• US 6,599,130 B2
• Filed: 02/02/2001
• Issued: 07/29/2003
• Est. Priority Date: 02/02/2001
• Status: Expired due to Term

• Current Assignee: Tamabo, Inc.
• Original Assignee: Illinois Institute of Technology
• Inventors: Moehrle, Armin
• Primary Examiner(s): Cheng, Joe H.
• Assistant Examiner(s): MOSSER, KATHLEEN MICHELE
• Application Number:
  US09/775,463
• Publication Number:
  US 20020106623A1
• Time in Patent Office: 907 Days
• Field of Search: 707/512, 707/300, 434/262, 434/322, 434/365, 434/307.R, 600/217, 600/345
• US Class Current: 434/365
• CPC Class Codes: G09B 23/28 for medicine

Abstract

A tutorial or diagnostic aid based on the representation and iterative interpretation of visual images is taught. A teaching or diagnostic session is created by overlaying an interpretation layer via a software application onto the visual image layer and synchronizing the two with the time code of the visual image. The interpretation layer allows the reviewer to identify image areas of interest by gesture and append comments thereto in real time; i.e. images or portions of images within the visual representation playback may be identified and labeled and have the concurrent commentary associated therewith. The comments are indexed and linked to a database of similar topics. The flow of the session is recorded to show the images, deictic gestures associated therewith, and commentary associated with the gestures, to enable subsequent users to playback a session and follow the flow of thought (i.e. image identification and commentary within the original session). Iterative sessions allow additional image identification and commentary to be accomplished. Additional database access through the index may further enhance a teaching session, or provide for research sessions and report generation.

Claims

1. A diagnostic and teaching aid including:

a. means for displaying a sequence of images;

b. means for selecting a portion of one image of the sequence of images via gestures of a diagnostician and means for recording a subframe image selection;

c. means for recording a commentary of the diagnostician concerning a selected image portion;

d. means for linking a recorded commentary concerning the selected image portion to the selected image portion;

e. means for recording a plurality of image portion selections and a sequence in which the plurality of image portion selections were selected so as to create a teaching session; and

f. means for indexing at least one of the recorded commentary and a selected image portion so as to make the teaching session retrievable through an index.

2. The diagnostic and teaching aid of claim 1 wherein the means for displaying sequence of images includes a digital video processor.

3. The diagnostic and teaching aid of claim 1 wherein the sequence of images are full motion images.

4. The diagnostic and teaching aid of claim 1 wherein the means for selecting a portion of one image includes a mouse.

5. The diagnostic and teaching aid of claim 1 wherein the means for selecting portion of one image includes a keyboard.

6. The diagnostic and teaching aid of claim 1 wherein the means for selecting portion of one image includes a touch screen.

7. The diagnostic and teaching aid of claim 1 wherein the means for recording commentary includes a menu of terminology related to the images.

8. The diagnostic and teaching aid of claim 7 further including means for associating related terms within the menu of terminology.

9. The diagnostic and teaching aid of claim 8 further including means for recording a frequency with which terms are related by the diagnostician and adjusting the menu of terminology according to the frequency.

10. The diagnostic and teaching aid of claim 1 further including means for indexing the selected image portion by automated image pattern recognition.

11. The diagnostic and teaching aid according to claim 1 further including:

means for playing back the sequence of images while displaying the plurality of image portion selections and recorded commentary concerning the selected image portions, the play back being in the sequence in which the plurality of image portion selections were selected.

12. A real time diagnostic and teaching aid including:

a. means for displaying full motion images;

b. means for selecting subframe images from displayed full motion images in real time via gestures of a diagnostician and means for recording a subframe selection;

c. means for recording and linking concurrent commentary of the diagnostician to selected subframe images in real time;

d. means for recording a sequence of subframe selection and linked commentary to create a teaching session;

e. means for indexing the session by at least one of the commentaries and subframe images;

f. means for play back of the session; and

g. means for adding additional commentary into the session.

13. The real time diagnostic and teaching aid according to claim 12 further including means for selecting additional sub-frames during the session playback to create a further session;

and means for indexing the further session.

14. The real time diagnostic and teaching aid according to claim 12 further including a separate application layer for overlaying image selection and concurrent commentary on the image data display without disturbing the image data.

15. The real time diagnostic and teaching aid according to claim 12 further including means for displaying more than one session at a time as inserted frames.

16. The real time diagnostic and teaching aid according to claim 12 wherein the means for indexing the further session includes means for indexing by at least one of the concurrent commentary, the additional commentary, and the subframes images.

17. A diagnostic aid or teaching system comprising a computer having:

a. a video layer for the display and quantization of a series of images;

b. an interpretation layer, time synchronized to, and running concurrently with, the video layer, and having a gesture recorder for identifying and recording selected portions of images in the video layer;

c. the interpretation layer further including a comment recorder which records and displays concurrent comments and associates the concurrent comments with the gestures;

d. synchronization means for synchronizing the video layer and the interpretation layer;

e. the interpretation layer overlaid on the video layer so as to be displayed and run concurrently and in synchronization therewith; and

f. an index manager for indexing of the comments or the select portions, or both, within a database; and

g. whereby a diagnostic or teaching session can be created comprising playback of the series of images concurrently with image portions selected by gestures and the comments associated with the gestures.

18. The diagnostic aid or teaching system of claim 17 further comprising:

storage means for recording activity within the interpretation application as synchronized to the video application.

19. A method of producing a teaching session with in-motion video comprising:

a. displaying frames or inserted frames of full motion image data;

b. selecting and recording subframe portions of the image data;

c. appending comments to the selected subframe portions of the image data;

steps “

a”

, “

b”

, and “

c”

making a session;

d. recording the session; and

e. indexing the session.

20. The method of producing a teaching session with in-motion video of claim 19, further comprising:

synchronizing the selecting and recording step and the appending step to the display of the full motion image data.

21. The method of producing a teaching session with in-motion video of claim 20, further comprising:

step a being performed in a separate application from steps b and c to prevent disturbing the full motion image data.

22. The method of producing a teaching session with in-motion video of claim 20, further comprising:

performing steps b) and c) during display of the full motion image data.

23. The method of producing a teaching session with in-motion video of claim 20, further comprising:

playing back the recorded session.

24. The method of producing a teaching session with in-motion video of claim 20, further comprising:

displaying more than one session at a time as inserted frames.

25. A diagnostic and teaching aid method including the steps of:

a. displaying a sequence of images;

b. selecting a portion of one image of the sequence of images via gestures of a diagnostician;

c. recording a commentary of the diagnostician concerning the image portion selection;

d. linking a recorded commentary concerning an image portion selection to an image portion selection;

e. recording a sequence of a plurality of image portion selections to create a teaching session;

f. indexing at least one of the commentary and selected image portions so as to make the teaching session retrievable through an index; and

g. playing back the sequence of images while displaying the plurality of image portion selections and commentary concerning the image portion selections, the play back including the sequence of image portion selection.