Method and computer program product for synchronizing, displaying, and providing access to data collected from various media

• US 8,275,243 B2
• Filed: 08/31/2007
• Issued: 09/25/2012
• Est. Priority Date: 08/31/2006
• Status: Active Grant

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Kientz, Julie, Boring, Sebastian, Abowd, Gregory
• Primary Examiner(s): HARVEY, DAVID E
• Application Number:
  US11/848,522
• Publication Number:
  US 20080063363A1
• Time in Patent Office: 1,852 Days
• Field of Search: 386/239, 386/241, 386/240
• US Class Current: 386/241
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

A method and computer program product for synchronizing, displaying and providing access to data collected from various media. The method for processing and displaying data from a plurality of media comprises obtaining handwriting data using an electronic writing instrument, recording video data, synchronizing the video data and the handwriting data, and displaying in a single screen the video data, a timeline of the video data, and handwriting data, the timeline indexed according to discrete periods of time. The computer program product comprises instructions for recording time-stamped video and audio data, obtaining time-stamped handwriting data entries from an electronic writing instrument, synchronizing and indexing the video and audio data with the handwriting data, and displaying in a single screen, a first region comprising the handwriting data, a second region comprising the video and audio data, and a third region comprising a timeline of the video and audio data.

Claims

1. A method for processing and displaying data from a plurality of media used during a session, comprising:

obtaining handwriting data using an electronic writing instrument, wherein the handwriting data comprises a handwriting time-stamp recorded during the session;

recording video data, wherein the video data comprises a video time-stamp recorded during the session;

synchronizing the video data and the handwriting data;

displaying in a single screen the video data, a timeline of the video data, and handwriting data, the timeline of the video data being indexed according to discrete periods of time;

initiating a program on a computing device;

selecting faculties to be developed;

printing at least one form comprising a list of tasks designed to develop at least one selected faculty, and a unique pattern representative of the vertical and horizontal coordinates of the paper;

performing separate exercises of each task during the session;

writing a result for each exercise on the form using the electronic writing instrument; and

recording video data comprising recording video of a subject performing each task;

wherein displaying the timeline of the video data comprises displaying a discrete segment on the timeline representative of the video data of the subject performing a discrete task.

2. The method of claim 1,

wherein obtaining handwriting data comprises using the electronic writing instrument to record data of the unique pattern of the area of the paper marked by the electronic writing instrument.

3. The method of claim 1, further comprising:

time-stamping each marking made on the form using the electronic writing instrument; and

time-stamping the video data recorded during the session.

4. The method of claim 1, further comprising:

storing an image of the printed form;

comparing the handwriting data with the stored image of the form to determine the marked areas of the form; and

determining the result of each separate exercise by analyzing the marked area of the form.

5. The method of claim 3, further comprising:

synchronizing the video and handwriting data using the time-stamps of the video and handwriting data and a predetermined amount of time necessary to execute each task performed during the session.

6. The method of claim 1, further comprising displaying the handwriting data in a table having a layout corresponding to the form, each discrete handwriting data entry separately displayed as a symbol representative of the symbol recorded on the form during the session.

7. The method of claim 1, further comprising:

downloading the video data and handwriting data to the computing device; and

displaying in a single screen the video and handwriting data, indexed according to separate exercises conducted during a session.

8. The method of claim 1, further comprising:

recording separate audio data; and

synchronizing the video data and handwriting data comprising using voice recognition to associate the handwriting data with portions of the video data.

9. A non-transitory computer readable medium having computer readable instructions stored thereon for execution by a processor to perform a method for synchronizing, displaying, and providing access to data collected from various media used during a session comprising:

recording time-stamped video and audio data;

obtaining time-stamped handwriting data entries from an electronic writing instrument, wherein the handwriting data comprises a handwriting time-stamp recorded during the session;

synchronizing and indexing the video and audio data with the handwriting data;

displaying in a single screen, a first region comprising the handwriting data, a second region comprising the video and audio data, and a third region comprising a first timeline of the video and audio data;

simultaneously displaying video for a separate session; and

simultaneously displaying in a third region at least two timelines, including the first timeline, each representative of video data recorded during the separate sessions.

10. A non-transitory computer readable medium having computer readable instructions stored thereon for execution by a processor to perform a method for synchronizing, displaying, and providing access to data collected from various media used during a session comprising:

recording time-stamped video and audio data;

obtaining time-stamped handwriting data entries from an electronic writing instrument, wherein the handwriting data comprises a handwriting time-stamp recorded during the session;

synchronizing and indexing the video and audio data with the handwriting data;

displaying in a single screen, a first region comprising the handwriting data, a second region comprising the video and audio data, and a third region comprising a timeline of the video and audio data;

displaying a selectable list of faculties to be developed during a session; and

printing on a sheet of paper at least one form comprising a list of tasks designed to develop at least one selected faculty, and a unique pattern representative of the vertical and horizontal coordinates of the paper.

11. The non-transitory computer readable medium of claim 10, further comprising instructions for separately and selectably displaying in the first region each discrete handwriting data entry in a tabular format.

12. The non-transitory computer readable medium of claim 10, displaying the timeline comprising displaying segments on the timeline indexed to correspond to the handwriting data entries displayed in the first region.

13. The non-transitory computer readable medium of claim 10, further comprising instructions for synchronizing the video and audio data with the handwriting data by associating a discrete period of time with each handwriting data entry, representative of the time during which an event related to the handwriting data entry occurred.

14. The non-transitory computer readable medium of claim 13, further comprising instructions for displaying in the second region the video data representation of the discrete period of time associated with a handwriting data entry selected in the first region.

15. The non-transitory computer readable medium of claim 14, further comprising instructions for displaying in the second region a toolbar with selectable video control buttons for sequentially displaying video data representative of the discrete time periods associated with each discrete handwriting data entry.

16. The non-transitory computer readable medium of claim 10, further comprising instructions for displaying in the first region handwriting data from at least two individuals, recorded during at least two temporally separate sessions.

17. The non-transitory computer readable medium of claim 10, further comprising instructions for displaying in the first region handwriting data representative of results of a plurality of separate exercises of a plurality of separate tasks.

18. The non-transitory computer readable medium of claim 10, further comprising instructions for:

displaying in a separate screen a fourth region comprising a graph and a fifth region comprising the selectable list of faculties;

displaying in the fourth region a graph representing the results of each sessions during which a faculty was developed when the faculty is selected from the fifth region;

displaying in the fourth region a pop-up window comprising results of a particular session when the user hovers a cursor over a plot on the graph representing the particular session.