Concurrent voice to text and sketch processing with synchronized replay
First Claim
1. A method for capturing, synchronizing, and replaying a sketching activity and media information associated to said sketching activity, said method comprising:
- a) simultaneously timestamped capturing said sketching activity and said media information;
wherein said sketching activity producing one or more sketch objects, each having a corresponding sketch object timestamp, wherein said media information includes audio data, and wherein said audio data are captured in an audio file;
b) transcribing said audio file, recording respectively recognized keywords or phrases and their corresponding timestamps;
c) converting all timestamps associated to said one or more sketch objects, said audio file, and said keywords or phrases to a common time base;
d) enabling a user to select a starting point for replay, said starting point is one or more of said sketch objects, a keyword, or a phrase;
e) based on said starting point, synchronizing said one or more sketch objects, said audio file, and said keywords or phrases, utilizing their respective corresponding common base timestamps; and
f) replaying said sketching activity and said media information based on said starting point.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention facilitates transparent and cost effective capture, sharing, and re-use of knowledge in informal media such as audio, video, and individual sketching actions. Sketches, handwritings, verbal comments and the likes captured during a session are timestamped and, once completed, automatically indexed and published on a web server that allows for real time, streamed and synchronized replay thereof. All timestamps are converted to a common time base. To control the synchronous replay, multiple treads collaborate based on common base timestamps corresponding to the sketching actions and correlated audio/video stream(s). An enhanced interactive graphical user interface enables a network user to create and capture a session, e.g., annotate an imported image with sketches and verbal comments. The user or others can access the published session in real time and select a keyword or an individual sketch entity from which to begin replay. Transcribed audio stream can also be synchronously replayed.
69 Citations
20 Claims
-
1. A method for capturing, synchronizing, and replaying a sketching activity and media information associated to said sketching activity, said method comprising:
-
a) simultaneously timestamped capturing said sketching activity and said media information;
whereinsaid sketching activity producing one or more sketch objects, each having a corresponding sketch object timestamp, wherein said media information includes audio data, and wherein said audio data are captured in an audio file;
b) transcribing said audio file, recording respectively recognized keywords or phrases and their corresponding timestamps;
c) converting all timestamps associated to said one or more sketch objects, said audio file, and said keywords or phrases to a common time base;
d) enabling a user to select a starting point for replay, said starting point is one or more of said sketch objects, a keyword, or a phrase;
e) based on said starting point, synchronizing said one or more sketch objects, said audio file, and said keywords or phrases, utilizing their respective corresponding common base timestamps; and
f) replaying said sketching activity and said media information based on said starting point. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A system for real time capturing and synchronous replaying a sketching activity and related media information, said system comprising:
-
means for capturing said sketching activity and said media information simultaneously;
wherein said media information includes audio signals and video signals and said sketching activity producing a plurality of sketch entities;
means for encoding said sketch entities with corresponding sketch object timestamps and said media information with corresponding media timestamps;
means for converting said sketch object timestamps to sketch object common base timestamps and said media timestamps to media common base timestamps;
means for comparing said sketch object common base timestamps and said media common base timestamps; and
an interactive graphical user interface for synchronously replaying said sketch activity and said media information;
- View Dependent Claims (14, 15, 16, 17, 18, 19, 20)
-
Specification