Method and system for recording and indexing audio and video conference calls allowing topic-based notification and navigation of recordings

• US 7,466,334 B1
• Filed: 09/17/2002
• Issued: 12/16/2008
• Est. Priority Date: 09/17/2002
• Status: Expired due to Fees

• Current Assignee: Commfore Corporation
• Original Assignee: Commfore Corporation
• Inventors: Baba, Michael
• Primary Examiner(s): Woo; Stella L
• Application Number:
  US10/244,684
• Time in Patent Office: 2,282 Days
• Field of Search: 348/14.06, 348/14.08, 348/14.09
• US Class Current: 348/14.06
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Recording and indexing audio and video conference calls allows topic-based notification, navigation and playback of one or many recordings. An external conferencing system with five internal modules, recording, indexing, management, notification, and playback allows a plurality of meeting participants to record a conference call, and a plurality of subscribers to access, search, and be notified of the occurrence of an event of interest. The system automatically creates contextual or topic based time intervals which are used to immediately notify a subscriber of an event occurrence. The subscriber can use the time stamps of the topic intervals to navigate and playback the recording. The system also indexes project management files and correlates recording intervals to project tasks, and subsequently correlates multiple recording intervals.

Claims

1. A system for recording and indexing audio and video conference calls comprising, a configuration subsystem, at least one recording subsystem connected to the configuration subsystem for connecting an external audio or video conferencing system, at least one indexing subsystem connected to the at least one recording subsystem for automatically determining topic based intervals, at least one management subsystem connected to the at least one indexing subsystem for storing results of the indexing subsystem, at least one notification and navigation system connected to the at least one management subsystem for allowing a plurality of subscribers to query the at least one management subsystem and to submit requests to be notified of an occurrence of a topic based event, and at least one playback subsystem connected to the at least one recording subsystem for storing recorded conference calls in a variety of formats suitable for playback on a variety of devices.

2. The system of claim 1, wherein the recording subsystem further comprises a control module, a conference menu module connected to the control module, a segmentation buffer connected to the recording module, a recording module connected to the segmentation buffer, and an encoding module connected to the control module.

3. The system of claim 2, wherein the recording module automatically segments conference recordings into discrete, continuous segments based on parameters specified by the control module.

4. The system of claim 2, wherein the recording module passes recorded segments to the encoding module via the segmentation buffer for encoding the segments in a format for playback by a media server.

5. The system of claim 2, wherein the conference menu module is connected to the external conferencing system, and generates voice or text menu prompts for allowing conference leaders to control the recording subsystem.

6. The system of claim 5, wherein the control module responds to Dual Tone Multi Frequency (DTMF) commands generated by the conferencing system.

7. The system of claim 5, wherein the control module responds to voice input by a conference leader through the conferencing system.

8. The system of claim 2, wherein the control module in response to a command from the external audio or video conferencing systems, controls recording and indexing of a conference call.

9. The system of claim 2, wherein the segmentation buffer holds recording segments until the indexing subsystem retrieves and has finished processing the segments.

10. The system of claim 2, wherein the control module in response to user prompts passes user data to the management subsystem.

11. The system of claim 1, wherein indexing subsystem further comprises a control module, a speech recognition indexing module connected to the control module, a phonic indexing module connected to the control module, a project task analysis module connected to the control module, and a contextual analysis module connected to the control module.

12. The system of claim 11 further comprising a speech recognition notification module connected to the speech recognition indexing module, a phonic recognition notification module connected to the phonic indexing module, and a contextual notification module connected to the contextual analysis module.

13. The system of claim 12, wherein the notification modules are connected to the notification and navigation subsystem, and upon recognition of a recognized word, phonic or topic send a message that contains the identified word, phonic or topic to a notification filter.

14. The system of claim 11, wherein the control module is connected to the recording subsystem for responding to commands from the recording subsystem for starting and stopping indexing of a recording segment.

15. The system of claim 11, wherein the speech recognition indexing module is connected to the recording subsystem for indexing recording segments and populating a database of recognized words and timestamps in the management subsystem.

16. The system of claim 15, further comprising a general co-occurrence database in the management subsystem and wherein the contextual analysis module automatically creates topic-based recording interval using recognized words from the speech recognition module and the general co-occurrence database in the management subsystem.

17. The system of claim 16, wherein the context analysis module further calculates a context-vector for each topic-interval.

18. The system of claim 11, wherein the phonic indexing module analyzes recording segments and populates a database of identified phonics and corresponding timestamps in the management subsystem.

19. The system of claim 11, further comprising a project database in the management subsystem and wherein the project task analysis module parses project file and stores task information in the project database in the management subsystem.

20. The system of claim 19, wherein the project task analysis module further creates a context-vector for each extracted task.

21. The system of claim 20, wherein the project task analysis module further calculates a task-task weighting.

22. The system of claim 21, wherein the project task analysis module further calculates a task-task weighting, and wherein the weighting is derived from task-subtask relationship, task dependencies, task resource assignments, and/or task scheduling correlations.

23. The systems of claim 20, wherein the project task analysis module further combines the context-vectors of the recording intervals with the context vectors of the project task to create an interval-task correlation factor, and stores the correlation factor in a database in the management system.

24. The system of claim 23, wherein the project task analysis module further calculates interval-to-interval associations as a product of two interval-task correlation factors for each pair of interval-task associations having a common task, and stores the interval-to-interval association factors in a database in the management system.

25. The system of claim 1, wherein the management subsystem further comprises a control module, a user access database connected to the user access database, a notification database connected to the control module, a configuration database connected to the user access database, a system database connected to the control module for containing tables and records for recording information, recording location, project tasks, recording intervals, recognized words, recognized phonics, interval-task correlation factors, and interval-to-interval correlation factors.

26. The system of claim 25, wherein the management subsystem further comprises a request parser module connected to the control module, a phonic match module, a speech recognition match module, and a contextual match module, all connected to the request parser, and to a result merger module connected to the user access database.

27. The system of claim 26, wherein the management subsystem further comprises a navigation request match module connected to the request parser.

28. The system of claim 26, wherein the request parser module responds to requests from the notification and navigation subsystem and depending upon the request type passes the request to the appropriate match module.

29. The system of claim 28, wherein said match modules determine a set of matching phonics, recognized words or topics by comparison with the set of phonics, recognized words or topics stored in the system database of the management system.

30. The system of claim 26, wherein the result merger module takes output of one or more of the match modules creating a merged set of results.

31. The system of claim 30, further comprising a user access database, and wherein the merged results are filtered by checking the user access database, so that only results that the requester is allowed access are returned to the user.

32. The system of claim 31, wherein filtered results set are ordered and graphically represented using a matrix format, with a cell of the matrix populated with an playback control icon.

33. The system of claim 1, wherein the notification and navigation subsystem further comprises a control module, a query module connected to the control module, a navigation module connected to the control module, and a notification module connected to the control module.

34. The system of claim 33, further comprising a notification database in the management system and plural notification filters in the management system, and wherein the notification module in response to a notification request from a subscriber, stores the notification parameters in the notification database in the management system, and further passes the notification parameters to each of the plural notification filters in the management system.

35. The system of claim 33, wherein upon reception of messages indicating that an event of interest has occurred, the notification module passes notification to each and every subscriber who has previously requested to be notified, and who has subscribed to events having a set of conditions that match a current notification event.

36. The system of claim 33, further comprising a request parser of the management system and wherein the query module accepts query requests from subscribers and passes the query request to the request parser of the management system.

37. The system of claim 36, wherein the navigation module accepts navigation requests from subscribers and passes the navigation request to the request parser of the management system.

38. The system of claim 37, wherein the navigation module communicates with the playback subsystem for initiating or controlling playback of a recording.

39. The system of claim 33, wherein the control module, based on parameters entered by project mangers, controls settings used by the query module, notification module and navigation module.

40. The system of claim 1, where playback subsystem further comprises a control module, a media server connected to the control module, a media transport module connected to the control module, and a media storage module connected to the media transport and media server modules.

41. The system of claim 40, wherein the media transport module upon issuance of a command from the recording subsystem retrieves a recording segment and stores the segment in the media storage module.

42. The system of claim 40, wherein upon the issuance of a command from the notification and navigation system, the control module instructs the media server to retrieve the requested media from the media storage module for playback of media to a subscriber.

43. The system of claim 40, wherein upon issuance of a navigation command the control module instructs the media server to perform the command, and playback, pause or stop media at a timeframe identified in the command.

44. A method of automatically indexing audio or video conference calls for creating topic-based intervals comprising the steps of partitioning an audio input into one or more discrete segments, processing each of the segments using speech recognition, and saving recognized words and corresponding timestamping in a database, iterating over local sets of the recognized words, and looking up co-occurrence of the recognized words from a generic co-occurrence database, saving start and stop time of every interval where an entire set of recognized words have sufficiently high co-occurrence values, iterating over a set of saved intervals and creating a context-vector further comprising a vector representation of words in an interval within some defined N-space of words.

45. A method of automatically indexing project management files to create task context-vectors and task correlation values comprising the steps of iterating over each task in a project file and extracting task information including task name, description, comments, resource assignment, task dependencies and task schedule, and saving the task information in a task database, calculating a context-vector using the task name, the description and the comments for creating a vector representation of the task within an N-space of words, and saving the context-vector in a context-vector database, calculating a task-to-task correlation weighting based on the relationship between tasks, and saving the correlation weighting in a correlation weighting database.

46. A method of determining recording interval to task relationships comprising the steps of iterating over a set of intervals determined using the method of automatically indexing audio or video conference calls for creating topic-based intervals comprising the steps of partitioning an audio input into one or more discrete segments, processing each of the segments using speech recognition, and saving recognized words and corresponding timestamping in a database, iterating local sets of the recognized wards, and looking up co-occurrence of the recognized words from a generic co-occurrence database, saving start and stop time of every interval where an entire set of recognized words have sufficiently high co-occurrence values, iterating over a set of saved intervals and creating a context-vector further comprising a vector representation of words in an interval within some defined N-space of words, and for each interval further iterating over each task determined using the method of automatically indexing project management files to task context-vectors and task correlation values comprising the steps of iterating over each task in a project file and extracting task information including task name, description, comments, resource assignment, task dependencies and task schedule, and saving the task information in a task database, calculating a context-vector using the task name, the description and the comments for creating a vector representation of the task within an N-space of words, and saving the context-vector in a context-vector database, calculating a task-to-task correlation weighting based on the relationship between tasks, and saving the correlation weighting in a correlation weighting database, and determining interval task pairs, for each interval task pair, obtaining a dot product of the context-vectors from recording intervals and the context-vectors of the task, storing interval-task correlation values in a database.

47. The method of claim 46, further comprising determining correlation of one recording interval to another recording interval including the steps of for each pair of interval-task correlations calculated, further taking the weighted dot-product of the interval-task corrections and task weight correlation values from the correlation weighting database, storing the results in a recording interval correlation database.

48. The method of claim 47, further comprising searching a collection of recording that have been indexed using the method of determining correlation of one recording interval to another recording interval including the steps of receiving a search request from a user in either text of vocal form, if in a vocal form audio input, processing the audio input using a speech recognition application resulting a an equivalent input text string, passing the input text string or an equivalent to a match module for comparing with the results of the recognized words from method of determining correlation of one recording interval to another recording interval, further processing the audio input for determining acoustical or phonic fingerprints and passing the fingerprints to a phone match module, for every word or phonic match of sufficient confidence, determining a time code when the match occurred within the recording, for each time code, looking-up all recording topic intervals calculated in determining correlation of one recording interval to another recording interval that overlap or span the time code, for each interval finding correlated tasks from the project analysis index, for each correlated task, further finding other recording intervals that are also correlated to the same task, merging a set of located recording intervals, filtering a result set so only results that are accessible by the requestor are returned, organizing and displaying the filtered result set.

49. The method of claim 48, further comprising organizing and displaying results of a query determined in the method including creating a matrix having rows, columns and cells, displaying task hierarchy or other such classification scheming in the rows, displaying a temporal timeframe, for discrete recording events as the columns, for each cell that has a significant interval-task correlation value, displaying one or more playback control icon having one or all of the following elements:

the interval start and stop time and duration, recording data for the interval, and a control object capable of sending start, stop, pause, fast-forward or rewind commands to a media server to control the playback of the recording interval.