Method and apparatus for recording and indexing an audio and multimedia conference

• US 5,710,591 A
• Filed: 06/27/1995
• Issued: 01/20/1998
• Est. Priority Date: 06/27/1995
• Status: Expired due to Term

• Current Assignee: RPX Corporation
• Original Assignee: AT&T, Inc.
• Inventors: Katseff, Howard P., Markowitz, Robert E., Robinson, Bethany S., Shahraray, Behzad, Stuntebeck, Peter H., Weber, Roy P., Bruno, Richard F., Gibbon, David C.
• Primary Examiner(s): WOO, STELLA L
• Application Number:
  US08/495,747
• Time in Patent Office: 938 Days
• Field of Search: 348/15, 348/14, 348/16, 348/17, 348/18, 348/19, 370/62, 379/96, 379/93, 379/94, 379/202, 379/204, 379/205, 379/206
• US Class Current: 348/14.09
• CPC Class Codes: H04M 2201/41 using speaker recognition
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method and apparatus for recording and indexing audio information exchanged during an audio conference call, or video, audio and data information exchanged during a multimedia conference. For a multimedia conference, the method and apparatus utilize the voice activated switching functionality of a multipoint control unit (MCU) to provide a video signal, which is input to the MCU from a workstation from which an audio signal is detected, to each of the other workstations participating in the conference. A workstation and/or participant-identifying signal generated by the multipoint control unit is stored, together or in correspondence with the audio signal and video information, for subsequent ready retrieval of the stored multimedia information. For an audio conference, a computer is connected to an audio bridge for recording the audio information along with an identification signal for correlating each conference participant with that participant'"'"'s statements.

Claims

1. A method for recording and indexing information exchanged during a multimedia conference having a plurality of participants at a plurality of spaced apart terminal devices and interacting through a multimedia bridge connected to a computer, each of the terminal devices having audio input means for inputting an audio signal to the each terminal device, and at least one of the each terminal devices having a video input means for inputting a video signal to the at least one terminal device and a data accessing means for accessing stored data files during the multimedia conference, said method comprising the steps of:

detecting at the computer an audio signal input to the audio input means of the at least one terminal device;

identifying the at least one terminal device from which the detected audio signal originated and generating a location signal representative of the identified terminal device; and

recording the detected audio signal, an address signal identifying any stored data file accessed during the multimedia conference by the at least one terminal device, at least a portion of the video signal and said location signal identifying the at least one terminal device from which the detected audio signal originated to correlate the recorded detected audio signal, the recorded portion of the video signal, the stored data file address signal and the recorded location signal with each other.

2. The method of claim 1, further comprising the step of determining an identity of each participant to the multimedia conference, and wherein said recording step further comprises recording a signal indicative of the determined identity.

3. The method of claim 1, further comprising the step of synchronizing the recorded location signal and at least one of the recorded portion of the video signal, the recorded data file address and the recorded audio signal to a reference clock signal by recording a representation of the reference clock signal in said recording step.

4. The method of claim 1, wherein a plurality of participants are present at said at least one terminal device, the method further comprising the steps of identifying from said detected audio signal a speaking participant of said plurality of participants at said at least one terminal device from whom the detected audio signal originates, and recording in said recording step a signal identifying the speaking participant.

5. The method of claim 1, further comprising the step of providing a transcription unit connected to the computer for generating a transcript of a multimedia conference by utilizing said location signal to correlate the recorded detected audio signal with the location of the terminal device from which the audio signal originated.

6. A method for recording and indexing information exchanged during a multimedia conference having a plurality of participants interacting with a multipoint control unit (MCU) through a plurality of terminal devices at spaced apart locations, each of the terminal devices having at least audio input means for inputting audio signals to the each terminal device, and at least one of the said each terminal devices having a video input means for inputting a video signal to the at least one terminal device and a data accessing means for accessing stored data files during the multimedia conference, said method comprising the steps of:

detecting at the MCU an audio signal input to one of the audio input means of the at least one terminal device;

identifying the at least one terminal device from which said detected audio signal originated and generating a location signal representative of the identified terminal device; and

recording said detected audio signal, an address signal identifying any stored data file accessed during the multimedia conference by the at least one terminal device, at least portion of said video signal, and said location signal identifying the at least one terminal device from which said audio signal originated to correlate the recorded detected audio signal, the at least a portion of the video signal, the stored data file address signal and the recorded location signal with each other.

7. The method of claim 6, wherein at least one additional of said terminal devices further comprise video input means for inputting video information and wherein said detected audio signal is associated with video information input to the MCU from the at least one terminal device by the video input means of the at least one terminal device, said method further comprising the step of transmitting to said at least one additional terminal device, for as long as said detected audio signal continues, the video signal associated with said detected audio signal.

8. The method of claim 7, further comprising the step of synchronizing the recorded location signal and at least one of the recorded portion of the video signal, the recorded data file address and the recorded audio signal to a reference clock signal by recording a representation of the reference clock signal in said recording step.

9. The method of claim 8, further comprising the step of time stamping the recorded audio signal, the at least a portion of the video signal, the stored data file address signal and the recorded location signal for subsequent retrieval.

10. The method of claim 7, further comprising the step of sampling the video signal input from the video input means, and wherein said recording step further comprises recording the sampled video signal.

11. The method of claim 10, further comprising the step of synchronizing the recorded location signal, audio signal, address signal and sampled video information to a reference clock signal received by the multipoint control unit by recording a representation of the reference clock signal in said recording step.

12. The method of claim 11, further comprising the step of time stamping the recorded audio signal, location signal, address signal and sampled video information for subsequent retrieval.

13. The method of claim 10, wherein said recording step further comprises sampling the video in a content-based manner.

14. The method of claim 10, wherein said recording step further comprises sampling the video signal at a predetermined rate.

15. The method of claim 7, wherein a plurality of participants are present at said one terminal device, the method further comprising the steps of identifying from said detected audio signal which participant of said plurality of participants at said one terminal device is speaking, and wherein said recording step further comprises recording a signal identifying the speaking participant.

16. The method of claim 7, wherein a voice-only terminal device is connected to said multipoint control unit for participation in said multimedia conference by an additional participant at the voice-only terminal device, said method further comprising the steps of identifying a stored image of the additional participant and displaying the stored video image of the additional participant on the plurality of terminal devices having audio and video input means when the voice of the additional participant is detected by the MCU.

17. The method of claim 6, further comprising the step of providing a transcription unit connected to the MCU for generating a transcript of a multimedia conference by utilizing said location signal to correlate the recorded detected audio signal, the address signal and the at least a portion of said video signal, with the terminal device from which the audio signal originated.

18. An apparatus for recording and indexing information exchanged during a multimedia conference having a plurality of participants communicating through a plurality of terminal devices at spaced apart locations, each said terminal device having an audio input means and at least one of the terminal devices having video input means for inputting a video signal to the at least one terminal device and data accessing means for accessing stored data files, said apparatus comprising:

a multimedia bridge connected to said plurality of terminal devices for receiving an audio signal and a video signal from the audio input means and video input means, respectively, of the at least one of the plural terminal devices;

means connected to said multimedia bridge for generating a location signal identifying the at least one terminal device; and

means connected to said multimedia bridge for recording said generated location signal, an address signal identifying any stored data file accessed during the multimedia conference by the at least one terminal device, at least a portion of the video signal, and the detected audio signal so as to correlate said detected audio signal, the address signal, the portion of the video signal, and the location signal identifying the at least one terminal device, with each other.

19. The apparatus of claim 18, further comprising means for synchronizing the recorded location signal, the address signal, the portion of the recorded video signal, and the recorded audio signal to a reference clock signal by recording a representation of the reference clock signal with the audio signal, the address signal, the video signal portion, and the location signal.

20. The apparatus of claim 18, further comprising means connected to said multimedia bridge for distinguishing between multiple participants present at a single one of the plurality of terminal devices using the audio signal originating at said single terminal device.

21. The apparatus of claim 20, wherein said generating means, recording means and distinguishing means comprise a digital computer.

22. The apparatus of claim 18, further comprising transcription means connected to said signal generating means for transcribing the recorded audio signal.

23. An apparatus for recording and indexing information exchanged during a multimedia conference having a plurality of participants communicating through a plurality of terminal devices at spaced apart locations, each said terminal device having at least an audio input means and at least one of the each terminal devices having a video input means for inputting a video signal to the at least one terminal device and a data accessing means for accessing stored data files during the multimedia, conference, said apparatus comprising:

a multipoint control unit (MCU) connected to said plurality of terminal devices, said MCU having means for detecting an audio signal and a video signal, respectively, input to said MCU from the audio input means of said at least one of the each terminal devices and for generating a location signal identifying the at least one terminal device; and

means connected to said MCU for recording said generated location signal, an address signal identifying any stored data file accessed during the multimedia conference by at least one terminal device, at least a portion of said video signal and the audio signal so as to correlate said audio signal, address signal, portion of said video signal and location signal with each other.

24. The apparatus of claim 23, wherein at least one additional of said terminal device has video input means and wherein said detected audio signal corresponds to video information input to said MCU from the video input means of said at least one terminal device, said MCU further comprising means for providing, upon detection of said detected audio signal and for as long as said audio signal is detected, the video signal to at least the one additional terminal device.

25. The device of claim 24, further comprising means for synchronizing the recorded location signal, recorded address signal, recorded portion of said video signal and recorded audio signal to a reference clock signal by recording a representation of the reference clock signal with said audio signal, said location signal, said address and said video signal portion.

26. The device of claim 25, further comprising means for time stamping the recorded audio signal, location signal and video information for subsequent retrieval.

27. The device of claim 24, further comprising means for sampling the video information input from the video input means, and wherein said recording means records the sampled video information.

28. The device of claim 27, wherein said sampling means samples the video information in a content-based manner.

29. The device of claim 27, wherein said sampling means samples the video information at a predetermined rate.

30. The device of claim 24, wherein said video input means comprises a video camera.

31. The device of claim 23, further comprising means connected to said multipoint control unit for distinguishing between multiple participants present at a single one of the plurality of terminal devices.

32. The device of claim 24, further comprising a connection line connected to said MCU for connecting a voice-only terminal device to said MCU for facilitating multimedia conference participation by a user of the voice-only terminal device.

33. The device of claim 24, further comprising transcription means connected to said MCU for transcribing the detected audio signal input to said MCU during a multimedia conference.