Audio/video storage and retrieval for multimedia workstations

• US 6,320,588 B1
• Filed: 04/18/1997
• Issued: 11/20/2001
• Est. Priority Date: 06/03/1992
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Compaq Computer Corporation (HP Inc.)
• Inventors: Palmer, Larry G., Palmer, Ricky S.
• Primary Examiner(s): Amsbury, Wayne
• Assistant Examiner(s): JUNG, DAVID YIUK
• Application Number:
  US08/844,688
• Time in Patent Office: 1,677 Days
• Field of Search: 707/1-540, 345/330, 345/333, 345/334, 345/473, 348/484-515, 348/473, 709/200-212, 725/109-110
• US Class Current: 345/473
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 21/4381 Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

An apparatus and method for storing and retrieving synchronized audio/video “filmclips” to and from a data file of a multimedia computer workstation includes a storage means for a workstation to store audio and video data as digital data packets to the data file, and retrieval means for the workstation to retrieve audio and video data from the data file. The video data is presented as an image on the display of the workstation, while the audio data is sent to either amplified speakers or headphones. An audio data stream is stored to the data file such that the audio data can be retrieved from the data file and reconstructed into a continuous audio signal. The video data is stored to the data file such that each frame of video data is inserted into the stored audio data stream without affecting the continuity of the audio signal reconstructed by the workstation. Timing information is attached to each frame of video data stored to the file, and indicates a point in the continuous audio data stream which corresponds in time to the frame of video data. A synchronizer displays a frame of video data when the point in the audio data stream, corresponding to the timing information of the retrieved video frame is audibly reproduced by the workstation. The invention also features a video teleconferencing “answering machine” which allows a user to leave an audio/video “filmclip” message on another workstation.

Claims

1. A method for storing and retrieving audio and video data on a multimedia workstation, comprising the steps of

a) storing an audio data stream from a workstation to a data file accessible by the workstation such that the audio data stream can be retrieved from the data file and reconstructed by the workstation into a continuous audio signal;

b) storing a non-continuous logical stream of video data from the workstation to the data file such that each frame of video data stored to the data file is inserted into the audio data stream also stored in the data file by the workstation without affecting the continuity of the audio signal when later retrieved from the data file and reconstructed by the workstation, said noncontinuous logical stream of video data being available video data remaining after portions of an initial video data stream have been discarded, while the audio data stream as stored in the data file remains unbroken; and

c) attaching timing information to each frame of video data stored to the data file indicating a point in the audio data stream also stored to the data file which corresponds in time to the frame of video data, such that the stored available video data in the form of a noncontinuous logical stream of video data is realigned with the stored unbroken audio data stream.

2. A method for storing and retrieving audio and video data as described in claim 1 wherein the non-continuous video data results from network congestion.

3. The method of claim 1 further comprising the step of

displaying a frame of video data, retrieved from the data file, on the display of the workstation when the point in the audio data stream, retrieved from the data file, corresponding to the timing information of the retrieved video frame is audibly reproduced by the workstation.

4. The method of claim 3 further comprising the steps of

a) the workstation internally generating timing information by counting the amount of audio data in the audio stream retrieved from the data file;

and

b) the workstation comparing the internally generated timing information with the timing information stored in the data file corresponding to the frame of video data most recently retrieved from the data file to determine when to display the most recently retrieved frame of video data on the display of the workstation.

5. A multimedia workstation, comprising:

a) storage means for a workstation to store a stream of audio data packages and a corresponding non-continuous logical stream of video data as digital data packets to a data file accessible by the workstation as audio and video data packets, the video packets including timing information such that the stored available video data in the form of a noncontinuous logical stream of video data can be realigned with a stored unbroken audio data stream, said noncontinuous logical stream of video data being available video data remaining after portions of an initial video data stream have been discarded, while the audio data stream as stored in the data file remains unbroken; and

b) retrieval means for the workstation to retrieve audio and video data from the data file which was stored as digital data packets by the storage means.

6. A multimedia workstation as described in claim 5 wherein the non-continuous video data results from network congestion.

7. The multimedia workstation of claim 5, wherein

the retrieval means retrieves the data packets from the data file and reconstructs the audio and video data stored by the storage process for audible and visual reproduction, respectively.

8. The multimedia workstation of claim 7 wherein

a) the storage means further comprises means for attaching timing information to each frame of video data stored to the data file indicating a point in the audio data stream also stored to the data file which corresponds in time to the frame of video data;

and

b) the retrieval means further comprises synchronization means for displaying a frame of video data, retrieved from the data file, on the display of the workstation when the point in the audio data stream, retrieved from the data file, corresponding to the timing information of the retrieved video frame is audibly reproduced by the workstation.

9. The multimedia workstation of claim 8 wherein

a) the retrieval means further comprises timing means for generating timing information by counting the amount of audio data in the audio stream retrieved from the data file;

and

b) the synchronization means further comprises comparison means for comparing the internally generated timing information with the timing information stored in the data file corresponding to the frame of video data most recently retrieved from the data file to determine when to display the most recently retrieved frame of video data on the display of the workstation.

10. The multimedia workstation of claim 5 wherein the data file is accessed using a TCP/IP protocol data socket.

11. A multimedia computer workstation, comprising

a video source that provides sequential frames of digitized video data;

an audio source that provides a digitized audio data stream that represents a continuous audio signal synchronized to the sequential frames of digitized video data;

data storage capable of storing a non-continuous logical stream of video data with a corresponding stream of audio data to a data file such that each frame of video data stored to the data file is sequentially inserted into the audio data stream without affecting the continuity of the audio signal represented by the audio data stream, said noncontinuous logical stream of video data being available video data remaining after portions of an initial video data stream have been discarded, while the audio data stream as stored in the data file remains unbroken;

means for audibly reproducing retrieved audio data stream into a continuous audio signal; and

a display that displays retrieved sequential frames of video data on the display of the workstation synchronized to the reproduced audio signal such that the stored available video data in the form of a noncontinuous logical stream of video data is realigned with the stored unbroken audio data stream.

12. The multimedia workstation of claim 11, wherein the video source comprises a video camera providing sequential frames of live analog video, and frame capture device for capturing, digitizing, and storing each sequential frame of live analog video.

13. The multimedia workstation of claim 11, wherein the video source comprises means for providing pre-recorded sequential frames of analog video, and frame capture means for capturing, digitizing, and storing each sequential frame of pre-recorded analog video.

14. The multimedia workstation of claim 13, wherein the video source comprises a video tape recorder.

15. The multimedia workstation of claim 13, wherein the video source comprises a video laser disk player.

16. The multimedia workstation of claim 11 wherein the audio source comprises a microphone for transducing live audio into an analog audio signal, and audio digitizer and storage for digitizing and storing the analog signal.

17. The multimedia workstation of claim 16 wherein the audio digitizer and storage applies mu-law compression in digitizing the analog audio signal.

18. The multimedia workstation of claim 11 wherein the audio source comprises means for providing pre-recorded audio signals corresponding to sequential frames of pre-recorded video.

19. The multimedia workstation of claim 11 wherein the data file is accessed using a TCP/IP protocol data socket.

20. The multimedia workstation of claim 11 wherein the storage comprises video compressor for reducing the quantity of data required to be stored to the data file to represent each frame of video data.

21. The multimedia workstation of claim 20 wherein the video compressor applies JPEG compression to the video data.

22. The multimedia workstation of claim 20 wherein the video compressor applies MPEG compression to the video data.

23. The multimedia workstation of claim 11 wherein the workstation comprises an IBM compatible personal computer.

24. The multimedia workstation of claim 11 wherein the workstation comprises a RISC processor.

25. A multimedia computer workstation as described in claim 11 wherein the non-continuous video data results from network congestion.

26. A method of processing audio/video data information, the method comprising the steps of:

capturing time-related audio and video data for transmission over a logical connection between two workstations coupled to a network;

at a transmitting workstation, digitizing the audio and video data into packets for transmission over a logical connection to a target workstation;

transmitting timing information to the target workstation for reconstructing the time-related audio and video data transmitted over the logical connection; and

monitoring a throughput capacity of the network and discontinuing transmission of a logical stream of the video data during periods of congestion, yet continuing to transmit a sensibly continuous logical stream of audio data for reconstruction at the target workstation, said timing information enabling realignment of available transmitted video data with the transmitted continuous audio data.

27. A method as in claim 26, wherein each of the workstations is capable of sonically reproducing the audio data and visually reproducing video data received from another workstation.

28. A method as in claim 26, wherein the transmitting workstation discontinues transmission of video data if a digitized block of video information is not transmitted over the network within a specified timeout period.

29. A method as in claim 26, wherein the audio and video data is initially encoded based on an NTSC (National Television System Committee) standard that is converted to digital data packets for transmission over the network.

30. A method as in claim 26, wherein the audio and video data is transmitted to a target workstation based on a TCP/IP protocol.

31. A method as in claim 26 further comprising the step of:

receiving the audio and video data at the target workstation and reconstructing an originally captured image and sound for a user.

32. A method as in claim 26, wherein the audio and video data sent across the network includes frames of video information inserted into a sensibly continuous stream of audio data.

33. A method as in claim 26, wherein the target workstation plays the sensibly continuous audio stream and time-related corresponding received frames of video data.

34. A method as in claim 26, wherein the video data is video frame information digitized into data packets comprising less than 32 kilobits of data.

35. A method as in claim 26 further comprising the steps of:

discarding video data when network congestion is detected so that newer video data can be transmitted to the target workstation.

36. A method as in claim 35 further comprising the steps of:

at the target workstation, synchronizing a playback of received audio data with received video data regardless of whether video data is discarded.

37. A method as in claim 26 further comprising:

storing the audio and video data in memory.

38. A method as in claim 26, wherein a transmission of audio and video data over a logical connection of the network supports a teleconference session.

39. A method as in claim 26 further comprising:

continuing transmission of a logical steam of video data when a logical connection supports appropriate throughput.

40. A method as in claim 39, wherein the video data is current video data.