Scalable networked multimedia system and applications

• US 20050144284A1
• Filed: 08/31/2004
• Published: 06/30/2005
• Est. Priority Date: 11/04/1997
• Status: Abandoned Application

• Current Assignee: Pragmatus AV LLC (Pragmatus, LLC)
• Original Assignee: Collaboration Properties Incorporated
• Inventors: Lauwers, Chris, Vanderlippe, Richard W., Brown, William Blake, Vuong, Anh T., Lui, Richard, Applebaum, Daniel, Inn, Yul J., Ludwig, Lester, Burnett, Gerald
• Application Number:
  US10/931,651
• Publication Number:
  US 20050144284A1
• Time in Patent Office: Days
• US Class Current: 709/226
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A networked multimedia system comprises a plurality of workstations and at least one storage server. A signal path interconnects the workstations and the storage server. Each workstation includes video and audio reproduction capabilities, as well as video and audio capture capabilities. Any given storage server comprises a set of storage cells that operate under the direction of a storage cell manager. A storage cell may include one or more encoding and/or transcoding converters configured to convert or transform audio and video signals originating at a workstation into a form suitable for storage. A storage cell may further include one or more decoding converters configured to convert stored signals into a form suitable for audio and video signal reproduction at a workstation. Each storage cell additionally includes at least one storage device and storage device controller capable of storing, for later retrieval, signals generated by one or more converters. The storage cell controller responds to signals received from the workstations, and oversees the operation of the storage cells to facilitate the storage of converted audio and video signals in at least one file that can be simultaneously accessed by one or more application programs executing on one or more workstations.

Claims

1. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one converter configured to

i) convert audio/video signals,

(a) originating at any one of the workstations,

(b) into a form suitable for storage;

C) at least one storage CELL configured to

i) store,

(a) the converted audio/video signals

for later retrieval; and

D) at least one signal path,

i) interconnecting

(a) the workstations, the converter(s) and the storage cell(s), wherein system is configured to

i) store converted signals,

(a) onto the storage cell,

(b) such that the stored converted signals

can later be retrieved

i. under direction of one or more of the workstations;

5. The networked multimedia system of any one of claims 1, 3, or 4 wherein

A) the system is configured to

i) facilitate transfer

(a) of converted audio/video signals

(b) previously stored by a first storage CELL

(c) to another storage CELL

6. The networked multimedia system of any one of claims 1 to 4, wherein

A) the converter is at least one of

i) a transcoder, an encoder and a combination encoder/transcoder.

7. The networked multimedia system of any one of claims 1 to 4, wherein

A) retrieved signals can

i) result in audio/video reproduction

ii) at one or more of the workstations.

8. The networked multimedia system of any one of claims 1 to 4, wherein

A) the converted signals are stored

i) in at least one file, which can be accessed by

(a) more than one workstation at the same time and/or

(b) more than one application type.

9. The networked multimedia system of claims 8, wherein

A) multiple copies of each such file can exist.

10. The networked multimedia system of claim 9 further comprising:

A) a storage server defined by at least the following

i) at least one of the storage CELLs,

ii) at least one of the converters and

iii) at least one controller configured to do at least one of the group of

(a) determine whether signal conversion is to occur;

(b) if the system comprises more than one converter,

determine which of a plurality of converters will accomplish the conversion;

(c) if the system comprises more than one storage CELL,

determine which of the storage cells will store the converted signals;

(d) control the subsequent retrieval of the stored signals; and

(e) determine which copy of the file is accessed for retrieval.

11. The networked multimedia system of claim 10, wherein

A) the storage server

i) is physically distributed or decentralized across the system.

12. The networked multimedia system according to claim 8, wherein the system is further configured to

A) generate a sequence of graphics rendering events

i) associated with computer generated images and/or overlay graphics at a workstation, and

B) save the generated sequence.

13. The networked multimedia system according to claim 12, wherein

A) the generated sequence of graphics rendering events

i) can be played back to render computer generated images and/or overlay graphics

(a) at a workstation,

(b) together with a workstation-user related audio and video

(c) in a determinable temporal relationship, and

B) the system can

i) save the event sequence

(a) related to the displayed computer generated and/or overlay graphics images and/or the audio and video

ii) such that the event sequence can later be reproduced

(a) at a workstation

(b) in the determined temporal relationship.

14. The networked multimedia system according to claim 13, wherein

A) the at least one file includes

i) the saved sequence of graphics rendering events.

15. The networked multimedia system according to claim 6, further comprising:

A) at least one decoder, configured to

i) decode signals retrieved from storage.

16. The networked multimedia system of claims 8, wherein

A) the file is a metafile including

i) at least one of the group of an audio-video file, audio only file, video only file, a graphics event file, a window event file, an application startup event file, a bit map file, a postscript file, a graphics file, a synchronization file and a text file.

17. The networked multimedia system of claim 16, wherein the metafile further includes

i) at least one pointer referencing

(a) a predetermined file or group of files

on one or more of the storage CELLs

2. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

(a) video and audio capture capabilities;

B) at least one converter configured to

i) convert audio/video signals,

(a) originating at any one of the workstations,

ii) into a form suitable for storage;

C) at least one storage CELL including

i) at least one storage CELL,

ii) each storage CELL being configured to

(a) store,

for later retrieval,

signals

i. converted by at least one of the converters, and

(b) facilitate transfer

of previously stored audio/video signals

to another storage CELL

whether included in that or another storage CELL system; and

D) at least one signal path,

i) interconnecting

(a) the workstations, the converters and the storage CELLs, wherein the system is configured

ii) to co-operate with the storage CELL

iii) to manage

the storage and

transfer,

i. of the converted audio/video signals,

ii. among storage CELLs.

3. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one storage server

i) the storage server including

(a) at least one storage CELL having

at least one storage disk,

a storage disk controller

i. associated with each disk, and

at least one converter; and

(b) a storage CELL manager (AVSM), and

ii) configured to

(a) store,

for later retrieval,

audio/video signals

converted by the converter(s); and

C) at least one signal path,

i) interconnecting,

(a) the workstations and the storage server,

 

wherein the networked multimedia system is configured to

ii) store the converted audio/video signals by

(a) selecting any one of the group consisting of disks, disk controllers, converters, storage CELLs and storage CELL managers (AVSM), according to the following approach;

if a first converter

i. of a first CELL

ii. has reached its capacity or bandwidth limit,

iii. then a second converter, of the first CELL, is selected;

if a first disk controller

i. of the first CELL

ii. has reached its capacity or bandwidth limit

iii. then a second disk controller of the first CELL is selected;

if a first disk,

i. associated with a first of the disk controllers

ii. has reached its capacity or bandwidth limit,

iii. then a second disk associated with the first disk controller is selected;

if the first storage CELL

i. associated with a first storage CELL manager (AVSM)

ii. has reached its capacity or bandwidth limit,

iii. then a second storage CELL associated with the first storage CELL manager (AVSM) is selected; and

if the first storage CELL manager (AVSM) has reached its capacity,

i. then a second storage server, is selected, and

(b) storing the converted signals using each of the selected ones of the group.

4. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one converter arranged to

i) convert audio and/or video information,

(a) originating at a workstation,

(b) into a form suitable for storage;

C) a least one storage CELL, each including

i) at least one storage disk

(a) configured to receive and store,

for later retrieval, the converted audio/video signals,

D) at least one storage CELL manager (AVSM); and

E) at least one signal path,

i) interconnecting

(a) the workstations, the converter and the storage CELLs,

wherein the system is configured to perform any one of the group of functions consisting of;

ii) transferring

(a) the stored converted signals

(b) between any two storage disks of any storage CELL;

iii) transferring

(a) the stored converted signals

(b) between any two storage CELLs,

under control of a single storage CELL manager (AVSM); and

iv) transferring

(a) the stored converted signals

(b) between any two storage CELLs

respectively under control of different CELL managers (AVSM).

18. A method of using a networked multimedia system comprising the steps of:

A) capturing audio and video at

i) one or more workstations, each including

(a) video and audio reproduction capabilities, and

(b) video and audio capture capabilities;

B) converting the captured audio/video signals,

(a) into a form suitable for storage;

C) storing,

(a) the converted audio/video signals

(b) within a storage CELL,

(c) such that the stored converted signals

can later be retrieved

i. under direction of one or more of the workstations.

19. A method of using a networked multimedia system comprising the steps of:

A) capturing audio and video at

i) one or more workstations, each including

(a) video and audio reproduction capabilities, and

(b) video and audio capture capabilities;

B) converting captured audio/video signals,

i) into a form suitable for storage

ii) by using at least one converter;

C) storing

i) for later retrieval,

(a) signals

(b) converted by at least one of the converters,

ii) onto at least one storage CELL;

D) transferring

i) previously stored audio/video signals

(a) to another storage CELL+

whether included in that or another storage CELL;

E) managing

i) the storage and

ii) transfer,

(a) of the audio/video converted signals,

(b) among storage CELLs.

22. The method of any one of claims 19, 21 or 22 further comprising the step of:

A) transferring

i) the stored audio/video signals

ii) to another storage CELL.

23. The method of any one of claims 19 to 22, further comprising the step of:

A) reproducing

i) audio and/or video

ii) at one or more of the workstations

iii) based on the retrieved signals.

24. The method of any one of claims 19 to 22, wherein

A) converted signals are stored

i) in at least one file, which can be accessed by

(a) more than one workstation at the same time and/or

(b) more than one application type.

25. The method of claim 24, wherein

A) multiple copies of each such file can exist.

26. The method of claim 24, further comprising the steps of:

A) generating a sequence of graphics rendering events

i) associated with computer generated images and/or overlay graphics at a workstation, and

B) saving the generated sequence.

27. The method of claim 26, wherein

A) the generated sequence of graphics rendering events

i) can be played back to render computer generated images and/or overlay graphics

(a) at a workstation,

(b) together with a workstation user related audio and video

(c) in a determinable temporal relationship

the method comprising the step of:

ii) saving the event sequence

(a) related to the displayed computer generated and/or overlay graphics images and/or the audio and video

(b) such that the event sequence can later be reproduced

at a workstation

in the determined temporal relationship.

28. The method of claim 27, wherein

A) the at least one file includes

i) the saved sequence of graphics rendering events.

29. The method of claim 25, further comprising the step of:

A) decoding signals retrieved from storage.

20. A method of using a networked multimedia system comprising:

A) capturing audio and video at

i) one or more workstations, each including

(a) video and audio reproduction capabilities, and

(b) video and audio capture capabilities;

B) converting the captured audio and video

i) into a form suitable for storage;

C) storing

i) for later retrieval,

ii) the converted audio/video signals

iii) by using

(a) at least one storage CELL having

at least one storage disk,

a storage disk controller

i. associated with each disk, and

at least one converter; and

(b) a storage CELL manager (AVSM); and

D) storing the converted audio/video signals,

i) by selecting any one of the group consisting of disks, disk controllers, converters, storage CELLs and storage CELL managers (AVSM), according to the following approach;

(a) if a first converter

of a first CELL

has reached its capacity or bandwidth limit,

then a second converter, of the first CELL, is selected;

(b) if a first disk controller

of the first CELL

has reached its capacity or bandwidth limit

then a second disk controller of the first CELL is selected;

(c) if a first disk,

associated with a first of the disk controllers

has reached its capacity or bandwidth limit,

then a second disk associated with the first disk controller is selected;

(d) if the first storage CELL

associated with a first storage CELL manager (AVSM)

has reached its capacity or bandwidth limit,

then a second storage CELL associated with the first storage CELL manager (AVSM) is selected; and

(e) if the first storage CELL manager (AVSM) has reached its capacity,

then a second storage server, is selected, and

ii) storing the converted signals using each of the selected ones of the group.

21. A method of using a networked multimedia system comprising the steps of:

A) capturing audio and video at

i) one or more workstations, each including

(a) video and audio reproduction capabilities, and

(b) video and audio capture capabilities;

B) converting captured audio and/or video,

i) into a form suitable for storage;

C) providing a least one storage CELL, each including

i) at least one storage disk

(a) configured to receive and store,

for later retrieval,

the converted audio/video signals,

D) providing at least one storage CELL manager (AVSM); and

E) performing any one of the group of transferring functions consisting of;

i) transferring

(a) the stored converted signals

(b) between any two storage disks of any storage CELL;

ii) transferring

(a) the stored converted signals

(b) between any two storage CELLs,

under control of a single storage CELL manager (AVSM); and

iii) transferring

(a) the stored converted signals

(b) between any two storage CELLs

respectively under control of different storage CELL managers (AVSM).

30. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one signal converter configured to

i) convert audio and/or video information

(a) originating at a workstation,

(b) into a form suitable for storage

C) at least one storage CELL configured to

i) store,

(a) audio and video information

captured at a workstation

(b) as at least one file,

for later retrieval;

D) at least one signal path,

i) interconnecting

(a) the workstations, the converter(s) and the storage CELL(s); and

E) at least one pointer referencing

i) a predetermined file or group of files,

(a) on one or more of the storage CELLs,

ii) wherein the system is configured to transmit,

(a) at least the pointer

(b) to a receiving application program

iii) render the contents

(a) of the file or group of files

(b) at a workstation

(c) by retrieving data

from the file or group of files

31. The networked multimedia system of claim 30, wherein

A) the pointer file

i) has at least one function selected from the group consisting of

(a) identifying a data file,

(b) identifying at least one storage CELL on which the data file is stored,

(c) identifying a media type of the data file,

(d) providing access authorization to the data file,

(e) providing a natural-language title of the data file,

(f) providing a natural-language description of the data file,

(g) identifying a creator of the data file, and

(h) noting a time of creation of the data file.

47. The networked multimedia system of any one of claims 31 to 46 wherein the system is configured to

A) establish real time data sharing sessions

i) between at least two workstations

ii) using a synchronized data sharing process

(a) as a shared conference window.

48. The system of claim 47, wherein

A) the synchronized data sharing process is shareboard, and

B) the use of shareboard renders

i) an application program simultaneously invocable on a plurality of workstations

(a) whereby a workstation user can

access a file,

import data as a window copy from the application program onto the video reproduction capability of a workstation,

scroll through screens of the application program independent of other invocations of the applications on other workstations, and

perform graphics functions utilizing graphics overlay files on the application program

i. independent of other invocations of the application program on other workstations.

49. The system of claim 48, further comprising at least one of the group of:

A) an authoring tool for at least one of

i) drawing objects on the shared conference window, and

ii) typing text on the shared conference window;

B) a grab tool for

i) grabbing another file program, and

ii) drawing at least a portion of the file program

(a) into the shared conference window; and

C) a telepointing tool

i) for independently pointing at objects

(a) on the shared conference window.

50. The networked multimedia system of any one of claims 31 to 46, further comprising:

A) at least one video conference application for

i) establishing a video conference session

(a) among the interconnected workstations, and

B) a video conference recording application for

(a) recording at least a portion

of the established video conference session

on at least one storage CELL.

51. The system of claim 50, wherein

A) the video conference recording application

i) is implemented at the option of at least one user.

52. The system of claim 50, wherein

A) the video conference recording application

i) is automatically implemented as at least one of

(a) an element of a video conference call initiator, and

(b) an element of a video conference call control routine.

53. The system of claim 50, wherein

A) the signal path includes at least

i) a conference bridge

(a) which is invoked at the initiation of the video conference recording application.

54. The system of claim 50, wherein

A) the signal path includes at least one

i) conference bridge

(a) previously invoked at the initiation of the video conference session.

55. The system of claim 50, wherein

A) the signal path includes

i) a conference bridge port

(a) providing a global view of the video conference session participants.

56. The system of claim 50, wherein

A) the signal path includes

i) a normally configured conference bridge port

(a) treated as a standard user port on the conference bridge.

57. The networked multimedia system of any one of claims 31 to 46 further comprising:

A) at least one video mail application configured to do at least one of

i) author a video mail message for transmission to the storage CELL and

ii) read a video mail message previously stored on the storage CELL.

58. The system of claim 57, wherein

A) the video mail application includes

i) a MIME-based video attachment capability

(a) provided to an existing MIME-based mail application.

59. The system of claim 57, wherein

A) the video mail application is configured to

i) share

(a) a video mail message

across a plurality of workstations as

(b) a broadcast video e-mail,

thereby requiring the storage of fewer copies of the message than the number of the plurality of workstations receiving the broadcast video e-mail.

60. The networked multimedia system of any one of claims 31 to 46, wherein the system is configured to

A) to record

B) on a storage CELL, at least one of

i) an audio and video message, and

ii) a multimedia message

C) from an incoming caller.

61. The system of claim 60 further configured to

A) record the video message

i) on one of

(a) the recipient'"'"'s storage CELL, and

(b) the caller'"'"'s storage CELL such that

ii) the message is stored to optimize at least one of

(a) message availability,

(b) system overhead for system network configuration, and

(c) system data storage facilities.

62. The system of claim 60, further configured to

A) respond to a call

i) to a workstation user

ii) by producing a recorded greeting selected from the group consisting of

(a) a stored video greeting,

(b) a text greeting, and

(c) a graphics greeting.

63. The system of claim 62, wherein

A) the recorded greeting

i) is stored on a storage CELL

ii) associated with the workstation user.

64. The system of claim 62, wherein

A) at least one recorded greeting

B) is stored on a plurality of storage CELLs.

32. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one signal converter configured to

i) convert audio and/or video information

(a) originating at a workstation,

(b) into a form suitable for storage

C) at least one storage CELL configured to

i) store,

(a) audio and video information

captured at a workstation

(b) as at least one file,

for later retrieval;

D) at least one signal path,

i) interconnecting the workstations, the converter(s) and the storage CELL(s);

E) a viewer configured to enable a workstation user to do at least one of

i) setting up connections and sessions;

ii) preparing files for viewing, and

iii) providing a viewer interface for the user to access and manipulate files.

33. The system of claim 32, wherein

A) the viewer is

i) invoked as a standard video call, and

ii) is capable of being merged with at least one of

(a) video calls, video conferences, video messages, multimedia calls, multimedia conferences, and multimedia messages.

34. The system of claim 33, wherein

A) the viewer is implemented as a process primitive.

35. The system of claim 33 wherein

A) the viewer includes a plurality of viewers,

the system further comprising

B) a browser for selecting at least one of the plurality of viewers.

36. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one signal converter configured to

i) convert audio and/or video information

(a) originating at a workstation,

(b) into a form suitable for storage

C) at least one storage CELL configured to

i) store,

(a) converted audio and video information

captured at a workstation

(b) as at least one file,

for later retrieval;

D) at least one signal path,

i) interconnecting

(a) the workstations, the converter and the storage CELL(s);

E) at least one stored-video application program

i) on at least one of

(a) the storage CELL

(b) a workstation; and

F) at least one process primitive

i) that can be shared

ii) across at least two of

(a) the stored-video application programs.

37. The networked multimedia system of claim 36, wherein

A) at least one of the stored-video application programs

i) is one from the group consisting of:

(a) videoconference recording;

video mail;

video answering system;

video documents; and

video publishing.

38. The networked multimedia system of claim 37, wherein

A) the process primitive

i) is one from the group consisting of:

(a) viewer;

browser;

MIME attachment;

an annotation authoring tool; and

video editor.

39. The networked multimedia system of claim 38, wherein

A) the system is further configured to

i) share

(a) at least one data primitive

(b) across at least two of

the plurality of stored-video application programs.

40. The networked multimedia system of claim 39, wherein

A) the data primitive

i) is from the group consisting of

(a) a video file, a graphics event file, and a metafile.

41. The networked multimedia system of claim 40, wherein

A) the metafile includes at least one of the group of

i) an audio-video file, audio only file, video only file, a graphics event file, a window event file, an application startup event file, a bit map file, a postscript file, a graphics file, a synchronization file, and a text file.

42. The networked multimedia system of claim 41, wherein

A) the metafile includes

i) an associated pointer file

B) to define an omnifile.

43. The networked multimedia system of claim 39, wherein

A) the system is configured to

i) invoke of at least one of the process primitives and the data primitives as

(a) an attachment to the file,

(b) the attachment being acceptable by third parties according to a standard data interchange protocol.

44. The system of claim 43 wherein

A) the standard data interchange protocol

i) is a MIME protocol, and

B) the attachment

i) is a MIME attachment.

45. The system of claim 44, wherein the MIME attachment

A) invokes one of a viewer and a browser, and

B) initiates signal transfer along the signal path.

46. The system of claim 44, further comprising:

A) a plurality of applications

i) accessible by at least one of the workstations,

ii) the applications being rendered interoperable by at least one of

(a) file sharing by the copying of MIME attachments and

(b) connection sharing.

65. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one signal converter configured to

i) convert audio and/or video information

(a) originating at a workstation,

(b) into a form suitable for storage

C) at least one storage CELL configured to

i) store,

(a) audio and video information

captured at a workstation

(b) as at least one file,

for later retrieval;

D) at least one signal path,

i) interconnecting the workstations, the converter(s) and the storage CELL(s)

E) wherein the system is configured to

i) enable files

(a) stored on a storage CELL

(b) to be included in an on-line electronic document.

66. The system of claim 65 where the system

A) provides access to stored files

i) by utilizing a standardized attachment methodology.

67. The system of claim 66, wherein

A) the standardized attachment methodology is MIME.

68. The system of claim 66, wherein

A) the files are selected from the group consisting of

i) archival video, video overlay graphics, and a compendium of a plurality of video files which further include at least one of additional text or graphics, standalone videos, and MIME attached video files.

69. A networked multimedia system comprising:

A) a plurality of workstations, each including

i) video and audio reproduction capabilities, and

ii) video and audio capture capabilities;

B) at least one signal converter configured to

i) convert audio and/or video information

(a) originating at a workstation,

(b) into a form suitable for storage

C) at least one storage CELL configured to

i) store,

(a) audio and video information

captured at a workstation

(b) as at least one file,

for later retrieval;

D) at least one network providing

i) a signal path,

(a) interconnecting the workstations, the converter(s) and the storage CELL(s)

E) wherein the system defines a proxy server configured to do at least one of

i) reducing the total number of copies of video files by file sharing,

ii) reducing network load by transporting video files over a separate audio-video network,

iii) transcoding among multiple audio-video formats, and

iv) allowing any one of the workstations to serve as a video publisher for the system.

70. The system of claim 69, wherein

A) the workstation serves as a video publisher

i) subject to user permissions.

71. The system of claim 69, wherein the system is configured

A) as an intranet video storage server for at least one of

i) reducing the total number of copies of video files in the enterprise by file sharing,

ii) reducing enterprise network load by transporting video files over a separate audio-video network,

iii) transcoding among multiple audio-video formats, and

iv) allowing any audio-video workstation in the enterprise to serve as a video publisher for the enterprise.

72. The networked multimedia system of claim 69, wherein

A) the system is configured as a proxy server

i) enabling the implementation of at least one animated annotation on the internet.

73. The system of claim 72, wherein

A) at least one of the animated annotations

i) is synchronized with respect to one or more of

(a) an accompanying video file,

(b) an accompanying audio file.

74. The system of claim 71, further configured to

A) enable the authoring of an animated annotation utilizing a window-sharing tool.

75. The system of claim 74, wherein

A) the window-sharing tool is selected from the group consisting of

i) a shareboard application and an interactive meeting enabling computer application.