Digital processing apparatus and method to support video conferencing in variable contexts

• US 6,330,022 B1
• Filed: 11/05/1998
• Issued: 12/11/2001
• Est. Priority Date: 11/05/1998
• Status: Expired due to Term

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Seligmann, Doree
• Primary Examiner(s): Kuntz, Curtis
• Assistant Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US09/186,181
• Time in Patent Office: 1,132 Days
• Field of Search: H04/N.714, H04/N.715, 348/14, 348/15, 345/330, 345/331, 345/327, 345/302, 709/204, 370/260, 370/261
• US Class Current: 348/14.08
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/155 involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Video processing apparatus for supporting a video conference includes a processor/server, a conference bridge and a network of terminals, each terminal equipped with at least a video camera, display and a user input device such as a mouse. A user may select a conference context including a conference location, type (setting) and purpose that may be different from the selection of another conferee. The user may select a conference location and type, view a conference according to a default selection or design their own location and type in advance of the conference. The user also may select personal requirements and signal the need for various resources prior to and in real time during a conference. At the conference bridge, each conferee that joins a video conference is assigned a position at a conference setting of their selected context, for example, a round table or U shaped table generated as a video graphic. At each terminal, a signal is either received or generated for display such that each conference participant is able to view all other participant conferees according to the assigned position arrangement within the context selected by the conferee. An algorithm is described for generating such a signal for display. An audio signal can also be generated according to the assigned positions of the conferees that can be heard on 3D or stereo speakers in such a way as to relate to the assigned positions around the virtual conference setting. Moreover, as conferees join or leave the conference, the virtual setting may increase in size to accommodate the new conferees.

Claims

1. Intelligent terminal apparatus for providing real-time live image video conference services comprising:

a display arrangement;

an audio speaker arrangement;

a data input device;

a controller; and

a network interface;

the controller of said intelligent terminal apparatus, responsive to actuation of said data input device for receiving input selection data of a virtual conference context and outputting to said display arrangement a video conference image according to said selected virtual conference context, said virtual conference context capable of being a different context than that input at another intelligent terminal apparatus in a video conference with said intelligent terminal apparatus.

2. The intelligent terminal apparatus as recited in claim 1 wherein said display arrangement includes a display having three screen portions, a first screen portion for displaying a video conference according to said selected virtual conference context, a second screen portion for displaying a video presentation of a video conferee and a third screen portion for displaying a plurality of icons, one of said icons for actuating the display of another screen.

3. The Intelligent terminal apparatus as recited in claim 2, wherein said display arrangement further includes a camera equipped with a fish-eye lens and said data input device receives input selections of pan, tilt, and zoom for manipulating a conference view of said selected virtual conference context displayed in said first screen portion according to said input selections of pan, tilt and zoom.

4. The intelligent terminal apparatus as recited in claim 2 wherein said display arrangement further comprises a microphone for obtaining conferee audio input for mixing with other conferee audio input in stereo according to conferee position within said selected virtual conference context.

5. Apparatus for supporting a video conference comprising:

a conference controller for assigning a position to a video conferee in a virtual conference context, said virtual conference context comprising a conference location and conference type;

a signal developer for developing a real-time video signal portion for each conferee depending on the assigned position of the conferee; and

a signal transmitter, responsive to inputs of a first conferee, for transmitting a first video signal representing a first virtual conference context to a first conferee and, responsive to inputs of a second conferee, for transmitting a second video signal representing a second virtual conference context to the second conferee, said second virtual conference context being different from said first context and including an image of said first conferee.

6. Apparatus as recited in claim 5 wherein said video said video signal developer comprises a video signal processor and memory for storing virtual conference context data, said video signal developer developing a video signal comprising a conferee portion and a context portion, said portions being relative to a number of conferees input by said first conferee.

7. Apparatus as recited in claim 5 further comprising an audio signal processor for providing a sound signal comprising left and right signal portions corresponding to said developed video signal.

8. Apparatus as recited in claim 6 wherein said video signal processor outputs a signal including a conferee portion representing images of all conferees where the number of all conferees is N and N is greater than 2.

9. Apparatus as recited in claim 5 wherein said apparatus comprises a conference bridge and terminal apparatus and said terminal apparatus comprises a signal generator for generating a video signal comprising image signal portions representing each of N-1 conferees for display.

10. Apparatus as recited in claim 6 wherein said video signal processor outputs a conferee signal portion comprising images of all conferees but one, said one image representing an image of the first conferee to which said video signal is transmitted.

11. Apparatus as recited in claim 5 wherein said apparatus comprises a conference server, a conference bridge and a plurality of intelligent terminal devices.

12. Apparatus as recited in claim 11 wherein each said intelligent terminal device comprises:

a camera, microphone and display arrangement;

an audio speaker arrangement;

a data input device; and

a network interface;

the intelligent terminal apparatus, responsive to actuation of said data input device for receiving input selection data of virtual conference context, said context capable of being a different virtual conference context than that input at another intelligent terminal apparatus.

13. Apparatus for supporting a real-time video conference comprising:

conference controller means for assigning a position to a video conferee in a virtual conference context, said virtual conference context comprising a conference location and conference type;

signal development means for developing a video signal portion for each conferee depending on the assigned position of the conferee; and

signal transmitting means, responsive to inputs of a first conferee, for transmitting a first video signal for display representing a first virtual conference context to a first conferee, such that a second video signal is transmitted by another signal transmitting means representing a second virtual conference context to a second conferee, said second virtual conference context being different from said first virtual conference context and including an image of said first conferee.

14. Apparatus as recited in claim 13 further comprising a memory means for storing virtual conference context data, said video signal development means developing video signal comprising a conferee portion and a virtual conference context portion, said portions being relative to a number of conferees input by said first conferee.

15. Video conferencing apparatus for providing a real-time live image of conferees of a video conference which comprises a video processor for (i) receiving a plurality of video signals of each conferee, (ii) receiving input signals from each of two conferees representing a different selected video conference context, (iii) assigning a position to a conferee within a selected virtual conference context and (iv) further outputting a video signal representing all conferees according to the assigned positions within the selected virtual conference context.

16. Video conferencing apparatus for providing a real-time live image of conferees of a video conference as recited in claim 15 further comprising an audio processor for receiving a plurality of audio signals of each conferee and generating a stereo audio signal for a conferee according to the assigned position of the conferee that is different from the stereo audio signal generated for another conferee.

17. Video conferencing apparatus as recited in claim 15 wherein said video processing is distributed between a central conference bridge and a plurality of conferee terminals.

18. Video conferencing apparatus as recited in claim 15 wherein said video processing is provided at a central conference bridge.

19. A method of conferencing a plurality of video terminals comprising the steps of

receiving a real-time live video signal from each of said plurality of video terminals;

receiving virtual conference context data from each of said plurality of video terminals, said virtual conference context data representing a selected location for a conference and type of conference;

assigning a position in a video conference to each video terminal;

developing a video signal for a video terminal depending on the assigned position; and

transmitting a first video signal for display at a first video terminal, said video signal representing a different virtual conference context than a virtual conference context represented by a video signal transmitted for display at a second video terminal.

20. A method of conferencing a plurality of video terminals as recited in claim 19 wherein the output of a concatenation step comprises a video signal comprising all the terminals of the plurality of terminals.

21. A method of conferencing a plurality of video terminals as recited in claim 19 further comprising the step of transmitting a unique video signal to each terminal for display wherein the output of a concatenation step comprises a video signal comprising all the terminals except the terminal to which said unique signal is transmitted.

22. A method of conferencing a plurality of video terminals as recited in claim 19 further comprising the step of receiving conferee inputs for at least one of the set consisting of pan, tilt and zoom within a selected context.

23. A method as recited in claim 19 further comprising the steps of receiving an audio signal from each terminal and generating a combined audio signal for each conferee, the combined audio signal for a conferee being attenuated according to an assigned position of a conferee within a received selected context.

24. A method as recited in claim 19 further comprising the step of generating a graphic image of a location and a graphic image of a conference type and chromakeying a concatenated video signal to superimpose images for terminal conferees at said conference location and type.

25. A method as recited in claim 24 further comprising the step of displaying a chromakeyed image at said terminal such that each conferee views a displayed image comprising all other conferees except themselves.

26. A method as recited in claim 19 further comprising the step of resizing said conference location and type at a conferee terminal responsive to inputs of a conferee after said conferee joins a conference in progress.

27. A method as recited in claim 19 further comprising the steps of generating a conference location view from 180 degree location image data, overlaying a conference type selection selected from one of round table, U-shape table arrangement, square table and rectangular table and overlaying a conferee image at said selected conference type within said selected location view.

28. A method as recited in claim 19 further comprising the steps, upon a conferee departing from a conference, of selectively, responsive to conferee input, reserving said assigned position at said conference type residing said conference type or repositioning remaining conferees at said conference type.

29. A computer readable media for use with a processor of an intelligent video conference controller, said controller for receiving image signals for each conferee of a video conference and for receiving input data of a first conferee representing the first conferee'"'"'s selection of a virtual conference context including a conference location and a conference type and input data of a second conferee representing the second conferee'"'"'s selection of a different virtual conference context, the media for storing instructions for controlling said processor to perform the steps of:

outputing a video signal for display at a first conferee terminal representing a video conference including the conferee real-time images within the selected virtual conference context; and

outputing for display a different video signal representing the video conference including the real-time conferee images within the different selected virtual conference context to said second conferee.

30. A media as recited in claim 29 for further storing instructions for controlling said processor according to the steps of generating a conference location view from 180 degree location image data, overlaying a conference type selection selected from one of round table, U-shape table arrangement, square table and rectangular table and overlaying a conferee image at said selected conference type within said selected location view.

31. A computer readable media for use with a processor of an intelligent video terminal, said controller for receiving an image signal for a conferee of a video conference and for receiving input data of said conferee representing the conferee'"'"'s selection of a virtual conference context including a conference location and a conference type, the media for storing instructions for controlling said processor to perform the step of:

outputting a video signal for display at said video terminal representing a video conference including real-time conferee images within the selected virtual conference context;

said output video signal being different video signal output for display by another intelligent video terminal coupled in video conference arrangement, said different video signal representing the video conference including the conferee images within a different selected virtual conference context by another conferee.

32. A media as recited in claim 31 for further storing instructions for controlling said processor according to the steps of generating a conference location view from 180 degree location image data, overlaying a conference type selection selected from one of round table, U-shape table arrangement, square table and rectangular table and overlaying a conferee image at said selected conference type within said selected location view.