System and associated methodology for multi-layered site video conferencing

• US 8,345,082 B2
• Filed: 10/08/2009
• Issued: 01/01/2013
• Est. Priority Date: 10/08/2008
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Tysso, Jorunn
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US12/576,018
• Publication Number:
  US 20100171807A1
• Time in Patent Office: 1,181 Days
• Field of Search: 348 1401- 1416
• US Class Current: 348/14.08
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A video conferencing system with more than three endpoints and a multipoint control unit displays a merged image of the video conference participants at an endpoint monitor. The participant images are separated from their corresponding background and merged to make all participants appear the same size on the monitor. The system also identifies an active endpoint based on the level of sound received from the endpoints and highlights a corresponding participant as a speaker.

Claims

1. A method to cause a video conferencing system to display a plurality of participants, comprising:

receiving, at a multipoint control unit, video data including sound and image data from a plurality of endpoints connected to the multipoint control unit, the video data corresponding to participants located at each of the plurality of endpoints, at least one of the endpoints capturing video data corresponding to more than one participant;

extracting, at the multipoint control unit, participant image data from the sound and image data included in the video data;

processing, at the multipoint control unit, the extracted participant image data to have a same appearance;

combining, at the multipoint control unit, the processed participant image data to generate a merged image data of participants from other endpoints, the merged image data excluding surrounding background image data corresponding to each of the participants, images of the participants overlapping in the merged image data;

transmitting the merged image data from the multipoint control unit to a respective endpoint; and

displaying, at a monitor of the respective endpoint, the merged image data received from the multipoint control unit, the monitor displaying participants from other endpoints as being located in a same room.

2. The method according to claim 1, further comprising:

identifying, at the multipoint control unit, an endpoint with a highest sound level as an active endpoint.

3. The method according to claim 1, wherein the extracted image data corresponds to a body image of the participants without corresponding background imagery.

4. The method according to claim 1, wherein the participant image data is extracted from the video data using a synthetic blue or green screen and an algorithm to analyze background imagery and image depth information.

5. The method according to claim 1, wherein the participants are displayed aligned horizontally on the monitor, the horizontal alignment and overlapping dynamically varying according to changes in a number of participants in the video conference.

6. The method according to claim 1, wherein the processing of the extracted participant image data includes determining mean values for image size, color balance, contrast and intensity based on the extracted participant image data, and based on the mean values determined scaling each participant image to a same size, adjusting each participant image to a same color balance and adjusting each participant image to a same contrast and intensity.

7. The method according to claim 2, wherein a participant image from the active endpoint is highlighted on the monitor.

8. The method according to claim 7, wherein the participant image from the active endpoint is highlighted with borders.

9. The method according to claim 2, wherein participant images from endpoints other than the active endpoint are softened.

10. The method according to claim 2, wherein a participant image corresponding to the active endpoint is displayed in front of other participant images on the monitor.

11. The method according to claim 2, wherein a participant image corresponding to the active endpoint is faded out according to a predetermined delay, and a participant image corresponding to a next active endpoint is faded in according to the predetermined delay.

12. A video conferencing system for displaying a plurality of participants as being located in a single room, comprising:

a multipoint control unit configured to receive video data including sound and image data from a plurality of endpoints connected to the multipoint control unit, the video data corresponding to a plurality of participants respectively located at the plurality of endpoints, at least one of the endpoints capturing video data corresponding to more than one participant, the multipoint control unit extracting participant image data from the sound and image data and processing the extracted participant image data to have a same appearance; and

at least one endpoint connected to the multipoint control unit and configured to display merged image data of the participants received from the multipoint control unit, the merged image data of the participants excluding background image data, the at least one endpoint providing the merged image data to a monitor to display the participants corresponding to the plurality of endpoints as being located in a same room,

wherein images of the participants are overlapped in the merged image data.

13. The system according to claim 12, wherein the plurality of endpoints are configured to extract the participant image data before transmitting the video data to the multipoint control unit.

14. The system according to claim 12, wherein the MCU separates the extracted participant image data from background image data.

15. The system according to claim 14, wherein the image data is extracted by applying a blue or green screen behind each participant and analyzing corresponding depth information.

16. A multipoint control unit device for a video conferencing system comprising:

a receiving unit configured to receive video data including sound and image data from a plurality of endpoints, the sound and image data corresponding to participants respectively located at each of the plurality of endpoints, at least one of the endpoints capturing video data corresponding to more than one participant;

an extracting unit configured to extract participant image data from the sound and image data included in the video data;

an image processing unit configured to process the extracted participant image data to have a same appearance;

a combining unit configured to combine the processed participant image data into a merged participant image data without corresponding background image data, the merged participant image data causing the participants to appear located in a same room, images of the participants overlapping in the merged image data; and

a transmitter configured to transmit the merged participant image data to at least one of the plurality of endpoints.

17. The multipoint control unit device according to claim 16, wherein the extracting unit extracts participant body image data from surrounding background image data, the extracted participant body image data being the extracted participant image data.

18. An endpoint device in a video conferencing system, comprising:

an imaging device configured to capture video data of a participant and a surrounding background, the video data including sound and image data;

an extraction unit configured to extract participant image information from the video data;

a transmitter configured to transmit the extracted participant image information to a multipoint control unit;

a receiver configured to receive, from the multipoint control unit, a merged participant image data formed by merging image data from a plurality of endpoints, at least one of the endpoints capturing video data corresponding to more than one participant, the merged participant image data excluding surrounding background image data, images of the participants overlapping in the merged image data; and

a monitor configured to display the merged participant image data, participants corresponding to the merged participant image data being displayed as located in a same room.

19. The endpoint device according to claim 18, wherein the extracting unit extracts participant body image data from surrounding background image data, the extracted participant body image data being the extracted participant image data.

20. The endpoint device according to claim 18, wherein a blue or green screen is placed behind the participant, and the extracting unit extracts the participant image data using a predetermined algorithm to analyze image depth information.

21. A non-transitory computer-readable medium storing computer-readable instruction thereon, the computer-readable instructions when executed by a computer cause the computer to perform a method comprising:

receiving, at a multipoint control unit, video data including sound and image data from a plurality of endpoints connected to the multipoint control unit, the video data corresponding to participants located at each of the plurality of endpoints, at least one of the endpoints capturing video data corresponding to more than one participant;

extracting participant image data from the sound and image data included in the video data;

processing the extracted participant image data to have a same appearance;

combining the processed participant image data to generate a merged image data of participants from other endpoints, the merged image data excluding surrounding background image data corresponding to each of the participants, images of the participants overlapping in the merged image data;

transmitting the merged image data from the multipoint control unit to a respective endpoint; and

displaying the merged image data received from the multipoint control unit on a monitor, the monitor displaying participants from other endpoints as being located in a same room.