Multi-site television conference system and central control apparatus and conference terminal for use with the system

• US 6,285,392 B1
• Filed: 11/23/1999
• Issued: 09/04/2001
• Est. Priority Date: 11/30/1998
• Status: Expired due to Term

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Satoda, Kozo, Hiraike, Ryuichi
• Primary Examiner(s): Kuntz, Curtis
• Assistant Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US09/447,268
• Time in Patent Office: 651 Days
• Field of Search: H04/N.714, H04/N.715, 348/14.01, 348/14.02, 348/14.03, 348/14.04, 348/14.05, 348/14.07, 348/14.08, 348/14.09, 345/330, 345/331, 709/204, 709/205, 379/202, 370/260, 370/261
• US Class Current: 348/14.09
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

The invention provides a multi-site television conference system which provide natural video images and acoustic images which provide a good feeling of presence to participants who participate in a conference through respective conference terminals. A plurality of cameras are connected to each of conference terminals located at different sites remote from each other and pick up video images of a participant who uses the conference terminal. A central control unit receives, at a multi-site image inputting section thereof, from the conference terminals, video images of participants of the respective conference terminals picked up in multi-directions. An image selection section of the central control unit selects video images of the participants of the conference terminals taking it into consideration how the other conference participants looks to each participant in a virtual conference space and synthesizes the video images into the virtual conference space, and then transmits resulting video images to the conference terminals so that the conference terminals may present computer graphics images to the participants.

Claims

1. A multi-site television conference system, comprising:

a central control unit for controlling a conference; and

a plurality of conference terminals connected to said central control unit;

each of said conference terminals including a plurality of image pickup apparatus for picking up video images of a participant, who participates in the conference through the conference terminal, from a plurality of directions, a multi-direction image inputting section for receiving the video images from said plurality of image pickup apparatus and transmitting the video images to said central control unit, an image outputting section for receiving a video image of a virtual conference room transmitted thereto from said central control unit and presenting the received video image to the participant, and a control information inputting section for inputting information for controlling the conference therethrough and transmitting the inputted information to said central control unit;

said central control unit including a multi-site image inputting section for receiving video images transmitted thereto from said conference terminals of individual participants to the conference, an image production timing generation section for designating one of the participants who should see a video image to be synthesized by and transmitted from said central processing unit in order that video images may be produced and transmitted to said conference terminals of the individual participants, a virtual space information storage section for storing information to be used for production of a video image of the virtual conference space, updating the information of the virtual conference space with control information from said conference terminals and transmitting the information to be used for production of a video image in response to an instruction from said image production timing generation section, an image selection section for selecting appropriate video images from among the multi-direction video images received by said multi-site image inputting section by distinguishing how, when the participant for which a video image is being produced at present views the participants of the other conference terminals in the virtual conference space based on the information from said virtual space information storage section, the participants look to the participant, and an image production section for producing a video image in which the video images selected by said image selection section are arranged in the virtual conference space based on the information from said virtual space information storage section and transmitting the produced video image to the conference terminal of the participant.

2. A multi-site television conference system as claimed in claim 1, wherein said image selection section of said central control unit has a function of producing, when the multi-direction video images received by said multi-site image inputting section do not include an appropriate video image, an appropriate video image using a plurality of ones of the multi-direction video images.

3. A multi-site television conference system as claimed in claim 1, wherein each of said conference terminals further includes a speech inputting section for inputting speech of the conference participant therethrough and transmitting the inputted speech to said central control unit, and a speech outputting section for receiving speech synthesized by said a central control unit and presenting the received speech to the participant, said virtual space information storage section of said central control unit further being capable of storing information to be used for production of an acoustic image of the virtual conference space, updating the information of the virtual conference space with control information from said conference terminals and transmitting the information to be used for production of an acoustic image, and said central control unit further includes a multi-site speech inputting section for receiving speech transmitted thereto from said conference terminals of the individual participants, and an acoustic image synthesis section for producing an appropriate acoustic image by distinguishing how, when the participant for which speech is being produced hears speech of the participants of the other conference terminals in the virtual conference space based on the information from said virtual space information storage section, the speech sounds to the participant from among the speech from the multi-sites and transmitting the produced acoustic image to the conference terminal of the participant.

4. A multi-site television conference system, comprising:

a plurality of conference terminals located at different sites from each other and connected to each other;

each of said conference terminals including a plurality of image pickup apparatus for picking up video images of a participant, who participates in a conference through the conference terminal, from a plurality of directions, a multi-direction image inputting section for receiving the video images from said plurality of image pickup apparatus and transmitting the video images to the other ones of said plurality of conference terminals connected thereto, a control information inputting section for receiving information for controlling the conference from the participant and transmitting the received information to the other conference terminals, a multi-site image inputting section for receiving video images transmitted thereto from the other conference terminals, a virtual space information storage section for storing information to be used for production of a video image of a virtual conference space, updating the information of the virtual conference space with the control information received from the other conference terminals and outputting the information to be used for production of a video image, an image selection section for selecting appropriate video images from among the multi-direction video images received from the other conference terminals by distinguishing how, when the participant of the conference terminal views the participants of the other conference terminals in the virtual conference space based on the information from said virtual space information storage section, the participants look to the participant of the conference terminal, and an image production section for producing a video image in which the video images selected by said image selection section are arranged in the virtual conference space based on the information from said virtual space information storage section such that the participants of said plurality of conference terminals share the virtual conference space.

5. A central control unit for a multi-site television conference system, comprising:

a multi-site image inputting section for receiving video images transmitted thereto from a plurality of conference terminals;

an image production timing generation section for designating one of participants of said conference terminals who should see a video image to be synthesized by and transmitted from said central processing unit in order that video images may be synthesized and transmitted to said conference terminals;

a virtual space information storage section for storing information to be used for production of a video image of a virtual conference space, updating the information of the virtual conference space with control information received from said conference terminals and transmitting the information to be used for production of a video image in response to an instruction from said image production timing generation section;

an image selection section for selecting appropriate video images from among the multi-direction video images received by said multi-site image inputting section by distinguishing how, when the participant for which a video image is being produced at present views the participants of the other conference terminals in the virtual conference space based on the information from said virtual space information storage section, the participants look to the participant; and

an image production section for producing a video image in which the video images selected by said image selection section are arranged in the virtual conference space based on the information from said virtual space information storage section and transmitting the produced video image to the conference terminal of the participant.

6. A central control unit for a multi-site television conference system as claimed in claim 5, wherein said image selection section has a function of producing, when the multi-direction video images received by said multi-site image inputting section do not include an appropriate video image, an appropriate video image using a plurality of ones of the multi-direction video images.

7. A conference terminal for a multi-site television conference system, comprising:

a plurality of image pickup apparatus for picking up video images of a participant, who participates in a conference through said conference terminal, from a plurality of directions;

a multi-direction image inputting section for receiving the video images from said plurality of image pickup apparatus and transmitting the video images to the other conference terminals;

a control information inputting section for receiving information for controlling the conference from the participant and transmitting the received information to the other conference terminals;

a multi-site image inputting section for receiving video images transmitted thereto from the other conference terminals;

a virtual space information storage section for storing information to be used for production of a video image of a virtual conference space, updating the information of the virtual conference space with the control information received from the other conference terminals and outputting the information to be used for production of a video image;

an image selection section for selecting appropriate video images from among the multi-direction video images received from the other conference terminals by distinguishing how, when the participant of the conference terminal views the participants of the other conference terminals in the virtual conference space based on the information from said virtual space information storage section, the participants look to the participant of the conference terminal; and

an image production section for producing a video image in which the video images selected by said image selection section are arranged in the virtual conference space based on the information from said virtual space information storage section such that the participants of the other conference terminals share the virtual conference space.