Video conferencing system, conference terminal and image server

• US 7,835,764 B2
• Filed: 02/17/2006
• Issued: 11/16/2010
• Est. Priority Date: 03/11/2005
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Kanada, Yasusi
• Primary Examiner(s): Appiah; Charles N
• Assistant Examiner(s): DOAN, KIET M
• Application Number:
  US11/356,172
• Publication Number:
  US 20060206560A1
• Time in Patent Office: 1,733 Days
• Field of Search: 348/15, 348/14.8, 348/14.09, 348/565, 709/204, 455/550.1, 370/260, 370/261, 715/738, 715/757, 715/740, 345/1.1
• US Class Current: 455/550.1
• CPC Class Codes: H04L 61/4535 using an address exchange platform which sets up a session between two nodes, e.g. rendezvous servers, session initiation protocols [SIP] registrars or H.323 gatekeepers
• CPC Class Codes: H04L 61/4557 Directories for hybrid networks, e.g. including telephone numbers
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/1104 Session initiation protocol [SIP]
• CPC Class Codes: H04L 65/4038 with floor control

Abstract

In a video conferencing system, required bandwidth is suppressed to a lower level and a key person or the like of a conference can be confirmed easily. A presence server 1 manages location information in a virtual space for a user of each conference terminal 2. Each conference terminal 2 sends image data of its own user to another conference terminal 2 in response to a request of that conference terminal 2. Further, each conference terminal 2 receives location information of its own user in the virtual space, sends the received location information to the presence server 1, and receives location information of the other users from the presence server 1. Then, based on the location information of its own user and the location information of the other users, the conference terminal 2 detects forward users located forward from its own user, and receives image data from the conference terminal 2 of each of the forward users.

Claims

1. A video conferencing system comprising:

a plurality of conference terminals and a presence server for managing a location of a user of each of said plurality of conference terminals in a virtual space, wherein;

each of said plurality of conference terminals comprises;

an image data sending means, which sends image data of said each conference terminal'"'"'s own user to another conference terminal in response to an image sending request from said another conference terminal;

a location determining means, which determines a location and a direction of said each conference terminal'"'"'s own user in said virtual space;

a terminal side location information sending means, which sends virtual location information of said each conference terminal'"'"'s own user, the virtual location information including the location and the direction determined by said location determining means to said presence server;

a terminal side location information receiving means, which receives virtual location information of each of other users who are users of conference terminals other than said conference terminal from said presence server;

a forward user detection means, which detects, as a at least one forward user, among said other users at least one user who is located in front of said each conference terminal'"'"'s own user in the virtual space, based on said determined location and direction of said each conference terminal'"'"'s own user and said virtual location information of each of said other users;

a communication control means which sends an image sending request to a conference terminal of each of said at least one forward user detected by said forward user detection means;

an image data receiving means, which receives image data of each of said at least one forward user from the conference terminal of each of the at least one forward user;

a display control means, which displays, on a display, a conference screen that includes image of each of said at least one forward user, the image of each of said at least one forward user being formed by the image data received by the image data receiving means;

a voice data sending means, which sends voice data of said each conference terminal'"'"'s own user to said conference terminals of said other users;

a voice data receiving means, which receives voice data of said other users from the conference terminals of said other users;

a voice synthesizing means, which applies stereophonic processing on each of said other users'"'"' voice data received by said voice data receiving means, depending on a respective relative position of said at least one forward user in relation to said user in the virtual space, with said relative position being specified by the virtual location information of each of said at least one forward user and the virtual location information of said each conference terminal'"'"'s own user, and synthesizes said other users'"'"' voice data submitted to the stereophonic processing, to generate stereophonic synthesized voice data;

a voice control means, which outputs the stereophonic synthesized voice data generated by said voice synthesizing means through speakers; and

a speaking detection means, which detects speaking of said each conference terminal'"'"'s own user, wherein

said image data sending means of each of said plurality of conference terminals sends, as image data of a speaking user, the image data of said each conference terminal'"'"'s own user to the conference terminals of said other users together with an indication of a speaking user when said speaking detection means detects speaking of said each conference terminal'"'"'s own user;

said image data receiving means of each of said plurality of conference terminals receives the image data indicated as image data of a speaking user; and

said display control means of each of said plurality of conference terminals displays the conference screen that further includes an image of said speaking user on the display, even when said speaking user is not said at least one forward user, and

said presence server comprises;

a management means, which manages users'"'"' virtual location information sent from said plurality of conference terminals; and

a server side location information sending means, which sends to each of said plurality of conference terminals the location information of each of said other users.

2. A video conferencing system according to claim 1, wherein:

the display control means of each of said plurality of conference terminals allocates, in order, the image data of said at least one forward user respectively to a plurality of display frames provided in said conference screen, according to a relative position of each of said at least one forward user in relation to said each conference terminal'"'"'s own user in the virtual space, said relative position being specified by the virtual location information of each of said at least one forward user and the virtual location information of said each conference terminal'"'"'s own user.

3. A video conferencing system according to claim 1, wherein:

said display control means of each of said plurality of conference terminals allocates the received image data of said speaking user to a display frame positioned on a right side or a left side of display frames to which image data of said at least one forward user is respectively allocated in said conference screen, according to a relative position of said speaking user and said each conference terminal'"'"'s own user in the virtual space, said relative position being specified by the location information of said each conference terminal'"'"'s own user and the location information of said speaking user, when the image data of the speaking user is image data of a user who is not said at least one forward user.

4. A video conferencing system comprising:

a plurality of conference terminals, a voice server for sending voice data to each of said plurality of conference terminals, a presence server for managing a location of a user of each of said plurality of conference terminals in a virtual space, and an image server for sending image data to each of said plurality of conference terminals, wherein;

each of said plurality of conference terminals comprises;

a terminal side image data sending means, which sends image data of said each conference terminal'"'"'s own user to said image server;

a location determining means, which determines a location and a direction of said each conference terminal'"'"'s own user in said virtual space;

a terminal side location information sending means, which sends virtual location information of said each conference terminal'"'"'s own user, the virtual location information including the location and the direction determined by said location determining means to said presence server;

a terminal side image data receiving means, which receives conference screen data from said image server;

a display control means, which displays, on a display, the conference screen data received by said terminal side image data receiving means;

a voice data sending means, which sends voice data of said each conference terminal'"'"'s own user to said voice server;

a stereophonic synthesized voice data receiving means, which receives stereophonic synthesized voice data from said voice server;

a voice control means, which outputs the stereophonic synthesized voice data received by said stereophonic synthesized voice data receiving means through speakers; and

a speaking detection means, which detects speaking of said each conference terminal'"'"'s own user and sends a speaking detection signal to said image server,

said image server comprises;

an image server side image data receiving means, which receives image data of said user of each of said plurality of conference terminals;

an image server side location information receiving means, which receives virtual location information of each of said plurality of conference terminals from said presence server;

a forward user detection means, which detects, as a at least one forward user, at least one other user who is located in front of said each conference terminal'"'"'s own user in the virtual space, based on the determined location and direction of said each conference terminal'"'"'s own user and the virtual location information of each of said at least one other user, said at least one other user being a user of a conference terminal other than said each conference terminal; and

an image server side image data sending means, which sends to each of said plurality of conference terminals conference screen data including the image data of each of the at least one forward user detected in relation to said each conference terminal by said forward user detection means, out of image data received by said image server side image data receiving means of respective users of said plurality of conference terminals;

said presence server comprises;

a presence server side location information receiving means, which receives said each conference terminal'"'"'s own user'"'"'s virtual location information sent from said each of said plurality of conference terminals;

a management means, which manages users'"'"' virtual location information sent from said plurality of conference terminals; and

a presence server side location information sending means, which sends the virtual location information of each of said plurality of conference terminals to said image server, said virtual location information being managed by said management means, and

said voice server comprises;

a voice data receiving means, which receives voice data of said each conference terminal'"'"'s own user;

a voice server side location information receiving means, which receives virtual location information of said each conference terminal'"'"'s own user from said presence server;

a voice synthesizing means, which applies, for each of said plurality of conference terminals, stereophonic processing on each of voice data of said other users among voice data received by said voice data receiving means, depending on respective relative positions of said at least one forward user in relation to said each conference terminal'"'"'s own user in the virtual space, said relative positions being specified by the virtual location information of each of said at least one forward users user and the virtual location information of said each conference terminal'"'"'s own user, and synthesizes said other users'"'"' voice data submitted to the stereophonic processing, to generate stereophonic synthesized voice data; and

a stereophonic synthesized voice data sending means, which sends the stereophonic synthesized voice data generated by said voice synthesizing means with respect to each of said plurality of conference terminals to said each conference terminal, and

wherein said image server further comprises a speaking detection signal receiving means, which receives the speaking detection signal from each of said plurality of conference terminals, and

wherein when said speaking detection signal receiving means receives a speaking detection signal, said image server side image data sending means of said image server sends image data of a user as a source of said speaking signal to each of said plurality of conference terminals if said user as a source of said speaking signal is other than the at least one forward user detected by said forward user detection means, said image data being included as image data of a speaking user in the conference screen data.

5. A video conferencing system according to claim 4, wherein:

said image server side image data sending means of said image server determines, for each of said plurality of conference terminals, a layout position of image data of each of said at least one forward user in said conference screen, according to a relative position of said at least one forward user detected by said forward user detection means in relation to said each conference terminal'"'"'s user in the virtual space, and sends, to said each conference terminal, the conference screen data in which the image data of the at least one forward user are respectively allocated, in order, to a plurality of display frames provided in said conference screen.

6. A video conferencing system according to claim 4, wherein:

for each of said plurality of conference terminals, said image server side image data sending means of said image server determines a layout position of the image data sent to the conference terminal of the speaking user in the conference screen, depending on a relative position of said speaking user in relation to said each conference terminal'"'"'s user in the virtual space, and sends the conference screen data to said each conference terminal, the image data of said speaking user being allocated based on said determined layout position to a display frame positioned in said conference screen on a right side or left side of display frames to which image data of said at least one forward user are respectively allocated in said conference screen data.

7. A conference terminal, comprising:

an image data sending means, which sends image data of said conference terminal'"'"'s own user to another conference terminal in response to an image sending request from said another conference terminal;

a location determining means, which determines a location and a direction of said conference terminal'"'"'s own user in said virtual space;

a terminal side location information sending means, which sends virtual location information of said conference terminal'"'"'s own user, the virtual information including the location and the direction determined by said location determining means to a presence server that manages a location, in said virtual space, of a user of each of a plurality of conference terminals participating in a video conference;

a terminal side location information receiving means, which receives virtual location information of each of other users of conference terminals other than said conference terminal from said presence server;

a forward user detection means, which detects, as at least one forward user, at least one of said other users located in front of said conference terminal'"'"'s own user in the virtual space, based on said determined location and direction of said conference terminal'"'"'s own user and said virtual location information of each of said other users;

a communication control means which sends an image sending request to a conference terminal of each of said at least one forward user detected by said forward user detection means;

an image data receiving means, which receives image data of each of said at least one forward user from the conference terminal of each of the at least one forward user;

a display control means, which displays, on a display, a conference screen that includes image of each of said at least one forward user, the image of each of said at least one forward user being formed by the image data received by the image data receiving means;

a voice data sending means, which sends voice data of said each conference terminal'"'"'s own user to said conference terminals of said other users;

a voice data receiving means, which receives voice data of said other users from the conference terminals of said other users;

a voice synthesizing means, which applies stereophonic processing on each of said other users'"'"' voice data received by said voice data receiving means, depending on a respective relative position of said at least one forward user in relation to said user in the virtual space, with said relative position being specified by the virtual location information of each of said at least one forward user and the virtual location information of said each conference terminal'"'"'s own user, and synthesizes said other users'"'"' voice data submitted to the stereophonic processing, to generate stereophonic synthesized voice data;

a voice control means, which outputs the stereophonic synthesized voice data generated by said voice synthesizing means through speakers; and

a speaking detection means, which detects speaking of said each conference terminal'"'"'s own user, wherein

said image data sending means of each of said plurality of conference terminals sends, as image data of a speaking user, the image data of said each conference terminal'"'"'s own user to the conference terminals of said other users together with an indication of a speaking user when said speaking detection means detects speaking of said each conference terminal'"'"'s own user;

said image data receiving means of each of said plurality of conference terminals receives the image data indicated as image data of a speaking user; and

said display control means of each of said plurality of conference terminals displays the conference screen that further includes an image of said speaking user on the display, even when said speaking user is not said at least one forward user.

8. A computer-readable program embodied on a non-transitory medium which makes a computer function as a conference terminal, said conference terminal comprising:

an image data sending means, which sends image data of said conference terminal'"'"'s own user to another conference terminal in response to an image sending request from said another conference terminal;

a location determining means, which determines a location and a direction of said conference terminal'"'"'s own user in said virtual space;

a terminal side location information sending means, which sends virtual location information of said conference terminal'"'"'s own user, the virtual location information including the location and the direction determined by said location determining means to a presence server that manages a location, in said virtual space, of a user of each of a plurality of conference terminals;

a terminal side location information receiving means, which receives virtual location information of each of other users who are users of conference terminals other than said conference terminal, from said presence server;

a forward user detection means, which detects, as a at least one forward user, at least one of said other users located in front of said conference terminal'"'"'s own user in the virtual space, based on said information determined location and direction of said conference terminal'"'"'s own user and said virtual location information of each of said other users;

a communication control means which sends an image sending request to a conference terminal of each of said at least one forward user detected by said forward user detection means;

an image data receiving means, which receives image data of each of said at least one forward user from the conference terminal of each of the at least one forward user;

a display control means, which displays, on a display, a conference screen that includes an image of each of said at least one forward user, the image of each of said at least one forward user being formed by the image data received by the image data receiving means;

a voice data sending means, which sends voice data of said each conference terminal'"'"'s own user to said conference terminals of said other users;

a voice data receiving means, which receives voice data of said other users from the conference terminals of said other users;

a voice synthesizing means, which applies stereophonic processing on each of said other users'"'"' voice data received by said voice data receiving means, depending on a respective relative position of said at least one forward user in relation to said user in the virtual space, with said relative position being specified by the virtual location information of each of said at least one forward user and the virtual location information of said each conference terminal'"'"'s own user, and synthesizes said other users'"'"' voice data submitted to the stereophonic processing, to generate stereophonic synthesized voice data;

a voice control means, which outputs the stereophonic synthesized voice data generated by said voice synthesizing means through speakers; and

a speaking detection means, which detects speaking of said each conference terminal'"'"'s own user, wherein

said image data sending means of each of said plurality of conference terminals sends, as image data of a speaking user, the image data of said each conference terminal'"'"'s own user to the conference terminals of said other users together with an indication of a speaking user when said speaking detection means detects speaking of said each conference terminal'"'"'s own user;

said image data receiving means of each of said plurality of conference terminals receives the image data indicated as image data of a speaking user; and

said display control means of each of said plurality of conference terminals displays the conference screen that further includes an image of said speaking user on the display, even when said speaking user is not said at least one forward user.