Video conferencing system, conference terminal and image server

• US 20060206560A1
• Filed: 02/17/2006
• Published: 09/14/2006
• Est. Priority Date: 03/11/2005
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Kanada, Yasusi
• Granted Patent: US 7,835,764 B2
• Time in Patent Office: Days
• US Class Current: 709/201
• CPC Class Codes: H04L 61/4535 using an address exchange platform which sets up a session between two nodes, e.g. rendezvous servers, session initiation protocols [SIP] registrars or H.323 gatekeepers
• CPC Class Codes: H04L 61/4557 Directories for hybrid networks, e.g. including telephone numbers
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/1104 Session initiation protocol [SIP]
• CPC Class Codes: H04L 65/4038 with floor control

Abstract

In a video conferencing system, required bandwidth is suppressed to a lower level and a key person or the like of a conference can be confirmed easily. A presence server 1 manages location information in a virtual space for a user of each conference terminal 2. Each conference terminal 2 sends image data of its own user to another conference terminal 2 in response to a request of that conference terminal 2. Further, each conference terminal 2 receives location information of its own user in the virtual space, sends the received location information to the presence server 1, and receives location information of the other users from the presence server 1. Then, based on the location information of its own user and the location information of the other users, the conference terminal 2 detects forward users located forward from its own user, and receives image data from the conference terminal 2 of each of the forward users.

Claims

1. A video conferencing system comprising:

a plurality of conference terminals and a presence server for managing a location of a user of each of said plurality of conference terminals in a virtual space, wherein;

each of said plurality of conference terminals comprises;

an image data sending means, which sends image data of its-own-user, i.e. a user of the conference terminal in question (its-own conference terminal) to another conference terminal in response to an image sending request from said another conference terminal, i.e. a source of the request;

a location receiving means, which receives a location and a direction of said its-own-user in said virtual space;

a terminal side location information sending means, which sends its-own-user'"'"'s virtual location information that includes the location and the direction received by said location receiving means to said presence server;

a terminal side location information receiving means, which receives virtual location information of each of other-users, i.e. users of conference terminals other than its-own conference terminal from said presence server;

a forward user detection means, which detects, as a forward user or forward users, at least one other-user located forward from its-own-user in the virtual space, based on said virtual location information of its-own-user and said virtual location information of each of the other-users;

an image data receiving means, which sends an image sending request to a conference terminal of each of the forward users detected by said forward user detection means and receives image data of each of said forward users from the conference terminal of the forward user in question; and

a display control means, which displays, on a display, a conference screen that includes the image data (received by said image data receiving means) of each of said forward users; and

said presence server comprises;

a management means, which manages users′

virtual location information sent from each of said plurality of conference terminals; and

a sever side location information sending means, which sends, to each of said plurality of conference terminals, the location information of the user of each of the conference terminals other than the conference terminal in question.

2. A video conferencing system according to claim 1, wherein:

the display control means of each of said plurality of conference terminals allocates, in order, image data of said forward users respectively to a plurality of display frames provided in said conference screen, according to relative positions of said forward users in relation to said its-own-user in the virtual space, with said relative positions being specified by the virtual location information of each of said forward users and the virtual location information of said its-own-user.

3. A video conferencing system according to claim 1, wherein:

each of said plurality of conference terminals further comprises;

a voice data sending means, which sends voice data of said its-own-user to each of the conference terminals other than the conference terminal in question;

a voice data receiving means, which receives voice data of other-user from each of the conference terminals other than the conference terminal in question;

a voice synthesizing means, which applies stereophonic processing on each of the other-users′

voice data received by said voice data receiving means, depending on respective relative positions of said forward users in relation to said its-own-user in the virtual space, with said relative positions being specified by the virtual location information of each of said forward users and the virtual location information of said its-own-user, and synthesizes the other-users′

voice data submitted to the stereophonic processing, to generate stereophonic synthesized voice data;

a voice control means, which outputs the stereophonic synthesized voice data generated by said voice synthesizing means through speakers; and

a speaking detection means, which detects speaking of its-own-user; and

said image data sending means of each of said plurality of conference terminals sends, as image data of a speaking user, the image data of the user of the conference terminal in question (its-own-user) to each of the conference terminals other than the conference terminal in question when said speaking detection means detects speaking of said its-own-user;

said image data receiving means of each of said plurality of conference terminals receives the image data of said speaking user; and

said display control means of each of said plurality of conference terminals includes the image data (which is received by said image data receiving means) of said speaking user in said conference screen and displays the conference screen on the display, when said image data of said speaking user is image data of another user than said forward users.

4. A video conferencing system according to claim 3, wherein:

said display control means of each of said plurality of conference terminals allocates the image data (which is received by said image data receiving means) of said speaking user to a display frame positioned on a right side or a left side of display frames to which image data of said forward users are respectively allocated in said conference screen, according to a relative position of said speaking user and said its-own-user in the virtual space, with said relative position being specified by the location information of said its-own-user and the location information of said speaking user, when the image data of the speaking user is image data of another user than the forward users.

5. A video conferencing system comprising:

a plurality of conference terminals, a presence server for managing a location of a user of each of said plurality of conference terminals in a virtual space, and an image server for sending image data to each of said plurality of conference terminals, wherein;

each of said plurality of conference terminals comprises;

a terminal side image data sending means, which sends image data of its-own-user, i.e., a user of the conference terminal in question (its-own conference terminal) to said image server;

a location receiving means, which receives a location and a direction of said its-own-user in said virtual space;

a terminal side location information sending means, which sends its-own-user'"'"'s virtual location information that includes the location and the direction received by said location receiving means to said presence server;

a terminal side image data receiving means, which receives conference screen data from said image server; and

a display control means, which displays, on a display, the conference screen data received by said terminal side image data receiving means;

said image server comprises;

an image server side image data receiving means, which receives image data of a user of each of said plurality of conference terminals;

an image server side location information receiving means, which receives virtual location information of each of said plurality of conference terminals from said presence server;

a forward user detection means, which detects, as a forward user or forward users, at least one other-user (i.e. a user other than a user of a conference terminal in question) located forward from its-own-user (i.e. the user of the conference terminal in question) in the virtual space, based on the virtual location information of its-own-user and the virtual location information of each of other-users, for each of said plurality of conference terminals; and

an image server side image data sending means, which sends, to each of said plurality of conference terminals, conference screen data including the image data of each of the forward users detected in relation to the conference terminal in question by said forward user detection means, out of image data (received by said image sever side image data receiving means) of respective users of said plurality of conference terminals;

said presence server comprises;

a presence server side location information receiving means, which receives user'"'"'s virtual location information sent from each of said plurality of conference terminals;

a management means, which manages user'"'"'s virtual location information sent from each of said plurality of conference terminals; and

a presence server side location information sending means, which sends the virtual location information of each of said plurality of conference terminals to said image server, with said virtual location information being managed by said management means.

6. A video conferencing system according to claim 5, wherein:

said image server side image data sending means of said image server determines, for each of said plurality of conference terminals, a layout position of image data of each of said forward users in said conference screen, according to relative positions of said forward users detected by said forward user detection means in relation to a user (its-own-user) of the conference terminal in question in the virtual space, and sends, to the conference terminal in question, the conference screen data in which the image data of the forward users are respectively allocated, in order, to a plurality of display frames provided in said conference screen.

7. A video conferencing system according to claim 5, wherein:

said video conferencing system further comprises a voice server for sending voice data to each of said plurality of conference terminals;

each of said plurality of conference terminals further comprises;

a voice data sending means, which sends voice data of its-own-user, i.e. a user of the conference terminal in question to said voice server;

a stereophonic synthesized voice data receiving means, which receives stereophonic synthesized voice data from said voice server;

a voice control means, which outputs the stereophonic synthesized voice data received by said stereophonic synthesized voice data receiving means through speakers; and

a speaking detection means, which detects speaking of said its-own-user and sends a speaking detection signal to said image server;

said voice server comprises;

a voice data receiving means, which receives voice data of a user of each of said plurality of conference terminals;

a voice server side location information receiving means, which receives virtual location information of a user of each of said plurality of conference terminals from said presence server;

a voice synthesizing means, which applies, for each of said plurality of conference terminals, stereophonic processing on each of voice data of other-users (i.e. users of conference terminals other than the conference terminal in question) among voice data received by said voice data receiving means, depending on respective relative positions of said forward users in relation to its-own-user (i.e. a user of the conference terminal in question) in the virtual space, with said relative positions being specified by the virtual location information of each of said forward users and the virtual location information of said its-own-user, and synthesizes the other-users'"'"' voice data submitted to the stereophonic processing, to generate stereophonic synthesized voice data; and

a stereophonic synthesized voice data sending means, which sends the stereophonic synthesized voice data generated by said voice synthesizing means with respect to each of said plurality of conference terminals to the conference terminal in question; and

said image server further comprises a speaking detection signal receiving means, which receives the speaking detection signal from each of said plurality of conference terminals; and

when said speaking detection signal receiving means receives a speaking detection signal, said image server side image data sending means of said image server sends image data of a user as a source of said speaking signal to each of said plurality of conference terminals if said user is other than the forward users detected by said forward user detection means with respect to the conference terminal in question, with said image data being included as image data of a speaking user in the conference screen data.

8. A video conferencing system according to claim 7, wherein:

for each of said plurality of conference terminals, said image server side image data sending means of said image server determines a layout position of the image data (which is sent to the conference terminal in question) of the speaking user in the conference screen, depending on a relative position of said speaking user in relation to a user of the conference terminal in question in the virtual space, and sends the conference screen data to the conference terminal in question, with the image data of said speaking user being allocated based on said determined layout position to a display frame positioned in said conference screen on a right side or left side of display frames to which image data of said forward users are respectively allocated in said conference screen data.

9. A conference terminal, comprising:

an image data sending means, which sends image data of its-own-user, i.e. a user of the conference terminal in question (its-own conference terminal) to another conference terminal in response to an image sending request from said another conference terminal, i.e. a source of the request;

a location receiving means, which receives a location and a direction of said its-own-user in said virtual space;

a terminal side location information sending means, which sends its-own-user'"'"'s virtual location information that includes the location and the direction received by a location receiving means to a presence server that manages a location, in said virtual space, of a user of each of a plurality of conference terminals participating in a video conference;

a terminal side location information receiving means, which receives virtual location information of each of other-users, i.e. users of conference terminals other than its-own conference terminal from said presence server;

a forward user detection means, which detects, as a forward user or forward users, at least one other-user located forward from its-own-user in the virtual space, based on said virtual location information of its-own-user and said virtual location information of each of the other-users;

an image data receiving means, which sends an image sending request to a conference terminal of each of the forward users detected by said forward user detection means and receives image data of each of said forward users from the conference terminal of the forward user in question; and

a display control means, which displays, on a display, a conference screen that includes the image data (received by said image data receiving means) of each of said forward users.

10. A computer-readable program whichmakes a computer function as a conference terminalsaid conference terminal comprising:

an image data sending means, which sends image data of its-own-user, i.e. a user of the conference terminal in question (its-own conference terminal) to another conference terminal in response to an image sending request from said another conference terminal, i.e. a source of the request;

a location receiving means, which receives a location and a direction of said its-own-user in said virtual space;

a terminal side location information sending means, which sends its-own-user'"'"'s virtual location information that includes the location and the direction received by a location receiving means to a presence server that manages a location, in said virtual space, of a user of each of a plurality of conference terminals;

that manages a location, in said virtual space, of a user of each of a plurality of conference terminals;

a terminal side location information receiving means, which receives virtual location information of each of other-users, i.e. users of conference terminals other than its-own conference terminal from said presence server;

a forward user detection means, which detects, as a forward user or forward users, at least one other-user located forward from its-own-user in the virtual space, based on said virtual location information of its-own-user and said virtual location information of each of the other-users;

an image data receiving means, which sends an image sending request to a conference terminal of each of the forward users detected by said forward user detection means and receives image data of each of said forward users from the conference terminal of the forward user in question; and

a display control means, which displays, on a display, a conference screen that includes the image data (received by said image data receiving means) of each of said forward users.

11. An image server, which sends image data to each of plurality of conference terminals, comprising:

an image server side image data receiving means, which receives image data of a user of each of said plurality of conference terminals;

an image server side location information receiving means, which receives virtual location information of each of said plurality of conference terminals from a presence server that manages a location, in a virtual space, of a user of each of a plurality of conference terminals;

a forward user detection means, which detects, as a forward user or forward users, at least one other-user (i.e. a user other than a user of a conference terminal in question) located forward from its-own-user (i.e. the user of the conference terminal in question) in the virtual space, based on the virtual location information of its-own-user and the virtual location information of each of other-users, for each of said plurality of conference terminals; and

an image server side image data sending means, which sends, to each of said plurality of conference terminals, conference screen data including the image data of each of the forward users detected in relation to the conference terminal in question by said forward user detection means, out of image data (received by said image sever side image data receiving means) of respective users of said plurality of conference terminals.

12. A computer-readable program, which makes a computer function as an image server, said image server comprising:

an image server side image data receiving means, which receives image data of a user of each of a plurality of conference terminals;

an image server side location information receiving means, which receives virtual location information of each of said plurality of conference terminals from a presence server that manages a location, in a virtual space, of a user of each of a plurality of conference terminals;

a forward user detection means, which detects, as a forward user or forward users, at least one other-user (i.e. a user other than a user of a conference terminal in question) located forward from its-own-user (i.e. the user of the conference terminal in question) in the virtual space, based on the virtual location information of its-own-user and the virtual location information of each of other-users, for each of said plurality of conference terminals; and

an image server side image data sending means, which sends, to each of said plurality of conference terminals, conference screen data including the image data of each of the forward users detected in relation to the conference terminal in question by said forward user detection means, out of image data (received by said image sever side image data receiving means) of respective users of said plurality of conference terminals.