Conference support device, conference support method, and computer-readable medium storing conference support program

• US 20100250252A1
• Filed: 03/12/2010
• Published: 09/30/2010
• Est. Priority Date: 03/27/2009
• Status: Active Grant

• Current Assignee: Brother Kogyo Kabushiki Kaisha
• Original Assignee: Brother Kogyo Kabushiki Kaisha
• Inventors: Yasoshima, Mizuho
• Granted Patent: US 8,560,315 B2
• Time in Patent Office: Days
• US Class Current: 704/246
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A conference support device includes an image receiving portion that receives captured images from conference terminals, a voice receiving portion that receives, from one of the conference terminals, a voice that is generated by a first participant, a first storage portion that stores the captured images and the voice, a voice recognition portion that recognizes the voice, a text data creation portion that creates text data that express the words that are included in the voice, an addressee specification portion that specifies a second participant, whom the voice is addressing, an image creation portion that creates a display image that is configured from the captured images and in which the text data are associated with the first participant and a specified image is associated with at least one of the first participant and the second participant, and a transmission portion that transmits the display image to the conference terminals.

Claims

1. A conference support device that, by controlling communication among a plurality of conference terminals, supports a video conference that is conducted among conference participants who are using the conference terminals, the conference support device comprising:

an image receiving portion that receives, from the plurality of the conference terminals, captured images that are captured by image capture devices of the conference terminals and in each of which at least one of the conference participants is visible;

a voice receiving portion that receives, from a first conference terminal that is one of the plurality of the conference terminals, a voice that is generated by a first participant, who is the conference participant who is using the first conference terminal, the voice being input from a voice input device of the first conference terminal;

a first storage portion that stores the captured images that are received by the image receiving portion and the voice that is received by the voice receiving portion;

a voice recognition portion that recognizes the voice that is stored in the first storage portion;

a text data creation portion that creates text data that express words that are included in the voice, based on the result of the recognizing by the voice recognition portion;

an addressee specification portion that, based on the result of the recognizing by the voice recognition portion, specifies a second participant, who is the conference participant whom the voice is addressing;

an image creation portion that creates a display image that is to be displayed on display devices of the plurality of the conference terminals from the captured images that are stored in the first storage portion, and in which the text data that are created by the text data creation portion are associated with a portion of the captured image that corresponds to the first participant and a specified image is associated with a portion of the captured image that corresponds to at least one of the first participant and the second participant who is specified by the addressee specification portion; and

a transmission portion that transmits, to the plurality of the conference terminals, the display image that is created by the image creation portion, in order for the display image to be displayed on the display devices of the plurality of the conference terminals.

2. The conference support device according to claim 1, wherein

the image creation portion creates the display image by associating a display frame, for the text data as the specified image, with the portion of the captured image that corresponds to the second participant.

3. The conference support device according to claim 1, wherein

the addressee specification portion, in a case where the result of the recognizing by the voice recognition portion includes a specified word, specifies all of the conference participants except the first participant as the second participant, and

the image creation portion, in a case where the addressee specification portion specifies all of the conference participants except the first participant as the second participant, creates the display image by associating the specified image with the portions of the captured image that correspond to the second participant.

4. The conference support device according to claim 2, wherein

the addressee specification portion, in a case where the result of the recognizing by the voice recognition portion includes a specified word, specifies all of the conference participants except the first participant as the second participant, and

the image creation portion, in a case where the addressee specification portion specifies all of the conference participants except the first participant as the second participant, creates the display image by associating the specified image with the portions of the captured image that correspond to the second participant.

5. The conference support device according to claim 1, further comprising:

a second storage portion that stores participant images that are images of the conference participants in association with first specifying information that specifies the conference participants,

wherein;

the voice receiving portion receives, along with the voice that is generated by the first participant, second specifying information that specifies the first participant; and

the image creation portion creates the display image by specifying the portion of the captured image that corresponds to the first participant, based on the participant image, among the participant images that are stored in the second storage portion, that is associated with the first specifying information that is the same as the second specifying information that is received by the voice receiving portion, and by associating the text data with the specified portion of the captured image.

6. The conference support device according to claim 2, further comprising:

a second storage portion that stores participant images that are images of the conference participants in association with first specifying information that specifies the conference participants,

wherein;

the voice receiving portion receives, along with the voice that is generated by the first participant, second specifying information that specifies the first participant; and

the image creation portion creates the display image by specifying the portion of the captured image that corresponds to the first participant, based on the participant image, among the participant images that are stored in the second storage portion, that is associated with the first specifying information that is the same as the second specifying information that is received by the voice receiving portion, and by associating the text data with the specified portion of the captured image.

7. The conference support device according to claim 3, further comprising:

a second storage portion that stores participant images that are images of the conference participants in association with first specifying information that specifies the conference participants,

wherein;

the voice receiving portion receives, along with the voice that is generated by the first participant, second specifying information that specifies the first participant; and

the image creation portion creates the display image by specifying the portion of the captured image that corresponds to the first participant, based on the participant image, among the participant images that are stored in the second storage portion, that is associated with the first specifying information that is the same as the second specifying information that is received by the voice receiving portion, and by associating the text data with the specified portion of the captured image.

8. The conference support device according to claim 4, further comprising:

a second storage portion that stores participant images that are images of the conference participants in association with first specifying information that specifies the conference participants,

wherein

the voice receiving portion receives, along with the voice, second specifying information for the first participant, who is the conference participant who generates the voice, and

the image creation portion creates the display image by specifying the portion of the captured image that corresponds to the first participant, based on the participant image, among the participant images that are stored in the second storage portion, that is associated with the first specifying information that is the same as the second specifying information that is received by the voice receiving portion, and by associating the text data with the specified portion of the captured image.

9. The conference support device according to claim 1, further comprising:

a second storage portion that stores participant images that are images of the conference participants in association with first specifying information that specifies the conference participants; and

a selection portion that selects, as a second participant image, from among the participant images that are stored in the second storage portion, one of the participant images that is associated with the first specifying information for the second participant who is specified by the addressee specification portion,

wherein

the image creation portion creates the display image by associating the text data with the portion of the captured image that corresponds to the first participant and by defining as the specified image the second participant image that is selected by the selection portion and associating the selected second participant image with the portion of the captured image that corresponds to the first participant.

10. The conference support device according to claim 9, wherein

the voice receiving portion receives, along with the voice, second specifying information for the first participant, who is the conference participant who generates the voice, and

the image creation portion creates the display image by specifying the portion of the captured images that corresponds to the first participant, based on the participant image, among the participant images that are stored in the second storage portion, that is associated with the first specifying information that is the same as the second specifying information that is received by the voice receiving portion, by associating the text data with the specified portion of the captured image, and by defining as the specified image the second participant image that is selected by the selection portion and associating the selected second participant image with the portion of the captured image that corresponds to the first participant.

11. A conference support method that, by controlling communication among a plurality of conference terminals, supports a video conference that is conducted among conference participants who are using the conference terminals, the conference support method comprising the steps of:

receiving, from the plurality of the conference terminals, captured images that are captured by image capture devices of the conference terminals and in each of which at least one of the conference participants is visible;

receiving, from a first conference terminal that is one of the plurality of the conference terminals, a voice that is generated by a first participant, who is the conference participant who is using the first conference terminal, the voice being input from a voice input device of the first conference terminal;

recognizing the voice that has been received;

creating text data that express words that are included in the voice, based on the result of the recognizing of the voice;

specifying, based on the result of the recognizing of the voice, a second participant, who is the conference participant whom the voice is addressing;

creating a display image that is to be displayed on display devices of the plurality of the conference terminals from the captured images that have been received, and in which the text data that have been created are associated with a portion of the captured image that corresponds to the first participant and a specified image is associated with a portion of the captured image that corresponds to at least one of the first participant and the second participant who has been specified; and

transmitting the display image that has been created to the plurality of the conference terminals, in order for the display image to be displayed on the display devices of the plurality of the conference terminals.

12. A computer-readable medium that stores a conference support program for a conference support device that, by controlling communication among a plurality of conference terminals, supports a video conference that is conducted among conference participants who are using the conference terminals, the program comprising instructions that cause a computer of the conference support device to perform the steps of:

receiving, from the plurality of the conference terminals, captured images that are captured by image capture devices of the conference terminals and in each of which at least one of the conference participants is visible;

receiving, from a first conference terminal that is one of the plurality of the conference terminals, a voice that is generated by a first participant, who is the conference participant who is using the first conference terminal, the voice being input from a voice input device of the first conference terminal;

recognizing the voice that has been received;

creating text data that express words that are included in the voice, based on the result of the recognizing of the voice;

specifying, based on the result of the recognizing of the voice, a second participant, who is the conference participant whom the voice is addressing;

creating a display image that is to be displayed on display devices of the plurality of the conference terminals from the captured images that have been received, and in which the text data that have been created are associated with a portion of the captured image that corresponds to the first participant and a specified image is associated with a portion of the captured image that corresponds to at least one of the first participant and the second participant who has been specified; and

transmitting the display image that has been created to the plurality of the conference terminals, in order for the display image to be displayed on the display devices of the plurality of the conference terminals.