Meeting visualization system

• US 20080255847A1
• Filed: 04/01/2008
• Published: 10/16/2008
• Est. Priority Date: 04/12/2007
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Moriwaki, Norihiko, Imaki, Tsuneyuki, Kashiyama, Toshihiko, Nishizawa, Itaru, Sato, Nobuo, Egi, Masashi
• Granted Patent: US 8,290,776 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Voice of plural participants during a meeting is obtained and dialogue situations of the participants that change every second are displayed in real time, so that it is possible to provide a meeting visualization system for triggering more positive discussions. Voice data collected from plural voice collecting units associated with plural participants is processed by a voice processing server to extract speech information. The speech information is sequentially input to an aggregation server. A query process is performed for the speech information by a stream data processing unit of the aggregation server, so that activity data such as the accumulation value of speeches of the participants in the meeting is generated. A display processing unit visualizes and displays dialogue situations of the participants by using the sizes of circles and the thicknesses of lines on the basis of the activity data.

Claims

1. A meeting visualization system which visualizes and displays dialogue situations among a plurality of participants in a meeting, the system comprising:

a plurality of voice collecting units which are associated with the participants;

a voice processing unit which processes voice data collected from the voice collecting units to extract speech information;

a stream processing unit to which the speech information extracted by the voice processing unit is sequentially input and which performs a query process for the speech information so as to generate activity data of the participants in the meeting; and

a display processing unit which visualizes and displays the dialogue situations of the participants on the basis of the activity data generated by the stream processing unit.

2. The meeting visualization system according to claim 1,

wherein the activity data include the number of accumulated speeches for each participant, and the number of dialogues among the participants.

3. The meeting visualization system according to claim 1, further comprising a key stroke information input unit with which a speaker among the plurality of participants and speech contents are input as key stroke information,

wherein the stream processing unit performs a query process for the key stroke information, so that the speech contents of the participants are extracted as the activity data.

4. The meeting visualization system according to claim 1,

wherein the speech information is voice energy extracted from the voice data.

5. The meeting visualization system according to claim 4,

wherein the stream processing unit determines the participant associated with the voice collecting unit that outputs the maximum value of the voice energy, as the speaker.

6. The meeting visualization system according to claim 1,

wherein the stream processing unit has a participant registration interface for associating the voice collecting units with the participants.

7. The meeting visualization system according to claim 6, comprising a detector which detects IDs (identifiers) indicating seated positions of the participants,

wherein the participant registration interface of the stream processing unit associates the voice collecting units with the participants on the basis of the IDs from the detector.

8. A meeting visualization method in a server where a voice process and an aggregation are performed for voice data collected from a plurality of microphones associated with a plurality of participants in a meeting and dialogue situations among the participants are displayed, the method comprising the steps of:

extracting stream data of speech information by performing a voice process for the voice data collected from the plurality of microphones associated with the participants;

generating activity data of the participants in the meeting by performing a query process for the stream data of the speech information; and

displaying dialogue situations among the participants on the basis of the activity data.

9. The meeting visualization method according to claim 8,

wherein the speech information is voice energy extracted from the voice data.

10. The meeting visualization method according to claim 8, further comprising the steps of:

receiving a speaker among the plurality of participants and speech contents as key stroke information; and

generating the speech contents of the participants as the activity data by performing a query process for the key stroke information.

11. The meeting visualization method according to claim 8,

wherein the activity data include the accumulation of speeches for each participant, and the number of dialogues among the participants.

12. The meeting visualization method according to claim 8,

wherein the activity data include a discussion activation degree determined by using a total number of speeches of the participants and a total number of participants who made speeches, per unit time.

13. The meeting visualization method according to claim 9, further comprising the step of associating the microphones with the participants.

14. The meeting visualization method according to claim 13,

wherein in the step of generating the activity data, the participant associated with the microphone that outputs the maximum value of the voice energy is determined as the speaker.

15. An aggregation server in a meeting visualization system, comprising:

a stream processing unit which performs a query process for speech information that is stream data extracted by processing voice data of a plurality of participants in a meeting, so that activity data of the participants in the meeting is calculated; and

a display processing unit which visualizes and displays the activity data input from the stream processing unit.

16. The aggregation server according to claim 15,

wherein the stream processing unit performs a query process for key stroke information relating to a speaker and speech contents, so that the speech contents of the participants are extracted as the activity data.

17. The aggregation server according to claim 15,

wherein the stream processing unit executes, as the query process for the speech information, a query for detecting the maximum value of the speech information associated with the plurality of participants at a given time, and a query for specifying the participant associated with the detected maximum value of the speech information as the speaker to generate a stream of the speaker.

18. The aggregation server according to claim 17,

wherein when executing the query for generating a stream of the speaker as the query process for the speech information, the stream processing unit specifies the speaker only in the case where the detected maximum value of the speech information exceeds a predetermined threshold.

19. The aggregation server according to claim 17,

wherein the stream processing unit complements intermittent portions of the same continuous speaker in the generated stream so as to extract a smoothed speech period.

20. The aggregation server according to claim 19,

wherein when extracting the smoothed speech period from the generated stream of the speaker, the stream processing unit deletes a momentary speech period.