Conversation support apparatus, control method of conversation support apparatus, and program for conversation support apparatus

• US 9,691,387 B2
• Filed: 10/27/2014
• Issued: 06/27/2017
• Est. Priority Date: 11/29/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Nakadai, Kazuhiro, Nakamura, Keisuke
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US14/524,068
• Publication Number:
  US 20150154957A1
• Time in Patent Office: 974 Days
• Field of Search: 704235
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/0272 Voice signal separating

Abstract

A conversation support apparatus includes: a speech input unit configured to input speech signals of two or more users; a speech recognizing unit configured to recognize the speech signals input from the speech input unit; a display unit configured to display the recognition results of the speech recognizing unit; and an image processing unit configured to set display areas respectively corresponding to the users into an image display area of the display unit.

Claims

1. A single conversation support apparatus which is formed into a plate shape comprises:

two or more microphones configured to input speech signals of two or more users;

a processor configured to recognize the inputted speech signals from the two or more microphones, wherein the processor is configured to convert the inputted speech signals into recognition results; and

one display unit configured to display the recognition results of the processor, wherein the two or more microphones are disposed separately from each other in a same plane as the single conversation support apparatus, wherein the two or more microphones and the display unit are included within the single conversation support apparatus;

wherein the processor is configured to;

divide a display area of the one display unit to respectively correspond to the users;

estimate sound source directions of the users based on signal levels of the inputted speech signals by the two or more microphones; and

display the recognition results of the processor to be listed in time-series in the divided display areas, which respectively correspond to the users, of the one display unit at display angles respectively facing front when viewed from the estimated sound source directions, wherein the recognition results in each of the divided display areas are aligned in accordance with the estimated sound source directions so as to face at least one of the users.

2. The single conversation support apparatus according to claim 1, wherein the display of the recognition results in the divided display areas of the one display unit, which respectively correspond to the users, is based on the estimated sound source directions.

3. The single conversation support apparatus according to claim 1, further comprising a sound source separating unit configured to separate the speech signals input from the speech input unit by the users,

wherein the display of the recognition results of the speech signals of the users other than one of the users corresponding to one of the divided display areas, out of the speech signals by the users separated by the sound source separating unit, in the one divided display area.

4. The single conversation support apparatus according to claim 1, further comprising a position estimating unit configured to estimate positions of the users,

wherein the divided display areas are reset or rearranged respectively corresponding to the users at positions respectively corresponding to the positions of the users estimated by the position estimating unit.

5. The single conversation support apparatus according to claim 4, wherein the position estimating unit estimates the positions of the users using the speech signals input from the two or more microphones.

6. The single conversation support apparatus according to claim 1, further comprising a translation unit configured to translate the recognition results of the speech recognizing unit,

wherein the translation results of the translation unit is displayed in the divided display areas, which respectively correspond to the users, of the one display unit.

7. The single conversation support apparatus according to claim 6, further comprising a language information detecting unit configured to detect languages uttered by the users,

wherein the translation unit translates the recognition results of the speech signals of the users other than one of the users corresponding to one of the divided display areas into the language of the one user detected by the language information detecting unit.

8. The single conversation support apparatus according to claim 1, further comprising a communication unit configured to communicate with another conversation support apparatus,

wherein the two or more microphones input a speech signal received by the communication unit from the another conversation support apparatus, and

wherein the processor recognizes a speech signal of an user other than the users corresponding to the divided display areas out of the speech signals input from the speech input unit.

9. The single conversation support apparatus according to claim 8, further comprising an input unit configured to select a part of an image displayed on the one display unit,

wherein the image processing unit displays other recognition candidates corresponding to the selected part of the image on the one display unit when the part of the image selected by the input unit is the recognition result displayed on the one display unit, corrects the recognition result to the recognition candidate selected by the input unit out of the recognition candidates, and transmits the corrected recognition result to the another conversation support apparatus through the use of the communication unit.

10. A control method of a single conversation support apparatus which is formed into a plate shape, comprising:

inputting speech signals of two or more users at two or more microphones;

recognizing the inputted speech signals at a processor, and converting the inputted speech signals into recognition results;

dividing one display area of a display unit to respectively correspond to the users, wherein the one display unit is configured to display the recognition results of the processor, wherein the two or more microphones and the display unit are included within the single conversation support apparatus;

estimating a sound source direction of the users based on signal levels of the inputted speech signals by the two or more microphones; and

displaying the recognition results of the processor to be listed in time-series in the divided display area, which respectively correspond to the users, of the one display unit at display angles respectively facing front when viewed from the estimated sound source directions, wherein the two or more microphones are disposed separately from each other in a same plane as the single conversation support apparatus, wherein the recognition results in each of the divided display areas are aligned in accordance with the estimated sound source directions so as to face at least one of the users.

11. A non-transitory computer-readable medium encoding instructions that, when executed in hardware, perform a process, the process comprising:

inputting speech signals of two or more users at two or more microphones;

recognizing the inputted speech signals at a processor, and converting the inputted speech signals into recognition results;

dividing one display area of a display unit to respectively correspond to the users, wherein the one display unit is configured to display the recognized speech signals of the processor, wherein the two or more microphones and the display unit are included within a single conversation support apparatus;

estimating a sound source direction of the users based on signal levels of the inputted speech signals by the two or more microphones; and

displaying the recognized speech signals of the processor are listed in time-series in the divided display area, which respectively correspond to the users, of the display unit at display angles respectively facing front when viewed from the estimated sound source directions, wherein the two or more microphones are disposed separately from each other in a same plane as the single conversation support apparatus, wherein the recognition results in each of the divided display areas are aligned in accordance with the estimated sound source directions so as to face at least one of the users.