Input auxiliary apparatus, input auxiliary method, and program

• US 9,728,189 B2
• Filed: 03/27/2012
• Issued: 08/08/2017
• Est. Priority Date: 04/26/2011
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Fujibayashi, Toshihiko
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US14/113,897
• Publication Number:
  US 20140052441A1
• Time in Patent Office: 1,960 Days
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

An object of the present invention is to provide an input auxiliary apparatus equipped with an input section to input character strings; an embellishment information retaining section to retain embellishment information on a plurality of postures in a storing section in advance to link each posture with the embellishment information; a posture detecting section to detect the posture; a reading section to read out the embellishment information linked with the posture detected by the posture detecting section from the storing section; and an embellishment applying section to apply the embellishment information read out by the reading section to the character strings. The input section preferably includes a speech recognition section to recognize voice data based on speech recognition and convert the voice data to the character strings. Accordingly, this enables emotions of a speaker to be correctly judged and suitable embellishments to be appended when performing speech recognition.

Claims

1. An input auxiliary apparatus comprising:

a voice input section for inputting a voice;

an embellishment information retaining section for retaining embellishment information on a plurality of posture patterns in a storing section in advance such that each piece of the embellishment information links each posture pattern;

a posture detecting section for detecting posture; and

a control section for being operable as;

by a speech recognition instruction by a user, recognizing, for each speech unit, voice inputted from the voice input section, as a character string,

determining, for the each speech unit, a posture pattern instructed by the user, based the posture detected by the posture detecting section during the speech recognition,

reading out an embellishment information linked with the posture pattern determined for the each speech unit from the storing section; and

applying the embellishment information read out for each speech unit to the character string recognized for the each speech unit,

wherein the embellishment information is character color of the character string, character size of the character string, and an emoticon or a pictograph added following the character string,

wherein the embellishment information retaining section also retains emotion information or season information such that each piece of the emotion information or season information is linked with the each posture pattern,

wherein, during the speech recognition, the control section sequentially changes a display mode of a display section based on a piece of the emotion information or season information linked with a posture pattern including displacement of the posture being detected by the posture detecting section, and

wherein the display mode is an avatar, or a mark or a diagram which represents an emotion.

2. The input auxiliary apparatus according to claim 1 further comprising a receiving section for receiving in an external signal corresponding to recognition results of the speech recognition section.

3. The input auxiliary apparatus according to claim 1,

wherein, in the emotion information or season information, different emotions or seasons are retained for different directions included in the plurality of posture patterns, and different levels of an emotion or a season corresponding to an arbitrary direction are retained for different angles of the arbitrary direction.

4. An input auxiliary method comprising:

a step of inputting a voice;

a step of retaining embellishment information on a plurality of posture patterns in a storing section in advance such that each piece of the embellishment information links each posture pattern;

a step of detecting the posture;

a step of determining, for the each speech unit, a posture pattern instructed by the user, based the detected posture during the speech recognition;

a step of reading out an embellishment information linked with the posture pattern determined for the each speech unit from the storing section; and

a step of applying the embellishment information read out for the each speech unit to the character string recognized for the each speech unit,

wherein the embellishment information is character color of the character string, character size of the character string, and an emoticon or a pictograph added following the character string,

wherein the embellishment information retaining step also retains emotion information or season information such that each piece of the emotion information or season information is linked with each posture pattern,

wherein, during the speech recognition, a display mode of a display section is changed based on a piece of the emotion information or season information linked with a posture pattern including displacement of the posture being detected, and

wherein the display mode being an avatar, or a mark or a diagram which represents an emotion.

5. A non-transitory computer-readable storage medium having a program stored thereon which causes a computer to function as:

a voice input section for inputting voice;

an embellishment information retaining section for retaining embellishment information on a plurality of posture patterns in a storing section in advance such that each piece of the embellishment information links each posture pattern;

a posture detecting section for detecting posture; and

a control section for being operable as;

by a speech recognition instruction by a user, recognizing, for each speech unit, voice inputted from the voice input section, as a character string;

determining, for the each speech unit, a posture pattern instructed by the user, based the posture detected by the posture detecting section during the speech recognition,

reading out an embellishment information linked with the posture pattern determined for the each speech unit from the storing section; and

applying the embellishment information read out for the each speech unit to the character string recognized for the each speech unit,

wherein the embellishment information is character color of the character string, character size of the character string, and an emoticon or a pictograph added following the character string,

wherein the embellishment information retaining section also retains emotion information or season information such that each piece of the emotion information or season information is linked with the each posture pattern,

wherein, during the speech recognition, the control section sequentially changes a display mode of a display section based on a piece of the emotion information or season information linked with a posture pattern including displacement of the posture being detected by the posture detecting section, and

wherein the display mode being an avatar, or a mark or a diagram which represents an emotion.