Method for recognizing speech sound, mobile terminal, and recording medium

• US 10,331,795 B2
• Filed: 09/22/2017
• Issued: 06/25/2019
• Est. Priority Date: 09/28/2016
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: Matsuoka, Tomomi
• Primary Examiner(s): Yen, Eric
• Application Number:
  US15/713,088
• Publication Number:
  US 20180089175A1
• Time in Patent Office: 641 Days
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/47 Machine-assisted translation, e.g. using translation memory
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information

Abstract

Speech information including a speech sound uttered by a user is obtained. A language of the speech sound included in the obtained speech information is recognized. A first determination for determining whether the recognized language is a first language registered to a memory in advance is made. If it is determined as a result of the first determination that the recognized language is the first language, the speech sound is translated into a second language, which is different from the first language.

Claims

1. A method, comprising:

obtaining, using a microphone of a mobile terminal, speech information, including a speech sound uttered by a user;

recognizing, using a server, a language of the speech sound included in the obtained speech information;

detecting, using the mobile terminal, a position of the mobile terminal;

making, using the mobile terminal, a first determination for determining whether the recognized language is a first language indicated by language information registered to a memory in advance, wherein when the recognized language is an official language of an area including the detected position, it is determined in the first determination that the recognized language is not the first language; and

translating, using the mobile terminal, when it is determined as a result of the first determination that the recognized language is the first language, the speech sound into a second language, which is different from the first language.

2. The method according to claim 1, further comprising:

receiving, using an input interface of the mobile terminal, the language information indicating the first language from the user; and

registering the received language information to the memory,

wherein the first determination is made after the registering.

3. The method according to claim 1, further comprising:

making a second determination for determining whether the detected position is included in a second area of which an official language is different from the first language,

wherein the second area is different from a first area of which an official language is the same as the first language, and

wherein, when it is determined as a result of the second determination that the detected position is included in the second area, the official language of the second area is used as the second language in the translating.

4. The method according to claim 1, further comprising:

obtaining notification information output from a certain application;

making a second determination for determining on the basis of the detected position and the obtained notification information whether translation is necessary at a current position of the mobile terminal in a current state of the user; and

performing, when it is determined as a result of the second determination that translation is necessary at the current position of the mobile terminal in the current state of the user, the first determination and the translation.

5. The method according to claim 4,

wherein a place where translation is necessary is an airport, a hotel, a restaurant, a store, or a station.

6. The method according to claim 4,

wherein it is determined in the second determination that translation is necessary at the current position of the mobile terminal in the current state of the user (1) when the detected position is an airport and the obtained notification information indicates that the user has checked in at the airport, (2) when the detected position is a hotel and the obtained notification information indicates that the user has checked in at the hotel, (3) when the detected position is a restaurant or a store and the obtained notification information indicates that the user has checked a coupon at the restaurant or the store, or (4) when the detected position is a station and the obtained notification information indicates that the user has gone through a gate at the station.

7. The method according to claim 1, further comprising:

detecting acceleration of a mobile terminal;

making a second determination for determining on the basis of the detected acceleration whether the user carrying the mobile terminal is moving at a certain speed or higher; and

performing, when it is determined as a result of the second determination that the user is not moving at the certain speed or higher, the first determination and the translation.

8. The method according to claim 1, further comprising:

measuring a noise level around a mobile terminal;

making a second determination for determining whether the measured noise level is higher than a certain noise level; and

performing, when it is determined as a result of the second determination that the noise level is equal to or lower than the certain noise level, the first determination and the translation.

9. A mobile terminal, comprising:

a microphone;

a processor; and

a memory,

wherein the processor

extracts speech information, including a speech sound uttered by a user from sounds collected by the microphone,

causes a server to recognize a language of the speech sound included in the extracted speech information, wherein the server is connected to the mobile terminal,

detects a position of the mobile terminal,

makes a first determination for determining whether the recognized language is a first language indicated by language information registered to the memory in advance, wherein when the recognized language is an official language of an area including the detected position, it is determined in the first determination that the recognized language is not the first language, and

translates, when it is determined as a result of the first determination that the recognized language is the first language, the speech sound into a second language, which is different from the first language.

10. A non-transitory recording medium, storing a program for causing a computer to perform a process, comprising:

extracting speech information, including a speech sound uttered by a user from sounds collected by a microphone;

causing a server to recognize a language of the speech sound included in the extracted speech information, wherein the server is connected to the computer;

detecting a position of the computer;

making a first determination for determining whether the recognized language is a first language indicated by language information registered to a memory of the computer in advance, wherein when the recognized language is an official language of an area including the detected position, it is determined in the first determination that the recognized language is not the first language; and

translating, when it is determined as a result of the first determination that the recognized language is the first language, the speech sound into a second language, which is different from the first language.