Mobile terminal and menu control method thereof

• US 9,900,414 B2
• Filed: 05/22/2017
• Issued: 02/20/2018
• Est. Priority Date: 04/08/2008
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Youn, Jong-Keun, Jung, Dae-Sung, Yu, Jae-Hoon, Kim, Tae-Jun, Joh, Jae-Min, Kwak, Jae-Do, Shin, Jong-Ho
• Primary Examiner(s): Timory, Kabir A
• Application Number:
  US15/601,922
• Publication Number:
  US 20170257470A1
• Time in Patent Office: 274 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 1/72448 with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 1/72469 for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
• CPC Class Codes: H04W 4/16 Communication-related supplementary services, e.g. call-transfer or call-hold

Abstract

A mobile terminal including a wireless communication unit configured to provide wireless communication, a display, and a controller configured to, activate a mode for voice recognition in response to a touch input to a soft button displayed on the display or to a hard button on the mobile terminal, receive a first voice input associated with a phone call relating operation of the mobile terminal, display an indicator on the display indicating the voice input is being recognized by the mobile terminal, analyze the context of a voice command in the voice input, execute the call relating operation only if there is a single contact in a phonebook that matches the voice command in the first voice input, if there is no single contact that matches the voice command of the received voice input, display a plurality of candidates that is analyzed based on the voice command, receive a second input according to a plurality of candidates, and execute the call relating operation based on the second input.

Claims

1. A mobile terminal, comprising:

a memory configured to store multiple information related to menu relating operations of the mobile terminal; and

a controller configured to;

activate a mode for voice recognition in response to an input;

receive a first voice input associated with a menu relating operation of the mobile terminal;

detect at least one of the multiple information related to menu relating operations included in the memory based on the received first voice input;

if there is a single menu that matches a meaning of the received first voice input based on the at least one of the multiple detected information, execute the matched single menu;

if there is no single menu that matches the meaning of the received first voice input based on the at least one of the multiple detected information;

receive a second voice input associated with the at least one of the multiple detected information;

execute a menu operation corresponding to a meaning of the received second voice input; and

update the multiple information stored in the memory to match the meaning of the received first voice input to the menu selected by the meaning of the received second voice input.

2. The mobile terminal of claim 1, wherein the menu relating operation of the mobile terminal comprises at least one of edit, add, find, call, send message, reply message, settings, select, application execute, search, or move.

3. The mobile terminal of claim 2, wherein the first voice input associated with a phone call relating operation of the mobile terminal comprises a name stored in a phonebook stored at the mobile terminal.

4. The mobile terminal of claim 1, wherein if there is no single menu that matches the meaning of the received first voice input based on the at least one of the multiple detected information, the controller is further configured to:

perform a learning process based on the second voice input for learning the meaning of the received first voice input; and

execute a menu operation based on the learned meaning according to the second voice input.

5. The mobile terminal of claim 4, further comprising:

a display unit configured to display information,

wherein the controller is further configured to output on the display unit all menus of the mobile terminal based on the at least one of the multiple detected information when the first voice input is received.

6. The mobile terminal of claim 5, wherein the controller is further configured to receive a voice input command for selecting one of the displayed menus and recognize the input voice command and outputs information inquiring about whether the recognized input voice command is accurate.

7. The mobile terminal of claim 5, wherein the controller is further configured to output on the display unit said all menus based on the at least one of the multiple detected information and the first voice input that has the recognition rate that is higher than a predetermined threshold in an order of a higher recognition rate to a lower recognition rate.

8. The mobile terminal of claim 5, wherein the controller is further configured to discriminately display on the display unit a menu that has a highest recognition rate by controlling at least one of size, a position, a color, a brightness, and a highlight of the menu.

9. The mobile terminal of claim 4, wherein the controller is further configured to determine a number of times a specific menu on the terminal has been previously selected and to adjust the recognition rate for the specific menu based on the determined number of times the specific menu has been previously selected.

10. The mobile terminal of claim 1, wherein a recognition rate for the first voice input is updated based on the second voice input, wherein the recognition rate is increased if the second voice input affirms the first voice input.

11. The mobile terminal of claim 1 further comprising an input unit, wherein the input unit includes at least one of 1) a touch soft button that is touched to activate the voice recognition function, 2) a hard button that is pressed or manipulated to activate the voice recognition function, 3) an arbitrary position of a touch screen included in the input unit that is touched to activate the voice recognition function, 4) an impact sound that is input to activate the voice recognition function, 5) a local area radio signal or a remote area radio signal, or 6) a body information signal from a user.

12. The mobile terminal of claim 1, further comprising:

a first database configured to store voice or pronunciation information that is used by the controller to recognize voice input;

a second database configured to store a word, a keyword or sentence information that is used by the controller to recognize the voice input;

a third database configured to store information related to menus of the mobile terminal; and

a fourth database configured to store help information informing that the controller is attempting to determine the meaning of the voice input.

13. The mobile terminal of claim 1, wherein the controller is further configured to output audio or video information indicating that the voice recognition is being executed while the voice input is being received.

14. The mobile terminal of claim 1, wherein the multiple information include at least two domains from an email domain corresponding to emails sent and received on the mobile terminal, a scheduling task domain corresponding to scheduling events assigned on the mobile terminal, a contact domain corresponding to contacts on the mobile terminal, a phonebook domain corresponding to phone numbers stored on the mobile terminal, a map domain corresponding to map information provided by the mobile terminal, a photo domain corresponding to photos stored on the mobile terminal, a message domain corresponding to messages sent and received on the mobile terminal, a multimedia domain corresponding to multimedia functions performed on the mobile terminal, an external device domain corresponding to external devices the mobile terminal can connect to, a call history domain corresponding to calls sent and received on the mobile terminal, and a settings domain corresponding to setting functions performed on the mobile terminal.

25. The method of claim 1, wherein the input includes at least one of 1) a touch soft button that is touched to activate the voice recognition function, 2) a hard button that is pressed or manipulated to activate the voice recognition function, 3) an arbitrary position of a touch screen included in the input unit that is touched to activate the voice recognition function, 4) an impact sound that is input to activate the voice recognition function, 5) a local area radio signal or a remote area radio signal, and 6) a body information signal from a user.

15. A method of controlling a mobile terminal, the method comprising:

activating a mode for voice recognition on the mobile terminal in response to an input;

receiving a first voice input associated with a menu relating operation of the mobile terminal;

detecting at least one of the multiple information related to menu relating operations included in a memory of the mobile terminal based on the received first voice input;

if there is a single menu that matches a meaning of the received first voice input based on the at least one of the multiple detected information, executing the matched single menu;

if there is no single menu that matches the meaning of the received first voice input based on the at least one of the multiple detected information;

receiving a second voice input associated with the at least one of the multiple detected information;

executing a menu operation corresponding to a meaning of the received second voice input; and

updating the multiple information stored in the memory to match the meaning of the received first voice input to the menu selected by the meaning of the received second voice input.

16. The method of claim 15, wherein the menu relating operation of the mobile terminal comprises at least one of edit, add, find, call, send message, reply message, settings, select, application execute, search, or move.

17. The method of claim 16, wherein the first voice input associated with a phone call relating operation of the mobile terminal comprises a name stored in a phonebook stored at the mobile terminal.

18. The method of claim 15, wherein if there is no single menu that matches the meaning of the received first voice input based on the at least one of the multiple detected information, the method further comprises:

performing a learning process based on the second voice input for learning the meaning of the received first voice input; and

executing a menu operation based on the learned meaning according to the second voice input.

19. The method of claim 18, further comprising:

outputting on a display unit of the mobile terminal all menus of the mobile terminal based on the at least one of the multiple detected information when the first voice input is received.

20. The method of claim 19, further comprising:

receiving a voice input command for selecting one of the displayed menus;

recognizing the input voice command; and

outputting information inquiring about whether the recognized input voice command is accurate.

21. The method of claim 19, further comprising:

outputting on the display unit said all menus based on the at least one of the multiple detected information and the first voice input that has the recognition rate that is higher than a predetermined threshold in an order of a higher recognition rate to a lower recognition rate.

22. The method of claim 19, further comprising:

discriminately displaying on the display unit a menu that has a highest recognition rate by controlling at least one of size, a position, a color, a brightness, and a highlight of the menu.

23. The method of claim 18, further comprising:

determining a number of times a specific menu on the terminal has been previously selected and to adjust the recognition rate for the specific menu based on the determined number of times the specific menu or operation has been previously selected.

24. The method of claim 15, wherein a recognition rate for the first voice input is updated based on the second voice input, wherein the recognition rate is increased if the second voice input affirms the first voice input.

26. The method of claim 15, further comprising:

storing, in a first database, voice or pronunciation information that is used to recognize voice input;

storing, in a second database, a word, a keyword or sentence information that is used to recognize the voice input;

storing, in a third database, information related to menus of the mobile terminal; and

storing, in a fourth database, help information informing that the meaning of the voice input is being determined.

27. The method of claim 15, further comprising:

outputting audio or video information indicating that the voice recognition function is being executed while the voice input is being received.

28. The method of claim 15, wherein the multiple information include at least two domains from an email domain corresponding to emails sent and received on the mobile terminal, a scheduling task domain corresponding to scheduling events assigned on the mobile terminal, a contact domain corresponding to contacts on the mobile terminal, a phonebook domain corresponding to phone numbers stored on the mobile terminal, a map domain corresponding to map information provided by the mobile terminal, a photo domain corresponding to photos stored on the mobile terminal, a message domain corresponding to messages sent and received on the mobile terminal, a multimedia domain corresponding to multimedia functions performed on the mobile terminal, an external device domain corresponding to external devices the mobile terminal can connect to, a call history domain corresponding to calls sent and received on the mobile terminal, and a settings domain corresponding to setting functions performed on the mobile terminal.