MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

• US 20150081291A1
• Filed: 05/07/2014
• Published: 03/19/2015
• Est. Priority Date: 09/17/2013
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: JEON, Hyunjoo
• Granted Patent: US 9,390,715 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/171 by use of digital ink
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A mobile terminal including a wireless communication unit configured to perform wireless communication; a microphone configured to receive an input voice; a touch screen; and a controller configured to receive a written touch input on the touch screen corresponding to the input voice, recognize the input voice while a voice recognition mode is activated, and display extracted information extracted from the recognized input voice on the touch screen based on a comparison of the recognized input voice and the written input.

Claims

1. A mobile terminal comprising:

a wireless communication unit configured to perform wireless communication;

a microphone configured to receive an input voice;

a touch screen; and

a controller configured to;

receive a written touch input on the touch screen corresponding to the input voice,

recognize the input voice while a voice recognition mode is activated, and

display extracted information extracted from the recognized input voice on the touch screen based on a comparison of the recognized input voice and the written input.

2. The mobile terminal of claim 1, wherein the controller is further configured to display the extracted information at a position on the touch screen corresponding to the written touch input.

3. The mobile terminal of claim 2, wherein the controller is further configured to stop displaying the written touch input when the extracted information is displayed at the position on the touch screen corresponding to the written touch input.

4. The mobile terminal of claim 1, wherein the controller is further configured to:

convert the recognized input voice into text, and

display the text on the touch screen at a first position different than a position where the written touch input is displayed.

5. The mobile terminal of claim 4, wherein the controller is further configured to display the extracted information when receiving a selection of the displayed text.

6. The mobile terminal of claim 1, wherein portions of the written touch input corresponding to the extracted information are highlighted from other portions of the extracted information.

7. The mobile terminal of claim 6, wherein the controller is further configured to automatically correct misspelling of the portions of the written touch input and automatically add remaining letters excluded from the written touch input.

8. The mobile terminal of claim 1, wherein the controller is further configured to change information displayed on the touch screen or display a new page to secure a memo space for receiving the written touch input.

9. The mobile terminal of claim 1, wherein when the written touch input includes a written picture, the controller is further configured to display an emoticon that is similar in shape to the written picture image with the extracted information displayed on the touch screen.

10. The mobile terminal of claim 1, wherein the controller is further configured to:

switch between a first screen mode and a second screen mode based on a predetermined input, and

wherein the first screen mode displays only the written touch input and the second screen mode displays both the written touch input and text of the recognized input voice.

11. The mobile terminal of claim 10, wherein the controller is further configured to:

display a playback display option for playing back a recording of the input voice, and

scroll the written touch input in the first screen mode and scroll the written touch input and the text of the recognized input voice in the second screen mode according to a speed at which the recorded voice is played back.

12. The mobile terminal of claim 1, wherein the controller is further configured to enter an edit mode and output an indicator indicating an input position to a point at which a predetermined touch input is detected.

13. The mobile terminal of claim 1, wherein when the voice recognition mode is changed to an inactivated state or recording of the voice input is finished, the controller is further configured to:

display a list of characters that can be added to the written input, and

add selected characters from the list of characters to the written input at a position corresponding to a touch and drag from the selected characters to the written input.

14. The mobile terminal of claim 1, wherein when an automatic input function is selected when the voice recognition mode is activated, the controller is further configured to:

convert the recognized input voice into text, and

add in real time the text to a region to which the written input is displayed.

15. The mobile terminal of claim 1, wherein when a first control command is detected when a screen of the written input corresponding to a first application is displayed on the touch screen, the controller is further configured to:

switch the screen of the written input to an execution screen corresponding to a second application, and

display a graphic object indicating that recording of the voice input is in progress on one region of the touch screen.

16. The mobile terminal of claim 15, wherein when it is detected that a predetermined-type touch input is applied to the graphic object, the controller is further configured to:

display the screen of the written input corresponding to the first application back on the touch screen, and

insert characters corresponding to the voice input recorded while the execution screen corresponding to the second application is displayed into a region to which the written input is displayed.

17. The mobile terminal of claim 1, wherein when the controller determines a speed of the written input applied on the touch screen is higher or lower by a predetermined range than an input speed of the voice, the controller is further configured to:

execute an automatic input function in which the recognized input voice is converted into text, and

add the text in real time to a region to which the writing input is displayed.

18. The mobile terminal of claim 1, wherein the controller is further configured to:

receive a control command for executing an additional search associated with the voice input,

analyze the voice input,

search a memory associated with the mobile terminal or conduct a web search for a voice file that matches the voice input, and

display a list of found voice files on the touch screen.

19. A method of controlling a mobile terminal, the method comprising:

receiving, via a microphone, an input voice;

receive, via a touch screen, a written touch input corresponding to the input voice;

recognizing, via a controller, the input voice while a voice recognition mode is activated; and

displaying, via the controller, extracted information extracted from the recognized input voice on the touch screen based on a comparison of the recognized input voice and the written input.

20. The method of claim 19, further comprising:

displaying the extracted information at a position on the touch screen corresponding to the written touch input.