MOBILE TERMINAL AND METHOD OF CONTROLLING THE MOBILE TERMINAL

• US 20160011847A1
• Filed: 01/24/2014
• Published: 01/14/2016
• Est. Priority Date: 03/05/2013
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: CHOE, Dami, KIM, Hyungsup, JOUNG, Wonseok, PARK, Jungmin, SHIN, Yongkyoung
• Granted Patent: US 10,241,743 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04M 1/72433 for voice messaging, e.g. dictaphones for answering incoming calls H04M1/64
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector digitizers for touch screens G06F3/041

Abstract

The present invention relates to a mobile terminal capable of recording audio and a method of controlling the mobile terminal. Furthermore, the present invention relates a mobile terminal capable of outputting visual information and a method of controlling the mobile terminal. Provided is the mobile terminal including a recording function unit that records external audio, a display unit to which visual information is output, and a controller that performs processing by associating the audio recorded through the recording function unit and the visual information that is output to the display unit with each other, in which in response to selection of one region of the display unit, the controller matches the visual information included in the one region and the audio recorded through the recording function unit.

Claims

1. A mobile terminal comprising:

a recording function unit that records external audio;

a display unit to which visual information is output; and

a controller that performs processing by associating the audio recorded through the recording function unit and the visual information that is output to the display unit with each other,

wherein in response to selection of one region of the display unit, the controller matches the visual information included in the one region and the audio recorded through the recording function unit.

2. The mobile terminal of claim 1, wherein the controller matches the audio being recorded through the recording function unit with a point in time corresponding to a point in time when a user'"'"'s request is received.

3. The mobile terminal of claim 2, wherein in response to any one of a request for outputting of the visual information included in the one region and a request for outputting of the matched audio, the controller outputs the visual information included in the one region and the audio that is matched with the visual information included in the one region.

4. The mobile terminal of claim 2, wherein the selection of the one region is made through a predetermined-type input touch that is applied with respect to the one region, and

wherein the controller matches the audio that has been recorded a predetermined time earlier than a point in time when the one region is selected with the visual information included in the one region.

5. The mobile terminal of claim 1, wherein a graphic object alerting a user that the audio is being recorded through the recording function unit is output to the display unit, and

wherein based on the selection of the graphic object and the one region, the controller matches the visual information included in the one region and the audio being recorded through the recording function unit.

6. The mobile terminal of claim 5, wherein the selection of the graphic object and the one region is made through a first input touch with respect to the graphic object and a second input touch that is in succession to the first input touch and is ended at one point corresponding to the one region.

7. The mobile terminal of claim 6, wherein the graphic object is a progress bar indicating a status on progress in the recording of the audio by the recording function unit, and

wherein when the first input touch is applied with respect to the progress bar, the controller matches the audio that is recorded at a point in time corresponding to a point in time when the first input touch is applied with the visual information included in the one region.

8. The mobile terminal of claim 7, wherein the controller converts the audio that is recorded through the recording function unit into text, based on speech-to-text (SST) conversion and output at least one part of the resulting text to close to or the vicinity of the progress bar, and wherein the at least one part of the text is output to a position that corresponds to a point on the progress bar that corresponds to a point in time when the audio corresponding to the at least one part of the text is recorded.

9. The mobile terminal of claim 7, wherein the controller outputs an icon that alerts a user that the visual information included in the one region is matched with a region corresponding to a point on the progress bar, to which the first input touch is applied.

10. The mobile terminal of claim 9, wherein the icon is selected after ending the recording of the audio by the recording function unit, the controller outputs the visual information included in the one region, along with the audio that is recorded at a point in time corresponding to the point to which the first input touch is applied.

11. The mobile terminal of claim 1, wherein the controller displays at least one part of the visual information, included in the one region, of the visual information being output to the display unit, in a highlighting manner, or outputs an icon that alerts a user to the presence of the audio matched with a region adjacent to the one region.

12. The mobile terminal of claim 11, wherein in response to selection of the at least highlighted one part of visual information or the icon, the controller outputs the matched audio.

13. The mobile terminal of claim 1, where an amount of the audio matched with the visual information included in the one region is represented by a time from a point in time corresponding to a point in time when the one region is selected to a point in time when a command to end the matching is generated.

14. A method of controlling a mobile terminal, comprising:

recording external audio in a state where the visual information is output to the display unit;

receiving a user'"'"'s request for matching the visual information that is output to the display unit and the audio being recorded; and

matching at least one part of the visual information and the audio that is recorded at a point in time corresponding to a point in time when the user'"'"'s request is received, in response to the receiving of the user'"'"'s request.

15. The method of claim 14 further comprising displaying a graphic object alerting the user that the audio is matched with the vicinity of the visual information, which corresponds to the at least one part.

16. The method of claim 14, wherein the displaying of the graphic object includes outputting the audio matched with the visual information corresponding to the at least one part, in response to the selection of the graphic object.