MOBILE TERMINAL AND CONTROLLING METHOD THEREOF

• US 20170187852A1
• Filed: 07/24/2016
• Published: 06/29/2017
• Est. Priority Date: 12/29/2015
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: BAEK, Bumhyeon, JANG, Hyukjoon
• Granted Patent: US 10,104,208 B2
• Time in Patent Office: Days
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Disclosed are a mobile terminal for providing an intelligent agent (IA) service and controlling method thereof. The mobile terminal includes a display unit, a camera, a sensing unit comprising a microphone configured to sense a surrounding voice of the mobile terminal, an angle sensor configured sense an angle of the mobile terminal, and a location sensor configured to sense a location of the mobile terminal, a communication unit configured to transmit/receive data with a server configured to store a context information of a user, and a controller, if a trigger signal including a preset voice signal is sensed through the microphone, activating an intelligent agent (IA), the controller, if a preset input signal is sensed in a state that the IA is activated, activating the camera.

Claims

1. A mobile terminal, comprising:

a display;

a camera;

a sensing unit comprising a microphone;

a communication unit configured to communicate with a server; and

a controller configured to;

activate an intelligent agent (IA) in response to a trigger signal, the trigger signal comprising a preset voice signal received via the microphone; and

activate the camera in response to a preset input signal sensed in a state that the IA has been activated.

2. The mobile terminal of claim 1, wherein the controller is further configured to extract image context information corresponding to an image recognized within a view angle area of the camera.

3. The mobile terminal of claim 2, wherein the controller is further configured to:

extract voice context information corresponding to a voice received via the microphone, the voice received within a preset period of time after the trigger signal is sensed; and

provide IA information based on the extracted image context information and voice context information.

4. The mobile terminal of claim 2, wherein the controller is further configured to:

extract location context information from location information of the mobile terminal; and

provide IA information based on the extracted image context information and location context information.

5. The mobile terminal of claim 2, wherein:

the image context information corresponds to at least two preview images sensed within the view angle area of the camera; and

the at least two preview images comprise a first preview image and a second preview image.

6. The mobile terminal of claim 5, wherein the controller is further configured to:

sense the second preview image within a preset period time after sensing the first preview image; and

provide IA information based on the first preview image and the second preview image when a preset period time expires after sensing the second preview image.

7. The mobile terminal of claim 6, wherein the controller is further configured to:

extract location context information from location information of the mobile terminal; and

provide the IA information based on the location context information, the first preview image, and the second preview image.

8. The mobile terminal of claim 2, wherein:

the image context information corresponds to at least two capture images sensed within the view angle area of the camera; and

the at least two capture images comprise a first capture image and a second capture image.

9. The mobile terminal of claim 8, wherein the controller is further configured to:

extract location context information from location information of the mobile terminal; and

provide IA information based on the location context information, the first capture image, and the second capture image.

10. The mobile terminal of claim 1, wherein while the IA has been activated, the controller is further configured to:

extract image context information corresponding to at least one screen shot image for content displayed via the display; and

provide IA information based on the extracted image context information.

11. The mobile terminal of claim 10, wherein the controller is further configured to cause the communication unit to transmit the IA information to the server when an IA of an external device for which user authentication is activated.

12. The mobile terminal of claim 2, wherein the image context information comprises at least one capture image having an input signal sensed for a preview image sensed within the view angle area of the camera and at least one screen shot image for content displayed via the display.

13. The mobile terminal of claim 12, wherein context information of a user is stored in the server and the controller is further configured to:

cause the communication unit to receive additional context information of the user sensed by an external device; and

provide IA information based on the received additional context information and the image context information.

14. The mobile terminal of claim 3, wherein the IA information is provided as at least one of visual information or voice information.

15. The mobile terminal of claim 3, wherein the IA information comprises tag information indicating a keyword included in the image context information or the voice context information.

16. The mobile terminal of claim 1, wherein the controller is further configured to:

execute an IA related application when the IA is activated; and

cause the display to display a preview image sensed in a view angle area of the camera via the IA related application when the camera is activated.

17. A method for controlling a mobile terminal, the method comprising:

sensing a trigger signal, the trigger signal comprising a preset voice signal received via a microphone of the mobile terminal;

activating an intelligent agent (IA) in response to the sensed trigger signal;

sensing a preset input signal in a state that the IA has been activated; and

activating a camera of the mobile terminal in response to the preset input signal.

18. The method of claim 17, further comprising extracting image context information corresponding to an image recognized within a view angle area of the camera.

19. The method of claim 18, further comprising:

extracting voice context information corresponding to a voice received via the microphone within a preset period of time after the trigger signal is sensed; and

providing IA information based on the extracted image context information and voice context information.

20. The method of claim 18, further comprising:

extracting location context information from location information of the mobile terminal; and

providing IA information based on the extracted image context information and location context information.