MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME

• US 20170134553A1
• Filed: 11/07/2016
• Published: 05/11/2017
• Est. Priority Date: 11/11/2015
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: JEON, Bongjeong, CHOI, Jian, KIM, Namki, OH, Sesook, KIM, Insuk
• Granted Patent: US 10,389,863 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/08 User group management
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 76/10 Connection setup

Abstract

A mobile terminal includes a camera unit configured to sense an image, a touch screen configured to output image information, a communication unit configured to perform wireless communication with at least one Internet of things (IOT) device, and a controller configured to, when the mobile terminal enters a specific indoor area, control IOT devices included in the sensed image or a previously stored image of the specific indoor area by using the image, wherein the controller stores a plurality of pieces of set information including a state in which at least some of the IOT devices are connected and operational states of the connected IOT devices, and when any one of the plurality of pieces of set information is selected, the controller controls at least some of the IOT devices to be connected to each other and operate according to the selected set information.

Claims

1. A mobile terminal comprising:

a camera;

a memory configured to store information;

a touchscreen configured to display information;

a communication unit configured to perform wireless communication with at least one device; and

a controller configured to;

cause the touchscreen to display an image of a preview image obtained by the camera or a previously stored image, wherein the displayed image includes a plurality of devices and corresponds to a specific indoor area where the mobile terminal is located;

cause the memory to store set information comprising connection information of two or more devices of the plurality of devices and operational state information of the two or more devices; and

transmit information via the communication unit to cause the two or more devices to be connected to each other according to the stored connection information and to operate according to the stored operational state information in response to a selection of the stored set information.

2. The mobile terminal of claim 1, wherein the controller is further configured to:

cause the set information to be stored to be associated with a specific situational setting; and

cause the memory to differently store multiple set information associated with a same specific situational setting based on corresponding surrounding environment information detected by the mobile terminal when each of the multiple set information is stored.

3. The mobile terminal of claim 2, wherein the surrounding environment information comprises a current time and weather information.

4. The mobile terminal of claim 1, wherein the controller is further configured to cause the touchscreen to display information for changing an operational state of at least one of the two or more devices based on surrounding environment information detected by the mobile terminal.

5. The mobile terminal of claim 4, wherein the displayed information for changing the operational state of the at least one of the two or more devices is further based on the detected surrounding environment information and a current operational state of the two or more devices.

6. The mobile terminal of claim 5, wherein the controller is further configured to update the selected set information when an operational state of at least one of the two or more devices is changed according to the displayed information.

7. The mobile terminal of claim 1, wherein the controller is further configured to:

cause the touchscreen to display graphic objects corresponding to each of the two or more devices; and

cause a selected one of the two or more devices to be disconnected when a graphic object corresponding to the selected one device is selected.

8. The mobile terminal of claim 1, wherein the controller is further configured to select the two or more devices from the plurality of devices for the stored set information according to a drag trace received via the touchscreen, wherein the drag trace forms a loop intersecting each of the two or more devices in the displayed image.

9. The mobile terminal of claim 1, wherein the controller is further configured to cause the set information to be stored when a preset period expires or when a connection state of one of the two or more devices is changed.

10. The mobile terminal of claim 1, wherein the controller is further configured to cause the touchscreen to display a control interface for each of the two or more devices, wherein the control interfaces are different based on a size of the corresponding device in the displayed image.

11. The mobile terminal of claim 10, wherein the size of the corresponding device in the displayed image is changed according to zooming-in or zooming-out of the image or according to movement of the mobile terminal within the specific indoor area.

12. The mobile terminal of claim 10, wherein the controller is further configured to change a displayed control interface to have a higher granularity of control of a corresponding device as a size of the corresponding device in the displayed image is increased.

13. The mobile terminal of claim 1, wherein the controller is further configured to:

cause the touchscreen to display a graphic object related to an operational state of a selected device of the two or more devices in response to a selection of the selected device in the displayed image; and

control the selected device based on a touch input to the displayed graphic object.

14. The mobile terminal of claim 13, wherein the controller is further configured to cause the touchscreen to display the image sensed by the camera which is reflected effects of the operating state change of the one or more IOT device.

15. The mobile terminal of claim 1, further comprising a sensor configured to detect a state in which the mobile terminal is gripped,

wherein the preview image obtained by the camera or the previously stored image is displayed based on the detected state in which the mobile terminal is gripped.

16. The mobile terminal of claim 15, wherein the controller is further configured to cause the touchscreen to display another previously stored image corresponding to another indoor area when a specific object included in the previously stored image is displayed on the touchscreen and a selection input is received to the displayed specific object,

wherein the displayed another previously stored image includes another plurality of devices.

17. The mobile terminal of claim 16, wherein the controller is further configured to connect one of the another plurality of devices with one of the two or more devices.

18. The mobile terminal of claim 16, wherein the controller is further configured to cause the touchscreen to display an indicator corresponding to a region of the specific object on the displayed image when the another previously stored image is stored.

19. The mobile terminal of claim 1, wherein the controller is further configured to:

establish a connection with an access point via the communication unit, wherein the access point includes information related to the plurality of devices in the specific indoor area; and

determine that the mobile terminal has entered the specific indoor area when the mobile terminal is connected to the access point.

20. A method for controlling a mobile terminal, the method comprising:

displaying a plurality set information items comprising connection state information and operational state information of a plurality of devices located in a specific indoor area, wherein each of the plurality of set information items are displayed differently based on surrounding environment information detected when each corresponding set information item was stored; and

controlling one or more of the plurality of devices to be connected to each other and operate according to a selected set information item of the plurality of set information items in response to selection of the selected set information item,

wherein each of the plurality of set information items corresponds to a different preset specific situational setting.