Mobile device and method for controlling the same

• US 9,635,248 B2
• Filed: 04/09/2015
• Issued: 04/25/2017
• Est. Priority Date: 12/29/2014
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Bae, Hyejin, Lee, Kyungha, Yang, Jeonghwa
• Primary Examiner(s): Tran, Dalena
• Application Number:
  US14/682,586
• Publication Number:
  US 20160191793A1
• Time in Patent Office: 747 Days
• Field of Search: 701 2, 345173, 4555561
• CPC Class Codes: A63H 30/04 using wireless transmission
• CPC Class Codes: G03B 15/006 Apparatus mounted on flying objects
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

A mobile terminal and a method of controlling therefor are disclosed. The mobile terminal according to one embodiment of the present invention includes a touch interface module configured to receive a touch input, a communication module configured to establish communication with at least one or more drones and a controller configured to control the communication module and the touch interface module. The controller transmits location information of the mobile terminal to the drone with which the communication is established, designates a position of the drone based on the transmitted location information, receives a preview image obtained by a camera of the drone, displays the received preview image on a predetermined first area and if the touch input is recognized, captures the preview image displayed on the first area.

Claims

1. A method of communicating with a drone via a mobile terminal, the method comprising:

calculating an angle of view covered by a camera of the mobile terminal;

transmitting view information related to the calculated angle of view and location information of the mobile terminal to the drone via a communication module to cause the drone to capture another view via a camera of the drone wherein the another view is not included in the angle of view covered by the camera of the mobile terminal;

designating a position of the drone based on the transmitted location information;

receiving a first preview image from the drone, the first preview image corresponding to the another view and obtained via the camera of the drone from the designated position;

displaying the first preview image on a first area of a touchscreen of the mobile terminal; and

capturing the first preview image in response to a first touch input received via the touchscreen.

2. The method of claim 1, further comprising displaying a second preview image obtained via a camera of the mobile terminal on a second area of the touchscreen.

3. The method of claim 2, further comprising displaying an option to select:

a first mode for controlling the first preview image and the second preview image with a single indicator;

or

a second mode for controlling the first preview image and the second preview image with two different corresponding indicators.

4. The method of claim 3, further comprising deactivating the first mode when a camera capturing mode of the drone is different from a camera capturing mode of the mobile terminal.

5. The method of claim 4, wherein the camera capturing mode of the drone and the camera capturing mode of the mobile terminal each comprise a video capturing mode or a picture capturing mode.

6. The method of claim 3, wherein:

the first preview image and the second preview image are simultaneously captured in response to an input to the single indicator in the first mode; and

the first preview image and the second preview image are individually captured in response to corresponding inputs to the two different corresponding indicators in the second mode.

7. The method of claim 2, further comprising transmitting object capture information to the drone for capturing an image of an object from a capturing position in response to receiving a selection of the object from the displayed second preview image.

8. The method of claim 7, wherein the object capture information comprises at least a distance between the capturing position and the object, a direction of the capturing position relative to the object, or a camera capturing scheme.

9. The method of claim 8, wherein the camera capturing scheme comprises at least a first scheme type for capturing an image of the object from a front view, a second scheme type for capturing the image while rotating 360 degrees around the object, or a third scheme type for capturing the image from above the object at an altitude higher than the object.

10. The method of claim 2, further comprising changing at least corresponding positions of the first area and the second area or corresponding sizes of the first area and the second area.

11. The method of claim 10, further comprising displaying an indicator of at least one specific area to which a position of the first area or the second area can be moved in response to a long touch input received via the touchscreen to the corresponding first area or second area, the long touch input received for longer than a threshold length of time.

12. The method of claim 2, further comprising displaying a third preview image received from an external mobile terminal on a third area of the touchscreen when a call signal is received from the external mobile terminal.

13. The method of claim 2, further comprising:

detecting a specific part of a user included in the second preview image; and

transmitting a command to the drone for controlling a motion of the drone based on detecting the specific part.

14. The method of claim 13, wherein the specific part comprises at least a mouth, eyes, or a hand of the user.

15. The method of claim 1, further comprising:

displaying a control for selecting a distance between the drone and the mobile terminal and direction information of the drone relative to the mobile terminal; and

transmitting the selected distance and direction information to the drone.

16. The method of claim 1, wherein the view information comprises a numerical value of the calculated angle of view.

17. The method of claim 1, further comprising displaying a position of the mobile terminal and a position of the drone on a map.

18. The method of claim 17, wherein the map comprises information indicating:

a path of movement of the mobile terminal;

a path of movement of the drone;

a direction a camera of the mobile terminal is facing; and

a direction a camera of the drone is facing.

19. The method of claim 1, further comprising displaying an option for selectively storing at least first audio data obtained via a microphone of the drone or second audio data obtained via a microphone of the mobile terminal.

20. The method of claim 1, wherein the location information is transmitted to the drone after the first preview image is captured or after a threshold length of time.

21. The method of claim 20, wherein the location information comprises information related to a direction the camera of the mobile terminal is facing.

22. The method of claim 1, further comprising:

dividing the first preview image or the second preview image into a plurality of sections; and

transmitting target location information related to a target location to the drone in response to a touch input received to a selected one of the plurality of sections via the touchscreen, wherein the selected one of the plurality of sections corresponds to the target location; and

receiving an updated preview image of the target location from the drone obtained via the camera of the drone.

23. The method of claim 22, further comprising displaying the updated preview image on the selected one of the plurality of sections.

24. The method of claim 23, wherein the updated preview image corresponds to an image of the target location captured at a closer distance compared to the first preview image or the second preview image.

25. A mobile terminal, comprising:

a touchscreen configured to display information;

a communication module configured to transmit and receive information; and

a controller configured to;

calculate an angle of view covered by a camera of the mobile terminal;

cause the communication module to transmit view information related to the calculated angle of view and location information of the mobile terminal to a drone to cause the drone to capture another view via a camera of the drone wherein the another view is not included in the angle of view covered by the camera of the mobile terminal;

designate a position of the drone based on the transmitted location information;

receive a first preview image from the drone via the communication module, the first preview image corresponding to the another view and obtained via the camera of the drone from the designated position;

cause the touchscreen to display the first preview image on a first area of the touchscreen; and

capture the first preview image in response to a first touch input received via the touchscreen.

26. The mobile terminal of claim 25, wherein the controller is further configured to cause the touchscreen to display a second preview image obtained via a camera of the mobile terminal on a second area of the touchscreen.

27. The mobile terminal of claim 26, wherein the controller is further configured to cause the touchscreen to display an option to select:

a first mode for controlling the first preview image and the second preview image with a single indicator;

or

a second mode for controlling the first preview image and the second preview image with two different corresponding indicators.

28. The mobile terminal of claim 27, wherein the controller is further configured to deactivate the option for the first mode if a camera capturing mode of the drone is different from a camera capturing mode of the mobile terminal.

29. The mobile terminal of claim 28, wherein the camera capturing mode of the drone and the camera capturing mode of the mobile terminal each comprise a video capturing mode or a picture capturing mode.

30. The mobile terminal of claim 27, wherein:

the first preview image and the second preview image are simultaneously captured in response to an input to the single indicator in the first mode; and

the first preview image and the second preview image are individually captured in response to corresponding inputs to the two different corresponding indicators in the second mode.

31. The mobile terminal of claim 26, wherein the controller is further configured to cause the communication module to transmit object capture information to the drone for capturing an image of an object from a capturing position with respect to a location of the object in response to receiving a selection of the object from the displayed second preview image.

32. The mobile terminal of claim 31, the object capture information comprises at least a distance between the capturing position and the object, a direction of the capturing position relative to the object, or a camera capturing scheme.

33. The mobile terminal of claim 32, wherein the camera capturing scheme comprises at least a first scheme type for capturing an image of the object from the front, a second scheme type for capturing the image while rotating 360 degrees around the object, or a third scheme type for capturing the image from above the object at an altitude higher than the object.

34. The mobile terminal of claim 26, wherein the controller is further configured to change at least corresponding positions of the first area and the second area or corresponding sizes of the first area and the second area.

35. The mobile terminal of claim 34, the controller is further configured to cause the touchscreen to display an indicator of at least one specific area to which a position of the first area or the second area can be moved in response to a long touch input to the corresponding first area or second area, the long touch input received for longer than a threshold length of time.

36. The mobile terminal of claim 26, the controller is further configured to cause the touchscreen to display a third preview image received from an external mobile terminal on a third area of the touchscreen when a call signal is received from the external mobile terminal.

37. The mobile terminal of claim 26, wherein the controller is further configured to:

detect a specific part of a user included in the second preview image; and

cause the communication module to transmit a command to the drone for controlling a motion of the drone based on detecting the specific part.

38. The mobile terminal of claim 37, wherein the specific part comprises at least a mouth, eyes, or a hand of the user.

39. The mobile terminal of claim 25, wherein the controller is further configured to:

cause the touchscreen to display a control for selecting a distance between the drone and the mobile terminal and direction information of the drone relative to the mobile terminal; and

cause the communication module to transmit the selected distance and the direction information to the drone.

40. The mobile terminal of claim 25, wherein the view information comprises a numerical value of the calculated angle of view.

41. The mobile terminal of claim 25, wherein the controller is further configured to cause the touchscreen to display a position of the mobile terminal and a position of the drone on a map.

42. The mobile terminal of claim 41, wherein the map comprises information indicating:

a path of movement of the mobile terminal;

a path of movement of the drone;

a direction a camera of the mobile terminal is facing; and

a direction a camera of the drone is facing.

43. The mobile terminal of claim 25, wherein the controller is further configured to cause the touchscreen to display an option for selectively storing at least first audio data obtained via a microphone of the drone or second audio data obtained via a microphone of the mobile terminal.

44. The mobile terminal of claim 25, the controller is further configured to cause the communication module to transmit the location information to the drone after the first preview image is captured or after a threshold length of time.

45. The mobile terminal of claim 44, wherein the location information comprises information related to a direction the camera of the mobile terminal is facing.

46. The mobile terminal of claim 25, wherein the controller is further configured to:

cause the touchscreen to display the first preview image or the second preview image divided into a plurality of sections; and

causing the communication module to transmit target location information related to a target location to the drone in response to a touch input received to a selected one of the plurality of sections via the touchscreen, wherein the selected one of the plurality of sections corresponds to the target location; and

receive, via the communication module, an updated preview image of the target location from the drone obtained via the camera of the drone.

47. The mobile terminal of claim 46, wherein the controller is further configured to cause the touchscreen to display the updated preview image on the selected one of the plurality of sections.

48. The mobile terminal of claim 47, wherein the updated preview image corresponds to an image of the target location captured at a closer distance compared to the first preview image or the second preview image.