Control device and control method for flying bot

• US 10,191,487 B2
• Filed: 11/14/2014
• Issued: 01/29/2019
• Est. Priority Date: 11/14/2014
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Hong, Kichu, Cho, Guimok, Kim, Kihyung
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US15/526,314
• Publication Number:
  US 20170329325A1
• Time in Patent Office: 1,537 Days
• CPC Class Codes: B64C 13/20 using radiated signals
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/30 User interface
• CPC Class Codes: G08C 2201/50 Receiving or transmitting feedback, e.g. replies, status updates, acknowledgements, from the controlled devices
• CPC Class Codes: G08C 2201/91 Remote control based on location and proximity
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

The present invention relates to a device capable of controlling a flying bot and a control method therefor, the device comprising: a communication unit for performing wireless communication with the flying bot; a display unit for displaying image information related to the control of the flying bot; a user input unit for controlling a flying state of the flying bot and a function which can be performed by the flying bot; a detection unit for detecting at least one of a rotating direction, a moving direction and a tilt state of the flying bot; and a control unit for receiving, from the flying bot, information about a location and/or a surrounding situation of the flying bot through the communication unit and changing a flight control manner of the flying bot on the basis of the received result, wherein, according to the changed flight control manner of the flying bot, the control unit determines, on the basis of a location of a user, the moving direction and the rotating direction of the flying bot according to an input into the user input unit.

Claims

1. A flying bot control device comprising:

a communication unit performing wireless communication with a flying bot;

a display unit displaying image information related to control of the flying bot;

a user input unit controlling a flying state of the flying bot and a function which can be performed by the flying bot;

a detection unit detecting at least one of a rotation direction, a movement direction, or a tilt state of the flying bot; and

a controller receiving information regarding at least one of a position or a surrounding situation from the flying bot through the communication unit and switching a flight control mode of the flying bot on the basis of the received information,

wherein, according to the changed flight control mode of the flying bot, the controller determines a movement direction and a rotation direction of the flying bot in accordance with an input to the user input unit on the basis of a location of a user,

wherein when a result of sensing an obstacle adjacent to the flying bot is received from the flying bot, the controller displays information regarding the sensed obstacle on the display unit, and

wherein the controller displays a position and a direction of the obstacle using separate graphic objects, and the graphic objects are displayed in different colors or forms according to a distance between the flying bot and the sensed obstacle.

2. The flying bot control device of claim 1, wherein the controller measures a distance between the flying bot and the user from a position of the flying bot and determines the flight control mode of the flying bot according to the measured distance.

3. The flying bot control device of claim 2, wherein the controller switches the flight control mode of the flying bot to any one of a third person control mode in which the flying bot is controlled according to a result of observing the flying bot by the user with his naked eyes on the basis of the measured distance, a first person control mode in which the flying bot is controlled on the basis of a point of view of a camera installed in the flying bot, and a position designation mode in which the flying bot is controlled on the basis of a position of the flying bot on a map.

4. The flying bot control device of claim 3, wherein:

when the measured distance is smaller than a preset first distance, the controller determines the flight control mode of the flying bot to the third person control mode,

when the measured distance is equal to or greater than a preset second distance, the controller determines the flight control mode of the flying bot to the position designation mode, and

in other cases, the controller determines the flight control mode of the flying bot to the first person control mode.

5. The flying bot control device of claim 4, wherein when the measured distance is smaller than the preset first distance, the controller further measures strength of a signal transmitted from the flying bot and determines whether to change the flight control mode of the flying bot to the first person control mode according to whether the measured strength of the signal is equal to or higher than a lower limit value of a predetermined range set on the basis of a reference value determined according to the measured distance.

6. The flying bot control device of claim 3, wherein when a currently set flight control mode is the first person control mode, the controller displays an image received from the camera provided in the flying bot on the display unit, determines a direction in which the camera is oriented as a forward direction of the flying bot, determines backward, leftward, and rightward directions of the flying bot on the basis of the determined forward direction, and synchronizes the determined directions with keys for controlling the movement direction of the flying bot in the user input unit.

7. The flying bot control device of claim 3, wherein when a currently set flight control mode is the third person control mode, the controller determines a direction in which the user is positioned as a backward direction of the flying bot on the basis of a position of the flying bot, determines forward, leftward, and rightward directions of the flying bot on the basis of the determined backward direction, and synchronizes the determined directions with keys for controlling the movement direction of the flying bot in the user input unit.

8. The flying bot control device of claim 3, wherein when a currently set flight control mode is the position designation mode, the controller displays map information including information related to a current position of the flying bot on the display unit, and controls the flying bot to move on the basis of a user'"'"'s touch input applied to the display unit.

9. The flying bot control device of claim 8, wherein

the controller further displays information regarding a current position of the user on the map information, and

when a touch input corresponding to a location of the user is applied, the controller controls the flying bot to be returned to the user.

10. The flying bot control device of claim 9, wherein the controller controls the flying bot to be returned to the user along a path along which the flying bot has moved to the current position.

11. The flying bot control device of claim 1, wherein the controller displays map information including a current position of the flying bot on the display unit, and displays a graphic object indicating information related to the obstacle on the map information.

12. The flying bot control device of claim 3, wherein when the flight control mode is determined as the third person control mode or the position designation mode, the controller switches the flight control mode of the flying bot to the first person control mode according to a result of sensing a surrounding situation of the flying bot, and the flight control mode of the flying bot is switched to the first person control mode when obstacles equal to or greater than a preset number are sensed in the vicinity of the flying bot or when obstacles equal to or greater than the preset number are recognized according to a result of analyzing an image received from the camera provided in the flying bot.

13. The flying bot control device of claim 2, wherein the controller switches the flight control mode of the flying bot on the basis of a result of further sensing a current altitude of the flying bot and a meteorological situation sensed by the flying bot.

14. The flying bot control device of claim 3, further comprising:

a gaze recognizing unit recognizing user'"'"'s eyes,

wherein when a time duration in which the user has gazed the flying bot control device is equal to or greater than a preset time duration according to a result of recognizing the user'"'"'s eyes through the gaze recognizing unit, the controller switches the flight control mode of the flying bot to the first person control mode.

15. The flying bot control device of claim 1, wherein the controller drives a camera provided in the flying bot to capture an image received from the camera according to a user selection, and controls a flying state of the flying bot differently according to an image capture mode of capturing the image.

16. The flying bot control device of claim 15, wherein

the image capture mode includes a panorama image capture mode for capturing a panorama image, and

when the image capture mode is the panorama image capture mode, the controller controls the flying bot to move according to a movement direction in accordance with control of the user in a state in which a direction in which the camera provided in the flying bot is fixated to a specific direction in accordance with control of the user.

17. The flying bot control device of claim 16, wherein the controller automatically switches the image capture mode on the basis of at least one of a position, an altitude, or a surrounding situation of the flying bot according to a result of analyzing an image sensed by the camera provided in the flying bot.

18. A method for controlling a flying bot, the method performed by a control device and comprising:

receiving information related to at least one of a position or a surrounding situation of the flying bot from the flying bot;

determining a flight control mode of the flying bot on the basis of the received information;

determining a movement direction and a rotation direction of the flying bot on the basis of a location of a user, in accordance with a user input according to the determined flight control mode of the flying bot;

displaying information related to control of the flying bot on a display according to the determined flight control mode of the flying bot;

moving the flying bot according to a user input and performing a function executable in the flying bot;

when a result of sensing an obstacle adjacent to the flying bot is received from the flying bot, displaying information regarding the sensed obstacle on the display; and

displaying a position and a direction of the obstacle using separate graphic objects, wherein the graphic objects are displayed in different colors or forms according to a distance between the flying bot and the sensed obstacle.