System and method for providing easy-to-use release and auto-positioning for drone applications

• US 10,168,704 B2
• Filed: 07/25/2017
• Issued: 01/01/2019
• Est. Priority Date: 06/05/2017
• Status: Active Grant

• Current Assignee: Hangzhou Zero Zero Technology Co.Ltd (Shenzhen Zero Zero Infinite Technology Co. Ltd.)
• Original Assignee: Hangzhou Zero Zero Technology Co.Ltd (Shenzhen Zero Zero Infinite Technology Co. Ltd.)
• Inventors: Zhang, Tong, Wang, Mengqiu
• Primary Examiner(s): Brainard, Timothy A
• Application Number:
  US15/658,572
• Publication Number:
  US 20180348764A1
• Time in Patent Office: 525 Days
• Field of Search: 342 28
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 20/65 Composite materials
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 50/19 using electrically powered motors
• CPC Class Codes: B64U 70/92 Portable platforms
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0669 specially adapted for vertical take-off
• CPC Class Codes: G05D 1/223 Command input arrangements on the remote controller, e.g. joysticks or touch screens
• CPC Class Codes: G05D 1/228 Command input arrangements located on-board unmanned vehicles
• CPC Class Codes: G05D 1/652 Take-off delivering or retrieving payloads G05D1/667
• CPC Class Codes: G05D 1/689 Pointing payloads towards fixed or moving targets positioning towed, pushed or suspended implements G05D1/672
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: H04N 23/45 : for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/62 : Control of parameters via user interfaces
• CPC Class Codes: H04N 23/632 : for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

System and method for controlling an aerial system to perform a selected operation using an easy-to-use release and auto-positioning process.

Claims

1. An aerial system, comprising:

a body;

a lift mechanism coupled to the body;

an optical system controllably mounted to the body by an actuation system;

a user interface mounted on a surface of the body, the user interface including a display screen; and

a processing system coupled to the user interface, the lift mechanism, the optical system, and the actuation system, the processing system configured to;

display a plurality of operations selectable by a user on the user interface and receive a user selection of an operation to perform via the user interface, each of the plurality of operations associated with a flight trajectory and a predefined action performed by the optical system;

detect a release event in which the aerial system is released from being physically held by the user and responsively operate the lift mechanism to hover the aerial system;

detect a flight event indicating that the aerial system has been supported substantially horizontally for greater than a threshold period of time;

automatically operate the lift mechanism in a flight mode upon detecting the flight event, including;

operating the optical system to detect a target;

establishing a corresponding flight trajectory relative to the target defined by the selected operation; and

controlling the lift mechanism to move the aerial system along the established flight trajectory;

operate the optical system to perform a corresponding predefined action defined by the selected operation;

operate the aerial system in a retrieving mode when the predefined action has completed;

detect a standby event; and

operate the aerial system in a standby mode in response to detecting the standby event.

2. An aerial system, as set forth in claim 1, wherein the user interface is implemented using one or more of the following:

a menu displayed on a display, a touchscreen device, and/or one or more buttons.

3. An aerial system, as set forth in claim 2, wherein the user interface is mounted on a top surface the body.

4. An aerial system, as set forth in claim 2, including a second user interface provided at a remote device, the remote device configured to send data related to user input on the second user interface to the processing system.

5. An aerial system, as set forth in claim 1, wherein the processing system is further configured to detect the standby event including detecting a grab indication that the aerial system has been captured by a retention mechanism.

6. An aerial system, as set forth in claim 1, wherein the processing system is configured to operate the optical system to perform a predefined action including automatically begin obtaining pictures and/or video of the target.

7. An aerial system, as set of forth in claim 6, wherein the processing system is configured to track the target during flight mode and automatically adjust a camera angle of the optical system to maintain the target within the pictures and/or video.

8. An aerial system, as set forth in claim 1, wherein the processing system is configured to detect an initial release speed and to responsively establish a distance travel command as a function of the detected initial release speed.

9. An aerial system, as set forth in claim 1, wherein the processing system is configured to detect an initial trajectory and to responsively establish a direction to travel.

10. An aerial system, as set forth in claim 1, wherein the processing system is configured to detect an initial release speed and responsively establish a distance travel command as a function of the detected initial release speed and to detect an initial trajectory and to responsively establish a direction to travel.

11. An aerial system, as set forth in claim 10, wherein the processing system is configured to establish a desired position of the aerial system relative to the target as a function of the distance travel command and initial trajectory.

12. An aerial system, as set forth in claim 11, wherein the processing system is further configured to control the aerial system to travel to the desired position and orientate the aerial system towards the target.

13. An aerial system, as set forth in claim 1, wherein the predefined action includes at least one of:

(1) automatically modify the camera angle and flight trajectory with the target in the picture without any interaction between the user and any device;

(2) automatically take photos or record videos without any interaction between the user and any device; and

(3) automatically select good candidates of photos and/or video clips from raw photo/video material for further user editing or automatic editing procedures.

14. An aerial system, as set forth in claim 13, wherein the processing system in performing the step of automatically taking photos or recording videos may perform one of more of:

(1) taking a snapshot, (2) taking a series of snapshots, (3) taking a video with or without snapshot(s), and (4) taking one or more videos with or without snapshot(s).

15. A method for controlling an aerial system including a body, a lift mechanism coupled to the body, an optical system controllably mounted to the body by an actuation system, a user interface mounted on a surface of the body, the user interface including a display screen, and a processing system, the method comprising the processing system performing the steps of:

displaying a plurality of operations selectable by a user on the user interface, each of the plurality of operations associated with a flight trajectory and a predefined action performed by the optical system;

receiving a user selection of an operation to perform via the user interface;

detecting a release event in which the aerial system is released from being physically held by the user and responsively operating the lift mechanism to hover the aerial system;

detecting a flight event indicating that the aerial system has been supported substantially horizontally for greater than a threshold period of time;

automatically operate the lift mechanism in a flight mode upon detecting the flight event, including;

operating the optical system to detect a target;

establishing a corresponding flight trajectory relative to the target defined by the selected operation; and

controlling the lift mechanism to move the aerial system along the established flight trajectory;

operating the optical system to perform a corresponding predefined action defined by the selected operation;

operating the aerial system in a retrieving mode when the predefined action has completed;

detecting a standby event; and

operating the aerial system in a standby mode in response to detecting the standby event.

16. A method, as set forth in claim 15, wherein the user interface is implemented using one or more of the following:

a menu displayed on a display, a touchscreen device, and/or one or more buttons.

17. A method, as set forth in claim 16, wherein the user interface is mounted on a top surface the body.

18. A method, as set forth in claim 16, wherein a second user interface is provided at a remote device, the method including the step of operating the aerial system based on data related to user input on the second user interface received from the remote device.

19. A method, as set forth in claim 15, wherein the step of detecting the standby event includes detecting a grab indication that the aerial system has been captured by a retention mechanism.

20. A method, as set forth in claim 15, including the step of operating the optical system to perform a predefined action including automatically begin obtaining pictures and/or video of the target.

21. A method, as set of forth in claim 20, including the step of tracking the target during flight mode and automatically adjusting a camera angle of the optical system to maintain the target within the pictures and/or video.

22. A method, as set forth in claim 15, including the steps of detecting an initial release speed and to responsively establishing a distance travel command as a function of the detected initial release speed.

23. A method, as set forth in claim 15, including the steps of detecting an initial trajectory and to responsively establishing a direction to travel.

24. A method, as set forth in claim 15, including the steps of detecting an initial release speed and responsively establishing a distance travel command as a function of the detected initial release speed and detecting an initial trajectory and responsively establishing a direction to travel.

25. A method, as set forth in claim 24, including the step of establishing a desired position of the aerial system relative to the target as a function of the distance travel command and initial trajectory.

26. A method, as set forth in claim 25, including the steps of controlling the aerial system to travel to the desired position and orientating the aerial system towards the target.

27. A method, as set forth in claim 15, wherein the predefined action includes at least one of:

(1) automatically modify the camera angle and flight trajectory with the target in the picture without any interaction between the user and any device;

(2) automatically take photos or record videos without any interaction between the user and any device; and

(3) automatically select good candidates of photos and/or video clips from raw photo/video material for further user editing or automatic editing procedures.

28. A method, as set forth in claim 27, wherein the step of automatically taking photos or recording videos includes one of more of:

(1) taking a snapshot, (2) taking a series of snapshots, (3) taking a video with or without snapshot(s), and (4) taking one or more videos with or without snapshot(s).