Systems and methods for adjusting flight control of an unmanned aerial vehicle

• US 9,817,394 B1
• Filed: 09/13/2016
• Issued: 11/14/2017
• Est. Priority Date: 01/06/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Lema, Pablo, Ip, Shu Ching
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Lewandroski, Sara J
• Application Number:
  US15/264,216
• Time in Patent Office: 427 Days
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A first pattern associated with a performer may be recognized based upon visual information. A sensor carried by an unmanned aerial vehicle may be configured to generate output signals conveying the visual information. A first distance may be determined between the first pattern and the unmanned aerial vehicle. A second pattern associated with a performee may be recognized based upon the visual information. A second distance may be determined between the second pattern and the unmanned aerial vehicle. Flight control may be adjusted based upon the first distance and the second distance. A flight control subsystem may be configured to provide the flight control for the unmanned aerial vehicle.

Claims

1. A system for adjusting flight control of an unmanned aerial vehicle in order to capture a moment when a performer and a stationary object are in proximity of each other, the system comprising:

a flight control subsystem configured to provide the flight control for the unmanned aerial vehicle;

an image sensor carried by the unmanned aerial vehicle configured to generate output signals conveying visual information; and

a processor configured to;

determine a first distance between the performer and the unmanned aerial vehicle based on the visual information;

recognize the stationary object based on the visual information;

determine a second distance between the stationary object and the unmanned aerial vehicle;

adjust the flight control based on the first distance and the second distance such that, responsive to the performer approaching the stationary object, the unmanned aerial vehicle is in a position for the image sensor to capture the moment when the performer and the stationary object are in proximity of each other; and

responsive to the unmanned aerial vehicle being in the position, control the image sensor to capture a video segment that includes the moment when the performer and the stationary object are in proximity of each other such that the performer and the stationary object are captured in a single field of view of the image sensor.

2. The system of claim 1, wherein the flight control includes one or more of an altitude, a longitude, a latitude, a geographical location, a heading, and a speed of the unmanned aerial vehicle.

3. The system of claim 1, wherein the image sensor includes one or more image sensors configured to capture a video segment.

4. The system of claim 3, wherein the processor is further configured to:

mark the video segment at a point in time in which a third distance between the performer and the stationary object is a predefined distance.

5. The system of claim 3, wherein the processor is further configured to:

mark the video segment at a point in time in which the performer and the stationary object overlap.

6. The system of claim 1, wherein recognition of the stationary object is based on user action specifying the stationary object, and wherein the stationary object is a physical object.

7. The system of claim 1, wherein the flight control is adjusted such that one or more of a target altitude differential, a target cardinal direction, and a target distance are maintained between the unmanned aerial vehicle and the stationary object based upon the first distance and the second distance.

8. The system of claim 7, wherein one or more of the target altitude differential, the target cardinal direction, and the target distance are adjusted between the unmanned aerial vehicle and the stationary object based upon one or more gestures associated with the performer.

9. The system of claim 1, wherein the flight control is adjusted such that one or more of a target altitude differential, a target cardinal direction, and a target distance are maintained between the unmanned aerial vehicle and the performer based upon the first distance and the second distance.

10. The system of claim 9, wherein one or more of the target altitude differential, the target cardinal direction, and the target distance are adjusted between the unmanned aerial vehicle and the performer based upon one or more gestures associated with the performer.

11. A method for adjusting flight control of an unmanned aerial vehicle in order to capture a moment when a performer and a stationary object are in proximity of each other, the method comprising:

recognizing the performer based upon visual information, wherein an image sensor carried by the unmanned aerial vehicle configured to generate output signals conveying visual information;

determining a first distance between the performer and the unmanned aerial vehicle;

recognizing the stationary object based upon the visual information;

determining a second distance between the stationary object and the unmanned aerial vehicle;

adjusting the flight control based on the first distance and the second distance such that, responsive to the performer approaching the stationary object, the unmanned aerial vehicle is in a position for the image sensor to capture the moment when the performer and the stationary object are in proximity of each other, wherein a flight control subsystem is configured to provide the flight control for the unmanned aerial vehicle; and

responsive to the unmanned aerial vehicle being in the position, controlling the image sensor to capture a video segment that includes the moment when the performer and the stationary object are in proximity of each other such that the performer and the stationary object are captured in a single field of view of the image sensor.

12. The method of claim 11, wherein the flight control includes one or more of an altitude, a longitude, a latitude, a geographical location, a heading, and a speed of the unmanned aerial vehicle.

13. The method of claim 11, wherein the image sensor includes one or more image sensors configured to capture a video segment.

14. The method of claim 13, further comprising:

marking the video segment at a point in time in which a third distance between the performer and the stationary object is a predefined distance.

15. The method of claim 13, further comprising:

marking the video segment at a point in time in which the first pattern and the second pattern overlap.

16. The method of claim 11, wherein recognition of the stationary object is based on user action specifying the stationary object, and wherein the stationary object is a physical object.

17. The method of claim 11, wherein the flight control is adjusted such that one or more of a target altitude differential, a target cardinal direction, and a target distance are maintained between the unmanned aerial vehicle and the stationary object based upon the first distance and the second distance.

18. The method of claim 17, wherein one or more of the target altitude differential, the target cardinal direction, and the target distance are adjusted between the unmanned aerial vehicle and the stationary object based upon one or more gestures associated with the performer.

19. The method of claim 11, wherein the flight control is adjusted such that one or more of a target altitude differential, a target cardinal direction, and a target distance are maintained between the unmanned aerial vehicle and the performer based upon the first distance and the second distance.

20. The method of claim 19, wherein one or more of the target altitude differential, the target cardinal direction, and the target distance is adjusted between the unmanned aerial vehicle and the performer based upon one or more gestures associated with the performer.