Action camera system for unmanned aerial vehicle

• US 10,187,580 B1
• Filed: 09/18/2017
• Issued: 01/22/2019
• Est. Priority Date: 11/05/2013
• Status: Active Grant

• Current Assignee: Draganfly Innovations Inc. (Frontier Energy Corp.)
• Original Assignee: Draganfly Innovations Inc. (Frontier Energy Corp.)
• Inventors: Beard, Paul, McDermott, Craig, Chell, Cameron, Racicot, Erika, Clarke, Jamie, McDonald, Bruce, Readwin, Paul
• Primary Examiner(s): Holder, Anner N
• Application Number:
  US15/707,752
• Time in Patent Office: 491 Days
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G06V 30/2247 Characters composed of bars, e.g. CMC-7
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

An action camera system for an unmanned aerial vehicle (UAV) selects a target based on a reference image captured by an onboard camera. Image processing determines a desired orientation of the target to the UAV, by which the UAV can track the target and provide streaming video images from the desired orientation. Image processing establishes a visual lock on the target and controls the UAV to maintain the desired orientation while capturing streaming images. Additional position data provided by sensors aboard a smartphone carried by the target enhances the tracking ability of the action camera system and enables predictive analysis of the target'"'"'s future position. The action camera system may additionally provide preselected modes of operation that control UAV movement and image capture depending on the user'"'"'s desired objectives.

Claims

1. An action camera system attachable to a mobile platform, comprising:

at least one electromagnetic (EM) sensor configured to generate a sequence of images by detecting EM radiation in at least two dimensions; and

a control system electronically coupled to the at least one EM sensor, the control system including at least one processor and configured to;

identify at least one potential target based on the image sequence;

select at least one target from the at least one potential target by determining at least one reference orientation of the selected target to the mobile platform based on a reference image of the image sequence, the at least one reference orientation associated with at least one first image element of the reference image;

determine at least one subsequent orientation of the selected target to the mobile platform based on a subsequent image of the image sequence, the at least one subsequent orientation associated with at least one second image element of the subsequent image;

compare the at least one subsequent orientation to the at least one reference orientation;

and

restore the reference orientation based on the comparison of the subsequent orientation to the reference orientation by adjusting at least one of a velocity of the mobile platform and a heading of the mobile platform.

2. The action camera system of claim 1, wherein the mobile platform includes an unmanned aerial vehicle (UAV).

3. The action camera system of claim 1, wherein:

the at least one EM sensor is pivotably mounted to the mobile platform;

and

the control system is configured to restore the reference orientation by adjusting an orientation of the at least one EM sensor to the mobile platform.

4. The action camera system of claim 1, wherein the control system is configured to restore the reference orientation based on one or more of the comparison of the subsequent orientation to the reference orientation and position data.

5. The action camera system of claim 4, wherein the position data includes at least one of:

position data of the mobile platform relative to at least one reference point, the position data generated by at least one position sensor coupled to the mobile platform; and

remote position data received from at least one mobile device attached to the selected target and wirelessly coupled to the action camera system.

6. The action camera system of claim 5, wherein the position data includes a signal strength associated with the received remote position data, the signal strength determined by the control system.

7. The action camera system of claim 5, wherein the control system is configured to determine a position of the mobile platform based on the received remote position data.

8. The action camera system of claim 1, wherein the at least one first image element and the at least one second image element include one or more of:

a pattern associated with the selected target;

a face associated with the selected target;

a pseudorandom pattern;

a retroreflective surface; and

encoded data.

9. The action camera system of claim 1, wherein the control system is configured to generate an audio stream corresponding to the image sequence by processing at least one audio signal received from a microphone coupled to the mobile platform.

10. The action camera system of claim 1, wherein:

the at least one reference orientation is associated with at least one of a) a size of the at least one first image element relative to the at least one reference image and b) a position of the at least one first image element relative to the at least one reference image; and

the at least one subsequent orientation is associated with at least one of a) a size of the at least one second image element relative to the at least one subsequent image and b) a position of the at least one second image element relative to the at least one subsequent image.