Cascade recognition for personal tracking via unmanned aerial vehicle (UAV)

• US 9,892,322 B1
• Filed: 07/17/2017
• Issued: 02/13/2018
• Est. Priority Date: 03/07/2014
• Status: Active Grant

• Current Assignee: Trace Live Network Inc.
• Original Assignee: Trace Live Network Inc.
• Inventors: Beard, Paul, Chell, Cameron, Clarke, Jamie, McDermott, Craig, Racicot, Erika, Readwin, Paul
• Primary Examiner(s): Jones, Heather
• Application Number:
  US15/651,672
• Time in Patent Office: 211 Days
• Field of Search: None
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

Systems and methods for tracking a subject using an unmanned aerial vehicle (UAV) are disclosed. An unmanned aerial vehicle (UAV) includes an onboard camera to capture/stream multiple images. An onboard visual recognition module isolates image elements by analyzing the captured images, and determines whether the image elements correspond or do not correspond to the subject. Current image elements corresponding to a subject are stored in a positive database, and non-corresponding current image elements are stored in a negative database. An onboard subject tracking module defines the chosen subject, determines attributes of the subject based on comparisons of image elements, and follows the subject by directing the attitude control system to adjust the velocity and orientation of the UAV based on the determined attributes.

Claims

1. An unmanned aerial vehicle (UAV), comprising:

an airframe;

at least one attitude control system coupled to the airframe and configured to adjust one or more of a position of the UAV, a velocity of the UAV, a heading of the UAV, and a first orientation of the UAV;

at least one camera pivotably attached to the airframe by a mount and configured to capture one or more images;

one or more processors coupled to the camera and configured to;

determine at least one subject for tracking via the UAV;

extract at least one current image element from the one or more captured images;

designate the at least one current image element as one of a subject image element corresponding to the subject, a non-subject image element not corresponding to the subject;

store the at least one subject image element to a subject database, and the at least one non-subject image element to a non-subject database, the subject database and the non-subject database associated with a memory coupled to the one or more processors;

determine at least one subject attribute based on the at least one subject image element, the at least one non-subject image element, and at least one reference image element stored to a reference database associated with the memory;

determine at least one command gesture based on the at least one subject image element, the at least one command gesture associated with one or more of the subject and a command procedure;

and

execute the at least one command procedure based on one or more of the at least one subject attribute and the at least one command gesture, the command procedure comprising one or more of;

adjusting one or more of the position of the UAV, the velocity of the UAV, the heading of the UAV, and the first orientation of the UAV;

and

adjusting a second orientation of the camera relative to the UAV by articulating the at least one mount.

2. The UAV of claim 1, wherein:

the one or more processors are configured to identify at least one non-subject element based on one or more of the at least one non-subject image element and the at least one reference image element.

3. The UAV of claim 1, further comprising:

at least one image stabilizer coupled to the camera and compared to articulate the at least one mount based on the orientation of the UAV.

4. The UAV of claim 1, wherein:

the one or more processors are configured to determine the subject by selecting the subject from a plurality of potential subjects based on the at least one reference image.

5. The UAV of claim 1, wherein the one or more processors are configured to determine at least one reference relationship based on the at least one subject image element, the at least one non-subject image element, and the at least one reference image element.

6. The UAV of claim 5, wherein the at least one reference relationship includes at least one of a relative position of the subject relative to a non-subject element and a relative distance of the subject from the non-subject element.

7. The UAV of claim 6, further comprising:

at least one satellite-based position sensor configured to determine at least one of;

a first absolute position of the UAV; and

a second absolute position of the subject based on at least one of the relative position, the first absolute position, and a third absolute position of the non-subject element.

8. The UAV of claim 5, wherein the at least one reference relationship includes at least one orientation of the subject.

9. The UAV of claim 5, wherein the one or more processors are configured to execute the at least one command procedure based on the at least one reference relationship.

10. The UAV of claim 1, further comprising:

at least one transceiver coupled to the one or more processors, the at least one transceiver configured to;

transmit at least one of;

1) the one or more images; and

2) at least one reference relationship based on the at least one subject image element, the at least one non-subject image element, and the at least one reference image element;

to at least one remote display device wirelessly coupled to the at least one transceiver, the remote display device configured to display at least one of the one or more images and the at least one reference relationship to at least one user;

and

receive at least one attitude signal from the at least one remote display device.

11. The UAV of claim 10, wherein the at least one remote display device includes at least one of a head-worn display (HWD), a smartphone, a portable computing device, and a base station.

12. The UAV of claim 10, wherein the one or more processors are configured to execute the at least one command procedure based on the at least one received attitude signal.

13. The UAV of claim 10, wherein the at least one remote display device is configured to allow the user to select the at least one subject.