Systems and methods for surveillance with a visual marker

• US 10,698,423 B2
• Filed: 04/03/2019
• Issued: 06/30/2020
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Liu, Ang, Hu, Xiao, Zhou, Guyue
• Primary Examiner(s): Cass, Jean Paul
• Application Number:
  US16/373,849
• Publication Number:
  US 20190235531A1
• Time in Patent Office: 454 Days
• Field of Search: 701 3
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G06K 19/06037 multi-dimensional coding
• CPC Class Codes: G06K 7/1417 2D bar codes
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

A method of controlling an unmanned aerial vehicle (UAV) includes receiving an image from one or more vision sensors of the UAV. The image is captured while the UAV is in flight and includes a marker within an environment. The marker has a visual pattern. The method further includes identifying the visual pattern to determine a plurality of instructions encoded in the visual pattern, and controlling, in response to the plurality of instructions, the UAV to perform an action including at least one of adjusting position of a payload, swapping a payload, or capturing an image of a selected subject.

Claims

1. A method of controlling an unmanned aerial vehicle (UAV) comprising:

receiving, from one or more vision sensors of the UAV, an image including a marker within an environment, the image being captured while the UAV is in flight, and the marker having a visual pattern;

identifying, with aid of one or more processors, the visual pattern to determine a plurality of instructions encoded in the visual pattern, the plurality of instructions including at least one of adjusting position of a payload or swapping the payload; and

controlling, in response to the plurality of instructions, the UAV to perform an action including at least one of adjusting the position of the payload or swapping the payload.

2. The method of claim 1, wherein the environment is an indoor environment or an environment in which global position system (GPS) signals are not reliable.

3. The method of claim 1, wherein the marker includes a ribbon.

4. The method of claim 1, wherein the visual pattern includes squares of different colors.

5. The method of claim 1, wherein the visual pattern includes at least one of AprilTags, QR codes, or bar codes.

6. The method of claim 1, wherein the action is independent of flight of the UAV.

7. The method of claim 1, wherein controlling the UAV to perform the action includes controlling the UAV to perform the action within at least one of a predetermined amount of time or a predetermined distance after the visual pattern is identified.

8. The method of claim 1, further comprising:

calculating a dimension of at least a portion of the visual pattern; and

determining, with aid of the one or more processors, a location of the UAV within a three-dimensional coordinate system, based on the dimension of the at least a portion of the visual pattern.

9. The method of claim 1, further comprising:

positioning the UAV within the environment by calculating a rotation and a position of the UAV relative to the marker using at least one of intrinsic properties of the one or more vision sensors, a predefined position of the marker, or a set of points in the image.

10. The method of claim 9, wherein the UAV is patrolling the environment autonomously while the one or more vision sensors capture the image.

11. A system for controlling an unmanned aerial vehicle (UAV) comprising:

one or more vision sensors configured to capture an image including a marker within an environment, the image being captured while the UAV is flight, and the marker having a visual pattern; and

one or more processors, individually or collectively configured to;

identify the visual pattern to determine a plurality of instructions encoded in the visual pattern, the plurality of instructions including at least one of adjusting position of a payload or swapping the payload; and

control, in response to the plurality of instructions, the UAV to perform an action including at least one of adjusting the position of the payload or swapping the payload.

12. The system of claim 11, wherein the environment is an indoor environment or an environment in which global position system (GPS) signals are not reliable.

13. The system of claim 11, wherein the marker includes a ribbon.

14. The system of claim 11, wherein the visual pattern includes squares of different colors.

15. The system of claim 11, wherein the visual pattern includes at least one of AprilTags, QR codes, or bar codes.

16. The system of claim 11, wherein the action is independent of flight of the UAV.

17. The system of claim 11, wherein the one or more processors are further configured to control the UAV to perform the action within at least one of a predetermined amount of time or a predetermined distance after the visual pattern is identified.

18. The system of claim 11, wherein the one or more processors are further configured to:

calculate a dimension of at least a portion of the visual pattern; and

determine a location of the UAV within a three-dimensional coordinate system, based on the dimension of the at least a portion of the visual pattern.

19. The system of claim 11, wherein the one or more processors are further configured to:

position the UAV within the environment by calculating a rotation and a position of the UAV relative to the marker using at least one of intrinsic properties of the one or more vision sensors, a predefined position of the marker, or a set of points in the image.

20. The system of claim 19, wherein the UAV is configured to patrol the environment autonomously while the one or more vision sensors capture the image.