Unmanned aerial vehicle privacy controls

• US 10,762,795 B2
• Filed: 03/31/2016
• Issued: 09/01/2020
• Est. Priority Date: 02/08/2016
• Status: Active Grant

• Current Assignee: Skydio, Inc.
• Original Assignee: Skydio, Inc.
• Inventors: Contreras, Dana Livonia, Doyle, Lucas Palage, Kuehn, Justin Eugene, Gurel, Volkan, Johnson, Eric David, Steakley, Edward Dale
• Primary Examiner(s): Senfi, Behrooz M
• Application Number:
  US15/088,042
• Publication Number:
  US 20180025473A1
• Time in Patent Office: 1,615 Days
• Field of Search: 3482113, 3482116
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/32 for cartography or topography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 2201/10 : Recognition assisted with metadata
• CPC Class Codes: G08G 5/006 : in accordance with predefined flight zones, e.g. to avoid prohibited zones
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: H04B 7/18506 : Communications with or from aircraft, i.e. aeronautical mobile service
• CPC Class Codes: H04W 4/021 : Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences

Abstract

Disclosed in this specification are methods, systems and apparatus, including computer programs encoded on non-transitory computer storage media for unmanned aerial vehicle (UAV) flight operation and privacy controls. Based on geofence types, and UAV distance from a geofence, sensors and other devices connected to a UAV are conditionally operational. Image data collected during a UAV flight may be obfuscated by the UAV while in flight, or via a post-flight process using log data generated by the UAV.

Claims

1. An unmanned aerial vehicle (UAV) flight processing system, comprising:

one or more processors operable to;

navigate the UAV within a geofence, the geofence being a geospatial boundary in which flight operations of the UAV are performed;

obtain a digital image captured with a camera of the UAV during flight operations within the geofence, wherein the digital image is a single image or a frame from a digital video;

determine that a portion of the digital image includes at least a portion of an object located outside of the geofence, the determining comprising;

projecting the digital image onto a geospatial coordinate system; and

determining in reference to the geospatial coordinate system that the portion of the digital image falls outside a geospatial boundary of the geofence; and

obfuscate the portion of the digital image located outside the geofence.

2. The system of claim 1, wherein the portion of the digital image is obfuscated while the UAV is navigating within the geofence.

3. The system of claim 1, wherein the portion of the digital image is obfuscated by image blurring.

4. The system of claim 1, wherein the portion of the digital image is obfuscated by adjusting one or more camera parameters prior to capturing the digital image.

5. The system of claim 1, wherein the portion of the digital image is obfuscated by removing pixels of the portion of the digital image.

6. The system of claim 1, wherein the portion of the digital image is obfuscated by adjusting one or more characteristics of pixels of the portion of the digital image.

7. The system of claim 1, wherein the one or more processors are further operable to:

determine a percentage of the digital image that comprises the portion of the digital image outside the geofence; and

delete or destroy the entire digital image if the percentage exceeds a threshold value.

8. The system of claim 1, where obfuscating the portion of the digital image further comprises:

recognizing an object in the portion of the digital image as being private; and

obfuscating the object.

27. The flight processing system of claim 1, wherein determining a portion of the object is located outside of the geofence comprises:

displaying in a user interface a representation of the geofence; and

determining that the portion includes pixels located outside of the geofence.

9. A method comprising:

navigating, by one or more processors of a flight processing system, an unmanned aerial vehicle (UAV) within a geofence, the geofence being a geospatial boundary in which flight operations of the UAV are performed;

capturing during flight operations within the geofence, by a camera of the UAV, a digital image, wherein the digital image is a single image or a frame from a digital video;

determining, by the one or more processors, that a portion of the digital image includes at least a portion of an object located outside the geofence, the determining comprising;

projecting the digital image onto a geospatial coordinate system; and

determining in reference to the geospatial coordinate system that the portion of the digital image falls outside a geospatial boundary of the geofence; and

obfuscating the portion of the digital image located outside the geofence.

10. The method of claim 9, wherein the portion of the digital image is obfuscated on the UAV while the UAV is navigating within the geofence.

11. The method of claim 9, wherein the portion of the digital image is obfuscated by image blurring.

12. The method of claim 9, wherein the portion of the digital image is obfuscated by adjusting one or more camera parameters prior to capturing the digital image.

13. The method of claim 9, wherein the portion of the digital image is obfuscated by removing pixels of the digital image.

14. The method of claim 9, wherein the portion of the digital image is obfuscated by adjusting one or more characteristics of pixels in the portion of the digital image.

15. The method of claim 9, wherein the one or more processors are further operable to:

determine a percentage of the digital image that comprises the portion of the digital image outside the geofence; and

delete or destroy the entire digital image if the percentage exceeds a threshold value.

16. The method of claim 9, where obfuscating the portion of the digital image further comprises:

recognizing an object in the portion of the digital image as being private; and

obfuscating the object.

28. The method of claim 9, wherein determining a portion of the object is located outside of the geofence comprises:

projecting the digital image onto a geospatial coordinate system; and

determining in reference to the geospatial coordinate system that the portion of the digital image falls outside of the geospatial boundary of the geofence.

29. The method of claim 9, wherein determining a portion of the object is located outside of the geofence comprises:

displaying in a user interface a representation of the geofence; and

determining that the portion includes pixels located outside of the geofence.

17. A processing system, comprising:

receiving, one or more digital images, obtained by an unmanned aerial vehicle (UAV) during flight operations of the UAV within a geofence, the geofence being a geospatial boundary in which flight operations of the UAV were performed;

receiving UAV flight log data from the UAV comprising geospatial locations of where the one or more images where taken;

determining, by the one or more processors, that a portion of a digital image includes at least a portion of an object located outside of a geofence, the determining comprising;

projecting the digital image onto a geospatial coordinate system; and

determining in reference to the geospatial coordinate system that the portion of the digital image falls outside a geospatial boundary of the geofence; and

obfuscating the portion of the digital image located outside the geofence.

18. The system of claim 17, wherein the portion of the digital image is obfuscated by removing pixels of the digital image.

19. The system of claim 17, wherein the one or more processors are further operable to:

determine a percentage of the digital image that comprises the portion of the digital image outside the geofence; and

delete or destroy the entire digital image if the percentage exceeds a threshold value.

20. The method of claim 17, where obfuscating the portion of the digital image further comprises:

recognizing an object in the portion of the digital image as being private; and

obfuscating the object.

21. The system of claim 17, wherein the portion of the digital image is obfuscated by adjusting one or more characteristics of pixels in the portion of the digital image.

22. A method implemented by a computer system comprising one or more processors, the method comprising:

receiving, one or more digital images, obtained by an unmanned aerial vehicle (UAV) during flight operations of the UAV;

receiving UAV flight log data obtained from the UAV comprising geospatial locations of where the one or more images where taken;

determining, by the one or more processors, that a portion of a digital image includes at least a portion of an object located outside of a geofence, the geofence being a geospatial boundary in which flight operations of the UAV are performed, the determining comprising;

projecting the digital image onto a geospatial coordinate system; and

determining in reference to the geospatial coordinate system that the portion of the digital image falls outside a geospatial boundary of the geofence; and

obfuscating the portion of the digital image located outside the geofence.

23. The method of claim 22, wherein the portion of the digital image is obfuscated by removing pixels of the digital image.

24. The method of claim 22, wherein the one or more processors are further operable to:

determine a percentage of the digital image that comprises the portion of the digital image outside the geofence; and

delete or destroy the entire digital image if the percentage exceeds a threshold value.

25. The method of claim 22, where obfuscating the portion of the digital image further comprises:

recognizing an object in the portion of the digital image as being private; and

obfuscating the object.

26. The method of claim 22, wherein the portion of the digital image is obfuscated by adjusting one or more characteristics of pixels in the portion of the digital image.