IMAGE PROCESSING FOR GESTURE-BASED CONTROL OF AN UNMANNED AERIAL VEHICLE

• US 20170242432A1
• Filed: 02/24/2016
• Published: 08/24/2017
• Est. Priority Date: 02/24/2016
• Status: Abandoned Application

• Current Assignee: Dronomy, Ltd.
• Original Assignee: Dronomy, Ltd.
• Inventors: Raz, Guy, Aphek, Ori
• Application Number:
  US15/051,709
• Publication Number:
  US 20170242432A1
• Time in Patent Office: Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

Apparatus for controlling an unmanned aerial vehicle (UAV) that includes an imaging device is described. The apparatus includes a touch screen and a processor, configured to (i) receive a gesture that is performed, on the touch screen, with respect to a first image acquired by the imaging device, the gesture indicating a requested change with respect to the first image, (ii) communicate, to the UAV, a first control signal that causes the UAV to begin executing a flying maneuver that is suggested by the gesture, (iii) identify a plurality of features in a subsequent image acquired by the imaging device, (iv) ascertain that respective positions of the features indicate that the flying maneuver has effected the requested change, and (v) in response to the ascertaining, communicate, to the UAV, a subsequent control signal that causes the UAV to stop execution of the flying maneuver. Other embodiments are also described.

Claims

1. Apparatus for controlling an unmanned aerial vehicle (UAV) that includes an imaging device, the apparatus comprising:

a touch screen; and

a processor, configured to;

receive a gesture that is performed, on the touch screen, with respect to a first image acquired by the imaging device, the gesture indicating a requested change with respect to the first image,

communicate, to the UAV, a first control signal that causes the UAV to begin executing a flying maneuver that is suggested by the gesture,

identify a plurality of features in a subsequent image acquired by the imaging device,

ascertain that respective positions of the features indicate that the flying maneuver has effected the requested change, and

in response to the ascertaining, communicate, to the UAV, a subsequent control signal that causes the UAV to stop execution of the flying maneuver.

2. The apparatus according to claim 1, wherein the processor is configured to ascertain that the respective positions of the features indicate that the flying maneuver has effected the requested change by:

based on the respective positions of the features, identifying a configurational property of the features, and

ascertaining that the configurational property of the features indicates that the flying maneuver has effected the requested change, by comparing the configurational property to a target configurational property.

3. The apparatus according to claim 1, wherein the flying maneuver is suggested by the gesture in that, while executing the flying maneuver, subsequent images captured by the imaging device become successively more exhibitory of the suggested change, relative to the first image.

4. The apparatus according to claim 1, wherein the gesture is a swipe gesture.

5. The apparatus according to claim 1, wherein the gesture is a pinch gesture.

6. The apparatus according to claim 1, wherein the gesture is a rotation gesture.

7. The apparatus according to claim 1,

wherein the subsequent image is a second subsequent image, the plurality of features are a second plurality of features, and the subsequent control signal is a second subsequent control signal, and

wherein the processor is further configured to:

identify a first plurality of features in a first subsequent image acquired by the imaging device prior to acquiring the second subsequent image, and

in response to respective positions of the first plurality of features, communicate, to the UAV, a first subsequent control signal that causes the UAV to change the execution of the flying maneuver.

8. The apparatus according to claim 7, wherein the first subsequent control signal causes the UAV to change a path of the flying maneuver.

9. The apparatus according to claim 7, wherein the first subsequent control signal causes the UAV to change a speed of the flying maneuver.

10. Apparatus for controlling an unmanned aerial vehicle (UAV) that includes an imaging device, the apparatus comprising:

a touch screen; and

a processor, configured to;

receive a gesture that is performed, on the touch screen, with respect to a first image acquired by the imaging device,

communicate, to the UAV, a first control signal that causes the UAV to begin executing a flying maneuver that is suggested by the gesture, at a first speed,

subsequently, compute a rate of change or a position of a feature in subsequent images that are acquired by the imaging device, and

in response to the rate of change being different from a target rate of change, communicate, to the UAV, a second control signal that causes the UAV to continue executing the flying maneuver at a second speed that is different from the first speed.

11. The apparatus according to claim 10, wherein the gesture indicates a requested change with respect to the first image, and wherein the flying maneuver is suggested by the gesture in that, while executing the flying maneuver, subsequent images captured by the imaging device become successively more exhibitory of the requested change, relative to the first image.

12. A method for controlling an unmanned aerial vehicle (UA) that includes an imaging device, the method comprising:

receiving a gesture that is performed with respect to a first image acquired by the imaging device, the gesture indicating a requested change with respect to the first image;

communicating, to the UAV, a first control signal that causes the UAV to begin executing a flying maneuver that is suggested by the gesture;

identifying a plurality of features in a subsequent image acquired by the imaging device;

ascertaining that respective positions of the features indicate that the flying maneuver has effected the requested change; and

in response to the ascertaining, communicating, to the UAV, a subsequent control signal that causes the UAV to stop execution of the flying maneuver.

13. The method according to claim 12, wherein ascertaining that the respective positions of the features indicate that the flying maneuver has effected the requested change comprises:

based on the respective positions of the features, identifying a configurational property of the features; and

ascertaining that the configurational property of the features indicates that the flying maneuver has effected the requested change, by comparing the configurational property to a target configurational property.

14. The method according to claim 12, wherein the flying maneuver is suggested by the gesture in that, while executing the flying maneuver, subsequent images captured by the imaging device become successively more exhibitory of the suggested change, relative to the first image.

15. The method according to claim 12, wherein the gesture is a swipe gesture.

16. The method according to claim 12, wherein the gesture is a pinch gesture.

17. The method according to claim 12, wherein the gesture is a rotation gesture.

18. The method according to claim 12,

wherein the subsequent image is a second subsequent image, the plurality of features are a second plurality of features, and the subsequent control signal is a second subsequent control signal, and

wherein the method further comprises:

identifying a first plurality of features in a first subsequent image acquired by the imaging device prior to acquiring the second subsequent image; and

in response to respective positions of the first plurality of features, communicating, to the UAV, a first subsequent control signal that causes the UAV to change the execution of the flying maneuver.

19. The method according to claim 18, wherein the first subsequent control signal causes the UAV to change a path of the flying maneuver.

20. The method according to claim 18, wherein the first subsequent control signal causes the UAV to change a speed of the flying maneuver.

21. A method for controlling an unmanned aerial vehicle (UAV) that includes an imaging device, the method comprising:

receiving a gesture that is performed with respect to a first image acquired by the imaging device;

communicating, to the UAV, a first control signal that causes the UAV to begin executing a flying maneuver that is suggested by the gesture, at a first speed;

subsequently, computing a rate of change of a position of a feature in subsequent images that are acquired by the imaging device; and

in response to the rate of change being different from a target rate of change, communicating, to the UAV, a second control signal that causes the UAV to continue executing the flying maneuver at a second speed that is different from the first speed.

22. The method according to claim 21, wherein the gesture indicates a requested change with respect to the first image, and wherein the flying maneuver is suggested by the gesture in that, while executing the flying maneuver, subsequent images captured by the imaging device become successively more exhibitory of the requested change, relative to the first image.