Drone controlling device and method

• US 10,310,617 B2
• Filed: 06/11/2015
• Issued: 06/04/2019
• Est. Priority Date: 06/11/2015
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ekandem, Joshua, Anderson, Glen J., Durham, Lenitra M., Beckwith, Richard T., Raffa, Giuseppe, Sherman, Jamie, Yuen, Kathy
• Primary Examiner(s): Whittington, Jess G
• Application Number:
  US14/737,322
• Publication Number:
  US 20160364004A1
• Time in Patent Office: 1,454 Days
• CPC Class Codes: B64C 13/04 actuated personally
• CPC Class Codes: B64C 13/24 Transmitting means
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/00 Type of UAV
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G08C 17/02 using a radio link

Abstract

In embodiments, apparatuses, methods and storage media (transitory and non-transitory) are described that receive sensor data from one or more sensor devices that depict a user gesture in three dimensional space, determine a flight path based at least in part on the sensor data, and store the flight path in memory for use to control operation of a drone. Other embodiments may be described and/or claimed.

Claims

1. An electronic drone controlling device comprising:

one or more processors;

a memory coupled with the one or more processors;

a translation module operated by the one or more processors to;

receive sensor data from one or more sensor devices that depict a preflight user drone training gesture along a training path in three dimensional space;

receive one or more of a scale command or a repetition command;

determine a flight path based at least in part on the sensor data;

adjust the flight path based at least in part on one or more of the scale command or the repetition command;

store the adjusted flight path and an identifier associated with the adjusted flight path in the memory; and

a navigation module operated by the one or more processors to retrieve the adjusted flight path from the memory in response to a user selection of the identifier associated with the adjusted flight path, and direct a drone to fly based at least in part on the retrieved adjusted flight path, wherein the preflight user drone training gesture and the one or more of the scale command or the repetition command are to be received before drone flight while the drone is not flying.

2. The electronic drone controlling device of claim 1, wherein the one or more sensor devices include at least one of a position sensor, an accelerometer, a gyroscope, a magnetometer, an ultrasonic sensor, an inertial sensor, or an optical sensor.

3. The electronic drone controlling device of claim 1, wherein the drone includes the one or more sensor devices, and the user gesture is made using the drone.

4. The electronic drone controlling device of claim 3, wherein the navigation module is further to:

receive a flight modification command from a mobile communications device;

alter the flight path based at least in part on the flight modification command; and

direct the drone to fly based at least in part on the altered flight path.

5. The electronic drone controlling device of claim 4, wherein the flight modification command includes at least one of a scale command or a repetition command.

6. The electronic drone controlling device of claim 5, wherein the flight modification command is a voice command.

7. The electronic drone controlling device of claim 1, wherein the navigation module is further to:

receive active mode sensor data from a mobile communications device; and

direct the drone to fly based at least in part on the active mode sensor data.

8. The electronic drone controlling device of claim 7, wherein the active mode sensor data is based at least in part on an acceleration of the mobile communications device, and wherein the navigation module is further to:

alter the flight path in a manner proportional to the acceleration; and

direct the drone to fly based at least in part on the altered flight path.

9. The electronic drone controlling device of claim 1, wherein the navigation module is further to:

receive a stop flight command from a mobile communications device; and

direct the drone to fly to within a predefined distance from the mobile communications device.

10. The electronic drone controlling device of claim 1, wherein the navigation module is further to:

receive sensor data from an object;

determine a movement path of the object based at least in part on the sensor data;

alter the flight path based at least in part on the movement path to generate a mimic object flight path; and

direct the drone to fly based at least in part on the mimic object flight path.

11. The electronic drone controlling device of claim 1, wherein the navigation module is further to:

receive object tracking sensor data from one or more object tracking sensors;

determine a movement path of an object based at least in part on the object tracking sensors;

alter the flight path based at least in part on the movement path to generate a mimic object flight path; and

direct the drone to fly based at least in part on the mimic object flight path.

12. A computer implemented drone controlling method comprising:

receiving sensor data, at a computing device, from one or more sensor devices that depict a plurality of preflight user drone training gestures that moves the one or more sensor devices along a training path in three dimensional space while a drone is not flying;

determining, by the computing device, a flight path based at least in part on the sensor data, wherein the flight path is based at least in part on an average version of the plurality of preflight user drone training gestures;

storing, by the computing device, the flight path in a memory;

retrieving the flight path from the memory in response to a user selection of the flight path; and

directing the drone to fly based at least in part on the retrieved flight path.

13. The method of claim 12, wherein the drone includes the one or more sensor devices and the user gesture is made using the drone.

14. The method of claim 12, wherein a mobile communications device includes the one or more sensor devices and the user gesture is made using the mobile communications device.

15. The method of claim 12, further comprising:

receiving, by the computing device, a flight modification command; and

directing the drone to fly based at least in part on the flight modification command.

16. The method of claim 12, wherein the sensor data is included in a first set of sensor data, the method further comprising:

receiving, by the computing device while the drone is flying, a mode command; and

directing the drone to fly based at least in part on a second set of sensor data from a mobile communications device in response to the mode command.

17. At least one non-transitory computer-readable medium comprising instructions stored thereon that, in response to execution of the instructions by one or more processors of a computing device, cause the computing device to:

determine whether a preflight training mode has been selected;

receive sensor data from one or more sensor devices that depict a preflight user drone training gesture along a training path in three dimensional space while a drone is not flying in response to a determination that the preflight training mode has been selected;

determine a flight path based at least in part on the sensor data;

store the flight path and an identifier associated with the flight path in a memory;

retrieve the flight path from the memory in response to a user selection of the identifier associated with the flight path; and

direct the drone to fly based at least in part on the retrieved flight path.

18. The at least one non-transitory computer-readable medium of claim 17, wherein the one or more sensor devices include at least one of a position sensor, an accelerometer, a gyroscope, a magnetometer, an ultrasonic sensor, an inertial sensor, or an optical sensor.

19. The at least one non-transitory computer-readable medium of claim 17, wherein, in response to execution of the instructions by the one or more processors of the computing device, the computing device is further caused to transmit the flight path to a drone, and wherein:

a mobile communications device includes the one or more sensor devices; and

the user gesture is made using the mobile communications device.

20. The at least one non-transitory computer-readable medium of claim 19, wherein the preflight user drone training gesture is a first user gesture, the sensor data is included in a first set of sensor data, and, in response to execution of the instructions by one or more processors of the computing device while the drone is flying, the computing device is further caused to:

receive a mode command;

receive a second set of sensor data from the one or more sensor devices that depict a second user gesture in three dimensional space; and

transmit a signal to the drone while the drone is flying based at least in part on the second set of sensor data in response to the mode command.

21. The at least one non-transitory computer-readable medium of claim 19, wherein, in response to execution of the instructions by one or more processors of the computing device, the computing device is further caused to:

receive a flight modification command; and

transmit a flight modification signal to the drone while the drone is flying based at least in part on the modification command, wherein the modification command includes at least one of a scale command or a repetition command.

22. The at least one non-transitory computer-readable medium of claim 21, wherein the modification command is a voice command.

23. The at least one non-transistory computer-readable medium of claim 19, wherein, in response to execution of the instructions by one or more processors of the computing device, the computing device is further caused to:

receive a stop flight command; and

transmit a signal to the drone while the drone is flying based at least in part on the stop flight command, wherein the signal includes data directing the drone to return to a starting point.

24. The at least one non-transitory computer-readable medium of claim 19, wherein, in response to execution of the instructions by one or more processors of the computing device, the computing device is further caused to receive a drone selection command, wherein the drone selection command indicates one or more drones selected from a plurality of drones and the computing device is caused to transmit the flight path to the drone based at least in part on the drone selection command.