MAGIC WAND INTERFACE AND OTHER USER INTERACTION PARADIGMS FOR A FLYING DIGITAL ASSISTANT
First Claim
Patent Images
1. A flying digital assistant (FDA) configured for control via a portable multi-function device (PMD) as a magic wand, the FDA comprising:
- one or more rotors and associated motors;
a flight controller configured to receive commands configured to maneuver the FDA using the one or more rotors and associated motors;
a network interface configured to facilitate wireless communication with the PMD;
one or more optical sensors;
an inertial measurement unit (IMU);
one or more image capture devices;
one or more processors;
one or more memory units having instructions stored thereon, which when executed by the one or more processors, cause the FDA to;
receive, via the communications interface, an input from the PMD indicating an intent to control the FDA;
determine a current position and orientation of the FDA and PMD relative to a point of reference based in part on sensor data gathered by the one or more optical sensors and the IMU;
define a vector between the current position of the FDA and the current position of the PMD, the vector fixed relative to the position and orientation of the PMD at all times;
track a change in position and orientation of the PMD and accordingly, update the defined vector based on the change in position and orientation of the PMD;
generate commands configured for the flight controller to maneuver the FDA such that while in flight, the position of the FDA is always along the updated vector.
3 Assignments
0 Petitions
Accused Products
Abstract
Methods and systems are described for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone. In some embodiments, a magic wand user interaction paradigm is described for intuitive control of an FDA using a PMD. In other embodiments, methods for scripting a shot are described.
-
Citations
45 Claims
-
1. A flying digital assistant (FDA) configured for control via a portable multi-function device (PMD) as a magic wand, the FDA comprising:
-
one or more rotors and associated motors; a flight controller configured to receive commands configured to maneuver the FDA using the one or more rotors and associated motors; a network interface configured to facilitate wireless communication with the PMD; one or more optical sensors; an inertial measurement unit (IMU); one or more image capture devices; one or more processors; one or more memory units having instructions stored thereon, which when executed by the one or more processors, cause the FDA to; receive, via the communications interface, an input from the PMD indicating an intent to control the FDA; determine a current position and orientation of the FDA and PMD relative to a point of reference based in part on sensor data gathered by the one or more optical sensors and the IMU; define a vector between the current position of the FDA and the current position of the PMD, the vector fixed relative to the position and orientation of the PMD at all times; track a change in position and orientation of the PMD and accordingly, update the defined vector based on the change in position and orientation of the PMD; generate commands configured for the flight controller to maneuver the FDA such that while in flight, the position of the FDA is always along the updated vector.
-
-
2. A method for controlling a flying digital assistant (FDA), the method comprising:
-
determining, by one or more processors, a current position of the FDA relative to a point of reference; receiving a first input, the first input based on a detected motion of the PMD; calculating, by one or more processors, a first target position of the FDA relative to the point of reference based on the detected motion of the PMD; tracking the motion the PMD over time; updating, by one or more processors, the first target position of the FDA based on the tracked motion the PMD over time; and generating, by one or more processors, control commands configured to cause the FDA to maneuver from the current position to the first target position. - View Dependent Claims (3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A system for controlling a flying digital assistant (FDA), the system comprising:
-
one or more processors; and one or more memory units, the one or more memory units having instructions stored thereon, which when executed by the one or more processors, cause the system to; determine a current position of the FDA relative to a point of reference; receive a first input, the first input based on a detected motion of the PMD; calculate, a first target position of the FDA relative to the point of reference based on the detected motion of the PMD; track the motion of the PMD over time; update the first target position of the FDA based on the tracked motion of the PMD over time; and generate control commands configured to cause the FDA to maneuver from the current position to the first target position. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
-
-
30. A method for scripting image capture by a flying digital assistant (FDA), the FDA including an image capture device, the method comprising:
-
tracking a position and orientation of a portable multifunction device (PMD), relative to a first point of reference, over a period of time; generating a first flight path, relative to the first point of reference, based on the tracked position and orientation of the PMD over the period of time; translating the first flight path into a second flight path, relative to a second point of reference, wherein the second flight path is a scaled version of the first flight path; determining a position and orientation of the FDA relative to the second point of reference; and generating a set of control commands configured to cause the FDA to move along the second flight path and adjust image capture by the image capture device, based on the second flight path and the determined position and orientation of the FDA. - View Dependent Claims (31, 32, 33, 34, 35, 36, 37)
-
-
38. A system for scripting image capture by a flying digital assistant (FDA), the FDA including an image capture device, the system comprising:
-
one or more processors; and one or more memory units having instructions stored thereon, which when executed by the one or more processors, cause the system to; track a position and orientation of a portable multifunction device (PMD), relative to a first point of reference, over a period of time; generate a first flight path, relative to the first point of reference, based on the tracked position and orientation of the PMD over the period of time; translate the first flight path into a second flight path, relative to a second point of reference, wherein the second flight path is a scaled version of the first flight path; determine a position and orientation of the FDA relative to the second point of reference; and generate set of control commands configured to cause the FDA to move along the second flight path and adjust image capture by the image capture device, based on the second flight path and the determined position and orientation of the FDA. - View Dependent Claims (39, 40, 41, 42, 43, 44, 45)
-
Specification