Remotely controlled motile device system

• US 10,197,998 B2
• Filed: 12/20/2016
• Issued: 02/05/2019
• Est. Priority Date: 12/27/2015
• Status: Active Grant

• Current Assignee: Spin Master Limited (Spin Master Corp.)
• Original Assignee: Spin Master Limited (Spin Master Corp.)
• Inventors: Steele, Edwin, Shivas, Michael
• Primary Examiner(s): Marc, McDieunel
• Application Number:
  US15/385,728
• Publication Number:
  US 20170185081A1
• Time in Patent Office: 777 Days
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/05 for sports or gaming, e.g. drone racing
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 50/19 using electrically powered motors
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: G09B 9/08 for teaching control of aircraft, e.g. Link trainer

Abstract

A remotely controlled motile device system comprises a remotely controlled motile device, and a mobile smart device that comprises a data processor operatively connected to a display screen, a memory, a user input interface, a camera, and a wireless transceiver. The memory stores computer-readable instructions that, when executed by the data processor, cause the mobile smart device to capture images of an optical reference background and the remotely controlled motile device, present the images on the display screen, register a target position relative to the optical reference background and entered via the user input interface, determine a pose of the remotely controlled motile device relative to the optical reference background, and transmit commands to the remotely controlled motile device to move to the target position.

Claims

1. A remotely controlled motile device system, comprising:

a remotely controlled motile device; and

a mobile smart device, comprising a data processor operatively connected to a display screen, a memory, a user input interface, a camera, and a wireless transceiver, wherein the memory stores computer-readable instructions that, when executed by the data processor, cause the mobile smart device to capture images via the camera of an optical reference background and the remotely controlled motile device, present the images on the display screen, register a target position relative to the optical reference background and entered via the user input interface, determine a pose of the remotely controlled motile device relative to the optical reference background at least partially based on at least one of the captured images, and transmit commands via the wireless transceiver to the remotely controlled motile device to move to the target position.

2. A remotely controlled motile device system according to claim 1, wherein the mobile smart device registers the target position received via the user input interface in two dimensions.

3. A remotely controlled motile device system according to claim 1, wherein the mobile smart device comprises a touch screen that registers the target position received via the user input interface as a point one of at and above a surface of the optical reference background.

4. A remotely controlled motile device system according to claim 1, wherein the mobile smart device comprises a touch screen that registers the target position received via the user input interface as a point along a surface of the optical reference background.

5. A remotely controlled motile device system according to claim 1, wherein the mobile smart device registers the target position received via the user input interface in three dimensions.

6. A remotely controlled motile device system according to claim 1, wherein the remotely controlled motile device comprises a set of fiducial points, and the computer-readable instructions that, when executed by the data processor, cause the mobile smart device to implement a visual pose estimator that comprises a blob tracker that determines the pose of the remotely controlled motile device by detecting the set of fiducial points relative to the optical reference background.

7. A remotely controlled motile device system according to claim 1, wherein the remotely controlled motile device comprises a set of fiducial points, and the computer-readable instructions, when executed by the data processor, cause the mobile smart device to implement a visual pose estimator that comprises a surface design tracker which determines the pose of the remotely controlled motile device based on the surface design of the remotely controlled motile device relative to the optical reference background.

8. A remotely controlled motile device system according to claim 1, wherein the computer-readable instructions, when executed by the data processor, cause the mobile smart device to determine the commands to move the remotely controlled motile device to the target position.

9. A remotely controlled motile device system, comprising:

a remotely controlled motile device; and

a mobile smart device, comprising a data processor operatively connected to a memory, a camera, and a wireless transceiver, wherein the memory stores computer-readable instructions that, when executed by the data processor, cause the mobile smart device to capture images via the camera of the remotely controlled motile device, determine a pose of the remotely controlled motile device relative to the mobile smart device at least partially based on at least one of the captured images, and transmit commands via the wireless transceiver to the remotely controlled motile device to rotate the remotely controlled motile device to a set orientation relative to one of the orientation and the position of the mobile smart device.

10. A remotely controlled motile device system according to claim 9, wherein the mobile smart device comprises a user input interface, and wherein the computer-readable instructions, when executed by the mobile smart device, cause the mobile smart device to receive commands to translate the remotely controlled motile device via the user input interface and to transmit the commands to the remotely controlled motile device.

11. A remotely controlled motile device system according to claim 10, wherein the remotely controlled motile device is a flying remotely controlled motile device.

12. A remotely controlled motile device system according to claim 10, wherein the user interface comprises a control for launching and landing the remotely controlled motile device without further player intervention.

13. A remotely controlled motile device system according to claim 10, wherein the flying remotely controlled motile device hovers stably without player intervention.

14. A remotely controlled motile device system according to claim 9, wherein the remotely controlled motile device comprises at least three fiducial points, and wherein the mobile smart device determines the pose of the remotely controlled motile device using the at least three fiducial points.

15. A remotely controlled motile device system according to claim 9, wherein the mobile smart device comprises an orientation sensor for registering changes in the orientation of the mobile smart device.

16. A remotely controlled motile device system according to claim 9, wherein the remotely controlled motile device comprises an inertial measurement unit, wherein the remotely controlled motile device transmits inertial data captured via the inertial measurement unit to the mobile smart device, wherein execution of the computer-readable instructions causes the mobile smart device to implement an inertial dead reckoning estimator that generates inertial pose estimates using the inertial data, and to determine the pose of the remotely controlled motile device by augmenting the visual pose estimates with the inertial pose estimates.