Interactive augmented reality using a self-propelled device

• US 9,827,487 B2
• Filed: 10/15/2013
• Issued: 11/28/2017
• Est. Priority Date: 05/14/2012
• Status: Active Grant

• Current Assignee: Sphero Inc.
• Original Assignee: Sphero Inc.
• Inventors: Polo, Fabrizio, Carroll, Jonathan, Castator-Smith, Skylar, Ingram, Ross
• Primary Examiner(s): Seth, Manav
• Application Number:
  US14/054,636
• Publication Number:
  US 20140051513A1
• Time in Patent Office: 1,505 Days
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/327 using wireless networks, e.g. Wi-Fi® or piconet
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

A method is disclosed for operating a mobile computing device. The method may include a communication link between the mobile computing device and a second computing device. The second computing device may provide a virtual environment for the mobile computing device. Furthermore, the mobile computing device may allow a user to control a self-propelled device, which may be rendered as a virtual entity upon the virtual environment.

Claims

1. A method for operating a mobile computing device to control a remote controlled vehicle, the method being performed by one or more processors of the mobile computing device and comprising:

in response to user inputs on a user interface generated on a display of the mobile computing device, transmitting control information to the remote controlled vehicle to control movement of the remote controlled vehicle in a real-world environment;

interfacing with a second computing device that provides, via execution of a program by one or more processors of the second computing device, a virtual environment;

transmitting position information, corresponding to the remote controlled vehicle, to the second computing device to cause the second computing device to render a virtual entity upon the virtual environment, the virtual entity representing the remote controlled vehicle moving in the real-world environment;

receiving, from the second computing device, data corresponding to the virtual entity within the virtual environment; and

displaying, on the display of the mobile computing device, the virtual environment including the virtual entity representing the remote controlled vehicle.

2. The method of claim 1, further comprising:

detecting an event relating to the remote controlled vehicle in the real-world environment;

determining a virtual event based on the detected event; and

incorporating the virtual event into the virtual environment.

3. The method of claim 1, further comprising:

accessing saved data associated with the virtual environment stored on the second computing device, the saved data corresponding to previous interactions within the virtual environment; and

congruently linking the saved data to a current control session comprising the virtual environment and control of the remote controlled vehicle in the real-world environment.

4. The method of claim 3, wherein the previous interactions and the current control session of the remote controlled vehicle correspond to gameplay, simultaneously controlling the remote controlled vehicle in the real-world environment and the virtual entity within the virtual environment.

5. The method of claim 4, wherein the gameplay is associated with one or more of a minigame, emergent gameplay, or a mobile game.

6. The method of claim 1, wherein the virtual environment displayed on the mobile computing device is linked to a display of the virtual environment through the second computing device.

7. The method of claim 1, wherein the virtual environment corresponds to augmented reality.

8. The method of claim 1, wherein the mobile computing device is one or more of a smart phone, a tablet computer, or a laptop computer.

9. The method of claim 1, wherein interfacing with the second computing device is performed by the mobile computing device.

10. The method of claim 1, wherein the step of interfacing is performed by the second computing device.

11. A system comprising:

a remote controlled vehicle; and

a mobile computing device comprising one or more processors executing a set of instructions that cause the one or more processors to;

in response to user inputs on a user interface generated on a display of the mobile computing device, transmit control information to the remote controlled vehicle to control movement of the remote controlled vehicle in a real-world environment;

interface with a second computing device that provides, via execution of a program by one or more processors of the second computing device, a virtual environment;

transmit position information, corresponding to the remote controlled vehicle, to the second computing device to cause the second computing device to render a virtual entity upon the virtual environment, the virtual entity representing the remote controlled vehicle moving in the real-world environment;

receive, from the second computing device, data corresponding to the virtual entity within the virtual environment; and

display, on the display of the mobile computing device, the virtual environment including the virtual entity representing the remote controlled vehicle.

12. The system of claim 11, wherein executed set of instructions further cause the one or more processors of the mobile computing device to:

detect an event associated with the remote controlled vehicle in the real-world environment;

determine a virtual event based on the detected event; and

incorporate the virtual event in the virtual environment.

13. The system of claim 11, wherein the executed set of instructions further cause the one or more processors of the mobile computing device to:

access saved data associated with the virtual environment stored on the second computing device, the saved data corresponding to previous interactions within the virtual environment.

14. The system of claim 13, wherein the executed set of instructions further cause the one or more processors of the mobile computing device to:

congruently link the saved data to a current control session comprising the virtual environment and control of the remote controlled vehicle in the real-world environment.

15. The system of claim 14, wherein the previous interactions and the current control session correspond to gameplay within the virtual environment.

16. The system of claim 15, wherein the gameplay is associated with one or more of a minigame, emergent gameplay, or a mobile game.

17. The system of claim 11, wherein the virtual environment corresponds to augmented reality.

18. The system of claim 11, wherein the mobile computing device is one or more of a smart phone, a tablet computer, or a laptop computer.

19. The system of claim 11, wherein the mobile computing device performs one or more operations that enable the mobile computing device to interface with the second computing device.

20. The system of claim 11, wherein the second computing device performs one or more operations to interface with the mobile computing device.

21. A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a mobile computing device, cause the one or more processors to:

in response to user inputs on a user interface generated on a display of the mobile computing device, transmit control information to a remote controlled vehicle to control movement of the remote controlled vehicle in a real-world environment;

interface with a second computing device that provides, via execution of a program by one or more processors of the second computing device, a virtual environment;

transmit position information, corresponding to the remote controlled vehicle to the second computing device to cause the second computing device to render a virtual entity upon the virtual environment, the virtual entity representing the remote controlled vehicle in the real-world environment;

receive, from the second computing device, data corresponding to the virtual entity within the virtual environment; and

display, on the display of the mobile computing device, the virtual environment including the virtual entity representing the remote controlled vehicle.

22. The non-transitory computer readable medium of claim 21, wherein the executed instructions further cause the one or more processors to:

detect an event relating to the remote controlled vehicle in the real-world environment;

determine a virtual event based on the detected event; and

incorporate the virtual event into the virtual environment.

23. The non-transitory computer readable medium of claim 21, wherein the instructions executed further cause the one or more processors to:

access saved data associated with the virtual environment stored on the second computing device, the saved data corresponding to previous interactions within the virtual environment; and

congruently link the saved data to a current control session comprising the virtual environment and control of the remote controlled vehicle in the real-world environment.

24. The non-transitory computer readable medium of claim 21, wherein the executed instructions further cause the one or more processors to:

render the virtual environment as augmented reality.