REMOTELY CONTROLLING A SELF-PROPELLED DEVICE IN A VIRTUALIZED ENVIRONMENT

• US 20130173089A1
• Filed: 02/13/2013
• Published: 07/04/2013
• Est. Priority Date: 01/05/2011
• Status: Active Grant

• Current Assignee: Sphero Inc.
• Original Assignee: Orbotix Incorporated
• Inventors: Wilson, Adam, Carroll, Jonathan, Bernstein, Ian, Polo, Fabrizio
• Granted Patent: US 9,218,316 B2
• Time in Patent Office: Days
• US Class Current: 701/2
• CPC Class Codes: A63H 30/04 using wireless transmission
• CPC Class Codes: A63H 33/005 Motorised rolling toys
• CPC Class Codes: B62D 63/025 Modular vehicles superstructures for busses having modular sections B62D31/025
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0259 using magnetic or electromagnetic means
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

A computing device operating as a controller can obtain image data from a camera component. The computing device can determine a location of the self-propelled device relative to the camera based on the image data. A virtual content may be generated on the computing device based at least in part on the location of the self-propelled device.

Claims

1. A computer-implemented method for operating a computing device, the method comprising:

programmatically detecting, from image data generated by a camera component of the computing device, a location of a self-propelled device relative to the computing device;

receiving input from the self-propelled device, the input including sensor information that is obtained on the self-propelled device;

generating content based on the location of the self-propelled device as the self-propelled device moves.

2. The method of claim 1, wherein programmatically detecting includes detecting a spherical aspect of the self-propelled device from the image data.

3. The method of claim 1, wherein generating the image includes generating a virtual environment, and rendering a graphic representation of the self-propelled device as part of the virtual environment.

4. The method of claim 2, wherein the graphic representation is a virtual character.

5. The method of claim 1, wherein generating the content includes:

determining a map that corresponds to the scene, based on points corresponding to a path determined from detecting multiple locations of the self-propelled device in the scene.

6. The method of claim 5, wherein generating the image further includes:

generating a virtual environment that includes one or more virtual objects, and locating the objects in the virtual environment based on the map.

7. The method of claim 6, wherein locating the one or more virtual objects includes placing the objects in a region that is determined from the map, but otherwise independent of the motion of the self-propelled device.

8. The method of claim 1, wherein the sensor input includes input from one or more sensors including (i) an accelerometer, (ii) a proximity sensor, (iii) a gyroscope, and/or a magnetometer.

9. The method of claim 1, wherein receiving information includes receiving position information regarding a position of the self-propelled device, from the self-propelled device.

10. The method of claim 9, further comprising determining a location of the computing device based on the position of the self-propelled device.

11. The method of claim 1, wherein generating an image includes generating a virtual environment that is shared by multiple computing devices, the virtual environment including events that are determined by a set of rules and by events occurring on other computing devices that generate the virtual environment.

12. A computer-implemented method for operating a computing device, the method comprising:

obtaining image data from a camera component that is provided with or coupled to the computing device;

determining a location of the self-propelled device relative to the camera component using the image data;

generating virtual content, based on the location of the self-propelled device.

13. The method of claim 12, further comprising:

receiving input; and

communicating one or more commands to control movement of the self-propelled device based on the input.

14. The method of claim 12, wherein determining the location includes performing image analysis to detect a spherical aspect of the self-propelled device.

15. The method of claim 12, wherein programmatically detecting is performed on a series of frames comprising video input.

16. A self-propelled device comprising:

a housing;

one or more sensors provided within the housing;

a communication link to communicate information determined from the one or more sensors to a controller;

wherein the housing includes a spherical aspect; and

wherein the housing is modularized to be assembled onto another device as a self-contained unit, either during manufacturing or after-sale.