Integration of a robotic system with one or more mobile computing devices

• US 8,882,560 B2
• Filed: 08/09/2013
• Issued: 11/11/2014
• Est. Priority Date: 05/28/2009
• Status: Active Grant

• Current Assignee: Digital Dream Labs, LLC
• Original Assignee: ANKI Incorporated
• Inventors: Sofman, Boris, Tappeiner, Hanns, Palatucci, Mark Matthew, DeNaele, Patrick Lee
• Primary Examiner(s): Happer, Tramar
• Application Number:
  US13/963,638
• Publication Number:
  US 20130324250A1
• Time in Patent Office: 459 Days
• Field of Search: 446/454
• US Class Current: 446/456
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63H 17/32 Acoustical or optical signalling devices
• CPC Class Codes: A63H 17/40 Toy vehicles automatically steering or reversing by collision with an obstacle
• CPC Class Codes: A63H 18/16 Control of vehicle drives by interaction between vehicle and track; Control of track elements by vehicles
• CPC Class Codes: A63H 30/04 using wireless transmission

Abstract

A robotic system is integrated with one or more mobile computing devices. Physical configurations of individual components of the system in physical space, or agents, under control of a user or users, are duplicated in a representation in virtual space. Some degree of real-time parity is maintained between the physical and virtual spaces, so as to implement a virtual environment that mirrors the physical one. Events occurring within one environment can directly influence and bear consequence on the course of events occurring within the other environment. Elements of virtual space thereby become truly interdependent and unified on a peer footing with elements in physical space. In at least one embodiment, the system of the present invention is implemented as an application in entertainment, such as the manifestation of a video game in physical space.

Claims

1. A system comprising:

a plurality of physical agents, the physical agents being configured for interaction with one another, each physical agent configured to;

receive signals;

perform actions in a physical environment in response to the received signals; and

transmit signals indicative of the physical agent'"'"'s state in the physical environment;

at least one controller, configured to receive user input for controlling at least one of the physical agents, and further configured to transmit signals for controlling at least one of the physical agents; and

a host device, configured to;

receive signals indicative of the physical state of the physical agents;

transmit signals specifying actions to be taken by the physical agents;

store virtual representations of the physical agents, the virtual representations of the physical agents being configured for interaction with one another; and

maintain substantial parity between the physical states of the physical agents and the states of the corresponding stored virtual representations by, for at least one of the physical agents;

receiving a signal from the physical agent, the signal indicating a position of the physical agent; and

responsive to the received signal indicating a position that differs from a position of a stored virtual representation of the physical agent, transmitting a signal to cause the physical agent to change its position to comport with position of the stored virtual representation of the physical agent.

2. The system of claim 1, wherein the host device maintaining substantial parity between the physical states of the physical agents and the states of the corresponding stored virtual representations further comprises:

adjusting the states of the stored virtual representations based on received signals indicative of the physical states of the physical agents; and

adjusting the states of the stored virtual representations based on interactions among the virtual representations.

3. The system of claim 2, wherein, for two physical agents having similar physical characteristics, the virtual representations of the two physical agents have at least one characteristic that differs substantially from one another;

and wherein the interaction between the virtual representations of the physical agents reflects the substantial difference in the at least one characteristic.

4. The system of claim 2, wherein, for two physical agents having similar mass, the virtual representations of the two physical agents have mass that differs substantially from one another;

and wherein the interaction between the virtual representations of the physical agents reflects the substantial difference in mass.

5. The system of claim 1, wherein the host device maintaining substantial parity between the physical states of the physical agents and the states of the corresponding stored virtual representations further comprises:

adjusting the states of the stored virtual representations based on interactions among the virtual representations; and

transmitting signals to cause the physical agents to change their physical states to comport with the states of the stored virtual representations.

6. The system of claim 5, wherein, for two physical agents having similar physical characteristics, the virtual representations of the two physical agents have at least one characteristic that differs substantially from one another;

and wherein the interaction between the virtual representations of the physical agents reflects the substantial difference in the at least one characteristic.

7. The system of claim 1, wherein the host device maintaining substantial parity between the physical states of the physical agents and the states of the corresponding stored virtual representations further comprises:

adjusting the states of the stored virtual representations based on received signals indicative of the physical states of the physical agents.

8. The system of claim 1, wherein the physical agents comprise mobile agents, wherein each mobile agent comprises:

a propulsion mechanism, configured to impart motive force to the mobile agent;

a sensor, configured to detect a position of the mobile agent;

a mobile wireless transceiver, and

a microcontroller operatively coupled to the propulsion mechanism, the sensor, and the mobile wireless transceiver, the microcontroller configured to control movement of the mobile agent.

9. The system of claim 8, further comprising:

a drivable surface having a plurality of machine-readable codes indicating locations on the surface;

wherein the mobile agents comprise vehicles configured to travel along the drivable surface, and wherein the sensor of each mobile agent detecting a position of the mobile agent comprises the sensor detecting the machine-readable codes as the vehicle travels along the surface.

10. The system of claim 1, wherein the host device comprises a wireless transceiver, and wherein:

the host device receiving signals comprises the wireless transceiver receiving signals; and

the host device transmitting signals comprises the wireless transceiver transmitting signals.

11. The system of claim 1, wherein each controller comprises at least one selected from the group consisting of:

a mobile computing device;

a smartphone;

a tablet computer;

a desktop computer;

a laptop computer;

a video game console; and

a kiosk;

and wherein the host device comprises at least one selected from the group consisting of;

a mobile computing device;

a smartphone;

a tablet computer;

a desktop computer;

a laptop computer;

a video game console; and

a kiosk.

12. The system of claim 1, further comprising:

an accessory, positionable within the physical environment;

wherein the host device stores a virtual representation of the accessory, and wherein the accessory is configured for affecting a state of the virtual representations of the physical agents;

and wherein, responsive to an interaction between the virtual representation of a physical agent and the virtual representation of the accessory, the host device transmits a signal to cause the physical agent to change its physical state to reflect the result of the interaction.

13. The system of claim 1, further comprising:

a mobile computing device, positionable within the physical environment and configured to display a dynamic representation of an accessory agent;

wherein the host device stores a virtual representation of the accessory agent, and wherein the virtual representation of the accessory agent is configured for interaction with virtual representations of the physical agents;

and wherein, responsive to an interaction between the virtual representation of the accessory agent and the virtual representation of a physical agent, the host device transmits a signal to cause the physical agent to change its physical state to reflect the result of the interaction.

14. The system of claim 13, wherein at least one controller is configured to receive user input for controlling the accessory agent, and further configured to transmit signals for controlling the accessory agent;

and wherein the host device adjusts the state of the stored virtual representation of the accessory agent responsive to receiving a signal for controlling the accessory agent.

15. The system of claim 14, wherein the accessory agent comprises a weapon, and wherein the virtual representation of the accessory agent is configured for firing upon a virtual representation of a physical agent.

16. The system of claim 1, wherein the virtual representation of at least one physical agent comprises a weapon, and wherein the virtual representation of the physical agent is configured for firing upon a virtual representation of another physical agent;

and wherein, responsive to a virtual representation of an agent being fired upon;

the host device causes the virtual representation to react as though fired upon; and

the host device transmits a signal to cause the corresponding physical agent to change its physical state to react as though fired upon.

17. The system of claim 1, wherein at least two controllers are configured to control a common physical agent.

18. The system of claim 1, further comprising:

an automated computing system, configured to control at least one of the physical agents in an automated manner;

wherein at least one of the physical agents is configured to receive signals from the automated computing system and to perform actions in the physical environment in response to the received signals, the physical agent being configured for interaction with at least one physical agent not controlled by the automated computing system.

19. The system of claim 1, wherein the at least one controller transmitting signals for controlling at least one of the physical agents comprises the at least one controller transmitting signals over the Internet.

20. The system of claim 1, further comprising:

at least one display device, configured to receive signals from the host device indicating the state of the virtual representations of the physical agents, and further configured to display a dynamic representation of the physical agents based on the received signals.

21. The system of claim 1, wherein the host device maintaining substantial parity between the physical states of the physical agents and the states of the corresponding stored virtual representations further comprises, for at least one of the physical agents:

receiving a signal from the physical agent, the signal indicating a position of the physical agent; and

adjusting a position of a stored virtual representation of the physical agent to comport with the indicated position of the physical agent.

22. A method for using mobile computing devices to control physical agents within a robotic system, the physical agents being configured for interaction with one another, the method comprising, at a host device, performing the steps of:

receiving signals indicative of the physical state of the physical agents in a physical environment;

transmitting signals specifying physical actions to be taken by the physical agents in the physical environment;

storing virtual representations of the physical agents, the virtual representations of the physical agents being configured for interaction with one another; and

maintaining substantial parity between the physical states of the physical agents and the states of the corresponding stored virtual representations of the physical agents by, for at least one of the physical agents;

receiving a signal from the physical agent, the signal indicating a position of the physical agent; and

responsive to the received signal indicating a position that differs from a position of a stored virtual representation of the physical agent, transmitting a signal to cause the physical agent to change its position to comport with position of the stored virtual representation of the physical agent;

wherein the physical agents are further configured to perform physical actions in response to signals received from mobile computing devices acting as user-operated controllers.

23. The method of claim 22, wherein maintaining substantial parity between the physical state of the physical agent and the state of the corresponding stored virtual representation further comprises:

adjusting the state of the stored virtual representations based on received signals indicative of the physical state of the physical agents; and

adjusting the state of the stored virtual representations based on interactions among the virtual representations.

24. The method of claim 22, wherein maintaining substantial parity between the physical state of the physical agent and the state of the corresponding stored virtual representation further comprises:

adjusting the state of the stored virtual representations based on interactions among the virtual representations; and

transmitting signals to cause the physical agents to change their physical state to comport with the state of the stored virtual representations.

25. The method of claim 22, wherein maintaining substantial parity between the physical state of the physical agent and the state of the corresponding stored virtual representation further comprises:

adjusting the state of the stored virtual representations based on received signals indicative of the physical state of the physical agents.

26. A computer program product for using mobile computing devices to control physical agents within a robotic system, the physical agents being configured for interaction with one another, the computer program product comprising:

a non-transitory computer-readable storage medium; and

computer program code, encoded on the medium, configured to cause at least one processor at a host device to perform the steps of;

receiving signals indicative of the physical state of the physical agents in a physical environment;

transmitting signals specifying physical actions to be taken by the physical agents in the physical environment;

storing virtual representations of the physical agents, the virtual representations of the physical agents being configured for interaction with one another; and

maintaining substantial parity between the physical states of the physical agents and the states of the corresponding stored virtual representations of the physical agents by, for at least one of the physical agents;

receiving a signal from the physical agent, the signal indicating a position of the physical agent; and

responsive to the received signal indicating a position that differs from a position of a stored virtual representation of the physical agent, transmitting a signal to cause the physical agent to change its position to comport with position of the stored virtual representation of the physical agent;

wherein the physical agents are further configured to perform physical actions in response to signals received from mobile computing devices acting as user-operated controllers.

27. The computer program product of claim 26, wherein the computer program code configured to cause at least one processor to maintain substantial parity between the physical state of the physical agent and the state of the corresponding stored virtual representation further comprises computer program code configured to cause at least one processor to perform the steps of:

adjusting the state of the stored virtual representations based on received signals indicative of the physical state of the physical agents; and

adjusting the state of the stored virtual representations based on interactions among the virtual representations.

28. The computer program product of claim 26, wherein the computer program code configured to cause at least one processor to maintain substantial parity between the physical state of the physical agent and the state of the corresponding stored virtual representation further comprises computer program code configured to cause at least one processor to perform the steps of:

adjusting the state of the stored virtual representations based on interactions among the virtual representations; and

transmitting signals to cause the physical agents to change their physical state to comport with the state of the stored virtual representations.

29. The computer program product of claim 26, wherein the computer program code configured to cause at least one processor to maintain substantial parity between the physical state of the physical agent and the state of the corresponding stored virtual representation further comprises computer program code configured to cause at least one processor to perform the step of:

adjusting the state of the stored virtual representations based on received signals indicative of the physical state of the physical agents.