System and method for simulating placement of a virtual object relative to real world objects

• US 8,502,835 B1
• Filed: 09/02/2010
• Issued: 08/06/2013
• Est. Priority Date: 09/02/2009
• Status: Active Grant

• Current Assignee: Groundspeak, Inc.
• Original Assignee: Groundspeak, Inc.
• Inventors: Meehan, Patrick Carson
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): McCulley, Ryan D
• Application Number:
  US12/875,049
• Time in Patent Office: 1,069 Days
• Field of Search: None
• US Class Current: 345/633
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels

Abstract

A computer-implemented method for virtually placing a virtual object in the real world is provided. A set of virtual objects are maintained in an actions database on a mobile computing device, which each include an action associated with an identifier and coordinates for physical locations. A set of real world object models is maintained in an objects database on the device, which each include a three-dimensional model. A physical location of the device is tracked. An identification tag within range is identified. The device'"'"'s physical location and the identification tag are respectively evaluated. For each coordinates and identifiers matched, the action associated with the matched virtual object is completed. Where required, the virtual object is evaluated against the real world object models and the virtual object is rendered in light of any such real world object models located within vicinity of the device. The action is performed using the device.

Claims

1. A computer-implemented method for virtually placing a virtual object in the real world based on physical location and tagging, comprising:

maintaining a set of virtual objects in an actions database on a mobile computing device, each virtual object comprising an action associated with predetermined pairings of an identifier and coordinates for one or more physical locations;

maintaining a set of real world object models in an objects database on the mobile computing device, each real world object model comprising a three-dimensional model;

tracking a physical location of the mobile computing device and concurrently identifying an identification tag within range of the mobile computing device;

pairing the physical location of the mobile computing device and the identification tag;

evaluating the paired mobile computing device'"'"'s physical location and the identification tag respectively against the predetermined pairings of coordinates and identifiers for the virtual objects in the actions database;

identifying at least one pairing of coordinates and identifiers that match the physical location and the identification tag; and

completing the action associated with the correspondingly matched virtual object, comprising;

evaluating the coordinates of the matched virtual object stored in the actions database against coordinates of the real world object models in the objects database;

rendering the virtual object based on any such real world object models located within a vicinity of the mobile computing device; and

performing the action using the mobile computing device.

2. A method according to claim 1, wherein the vicinity of the mobile computing device comprises a perimeter surrounding the mobile computing device.

3. A method according to claim 2, further comprising:

identifying the real world object models in the objects database located within the perimeter of the mobile computing device; and

rendering the virtual object relative to only those real world object models identified as falling within the perimeter.

4. A method according to claim 3, further comprising:

determining whether the matched virtual object overlaps with each identified real world object model; and

adjusting the rendering of the virtual object in light of such overlap.

5. A method according to claim 1, wherein the real world object model corresponds to at least one of a stationary physical object, a physical fixture, and a movable physical object.

6. A method according to claim 1, further comprising:

executing an application program on the mobile computing device comprising a downloadable cartridge written as an executable script that defines triggerable events, at least one of which corresponds to the action comprised in at least one of the virtual objects.

7. A method according to claim 1, further comprising:

determining the physical location of the mobile computing device through one or more of a global positioning system receiver, a stationary data marker, and a user input.

8. A method according to claim 1, further comprising:

providing the identification tag either associated with or attached to a physical object that can be movable or stationary.

9. A method according to claim 1, further comprising one of:

defining the identification tag passively and comprising at least one of a bar code, a grid, an image tag, and a sequence that is either alphabetic, numeric, symbolic, or combinations thereof; and

defining the identification tag actively and comprising at least one of a wireless transmitter and a radio frequency identification tag.

10. A method according to claim 1, further comprising:

performing the action through the mobile computing device or through a device affiliated with the mobile computing device and comprising one or more of displayed text, visualized images or scenes, auditory sounds, and tactile feedback.

11. A computer-implemented method for virtually placing a virtual object in the real world through a network based on physical location and tagging, comprising:

maintaining a set of virtual objects in an actions database on a centralized server, each virtual object comprising an action associated with predetermined pairings of an identifier and coordinates for one or more physical locations;

maintaining a set of real world object models in an objects database on the centralized server, each real world object model comprising a three-dimensional model;

interfacing a mobile computing device with the centralized server over a data communications network; and

providing the virtual objects and the real world object models to the mobile computing device, comprising;

tracking a physical location of the mobile computing device and concurrently identifying an identification tag within range of the mobile computing device;

pairing the physical location of the mobile computing device and the identification tag on the mobile computing device;

evaluating the paired mobile computing device'"'"'s physical location and the identification tag respectively against the predetermined pairings of coordinates and identifiers for the virtual objects in the actions database;

identifying at least one pairing of coordinates and identifiers that match the physical location and the identification tag; and

completing the action associated with the correspondingly matched virtual object, comprising;

evaluating the coordinates of the matched virtual object stored in the actions database against coordinates of the real world object models in the objects database;

rendering the virtual object based on any such real world object models located within a vicinity of the mobile computing device; and

performing the action using the mobile computing device.

12. A method according to claim 11, wherein the vicinity of the mobile computing device comprises a perimeter surrounding the mobile computing device, further comprising:

identifying the real world object models in the objects database located within the perimeter of the mobile computing device; and

rendering the virtual object relative to only those real world object models identified as falling within the perimeter.

13. A method according to claim 12, further comprising:

determining whether the matched virtual object overlaps with each identified real world object model; and

adjusting the rendering of the virtual object in light of such overlap.

14. A method according to claim 11, wherein the real world object model corresponds to at least one of a stationary physical object, a physical fixture, and a movable physical object.

15. A method according to claim 11, further comprising at least one of:

performing the evaluation of the mobile computing device'"'"'s physical location and the identification tag remotely on the centralized server;

performing the evaluation of the mobile computing device'"'"'s physical location and the identification tag locally on the mobile computing device;

performing the evaluation of the matched virtual object against the real world object models remotely on the centralized server; and

performing the evaluation of the matched virtual object against the real world object models locally on the mobile computing-device.

16. A method according to claim 11, further comprising:

interfacing the mobile computing device with one or more other mobile computing devices over the data communications network; and

exchanging information regarding the virtual objects with the other mobile computing devices.

17. A method according to claim 11, further comprising:

maintaining a further set of virtual objects in a local actions database on the mobile computing device; and

performing the evaluation of the mobile computing device'"'"'s physical location and the identification tag against at least one of the actions database on the centralized server and the local actions database on the mobile computing device.

18. A method according to claim 11, further comprising:

maintaining a further set of real world object models in a local objects database on the mobile computing device; and

performing the evaluation of the matched virtual object against the real world object models against at least one of the objects database on the centralized server and the local objects database on the mobile computing device.

19. A computer-implemented method for performing an augmented reality scenario with virtual placement of a virtual object in the real world, comprising:

defining a scenario, comprising;

placing an identifier at a plurality of physical locations;

specifying at least one action to be performed at each of the physical locations at which one such identifier has been placed;

specifying at least one each real world object model comprising a three-dimensional model; and

defining virtual objects comprising each of the actions associated with predetermined pairings of the identifier and coordinates for the physical location of the action;

assembling all of the virtual objects into an actions database;

assembling all of the real world object models into an objects database; and

performing the scenario through a mobile computing device, comprising;

tracking a physical location of the mobile computing device and concurrently identifying an identification tag within range of the mobile computing device;

pairing the physical location of the mobile computing device and the identification tag on the mobile computing device;

evaluating the paired mobile computing device'"'"'s physical location and the identification tag respectively against the predetermined pairings of coordinates and identifiers for the virtual objects in the actions database;

identifying at least one pairing of coordinates and identifiers that match the physical location and the identification tag; and

completing the action associated with the correspondingly matched virtual object, comprising;

evaluating the coordinates of the matched virtual object stored in the actions database against coordinates of the real world object models in the objects database;

rendering the virtual object in light of any such real world object models located within a vicinity of the mobile computing device; and

performing the action using the mobile computing device.

20. A method according to claim 19, further comprising at least one of:

sequencing the virtual objects of the scenario in a specific order, wherein the performance of each action is conditioned on the performance of any of the actions occurring earlier in the scenario; and

listing the virtual objects of the scenario as an unordered list, wherein the performance of each action can occur independent of the performance of any of the actions occurring elsewhere in the scenario.

21. A method according to claim 19, further comprising:

maintaining the actions database on at least one of the mobile computing device and a centralized server interconnected to the mobile computing device over a data communications network; and

performing the evaluation of the mobile computing device'"'"'s physical location and the identification tag against at least one of the actions database on the centralized server and the actions database on the mobile computing device.

22. A method according to claim 19, further comprising:

maintaining a further set of real world object models on at least one of the mobile computing device and a centralized server interconnected to the mobile computing device over a data communications network; and

performing the evaluation of the matched virtual object against the real world object models against at least one of the objects database on the centralized server and the local objects database on the mobile computing device.

23. A method according to claim 19, further comprising:

interfacing the mobile computing device with one or more other mobile computing devices over a data communications network; and

exchanging information regarding the virtual objects with the other mobile computing devices.

24. A method according to claim 23, further comprising:

conditioning the performance of the actions in the virtual objects of the scenario on the mobile computing device based on progress through the scenario of the other mobile computing devices.