Augmented reality and filtering

• US 8,687,021 B2
• Filed: 08/17/2012
• Issued: 04/01/2014
• Est. Priority Date: 12/28/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bathiche, Steven N., Whitten, Jon Marcus Randall, Miller, John Mark, MacLaurin, Matthew B., Multerer, Boyd Cannon, Alkove, James M., Russell, Zachary Lewis, Wilfrid, Eric P., O'Rourke, Bret P., Roup, Oliver R.
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US13/588,147
• Publication Number:
  US 20120306920A1
• Time in Patent Office: 592 Days
• Field of Search: 345/629, 345/633
• US Class Current: 345/633
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/6009 for importing or creating game content, e.g. authoring tools during game development, adapting content to different platforms, use of a scripting language to create content
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A system (and corresponding method) that can enhance a user experience by augmenting real-world experiences with virtual world data to is provided. The augmented reality system discloses various techniques to personalize real-world experiences by overlaying or interspersing virtual capabilities (and data) with real world situations. The innovation can also filter, rank, modify or ignore virtual-world information based upon a particular real-world class, user identity or context.

Claims

1. A system that facilitates augmenting reality, comprising:

at least one processor;

at least one computer-readable storage media storing instructions which, when executed by the at least one processor, implement;

an information gathering component having a reality monitor component that gathers real-world data of at least one real-world object and a query component that obtains, via a communications network, virtual-world data based at least in part on a query related to the at least one real-world object; and

an interface component configured to receive at least a portion of the gathered real-world data and at least a portion of the virtual-world data, the interface component having a consolidation component that aggregates the gathered real-world data and the virtual-world data into aggregated data, the interface component establishes an augmented-reality experience in which at least a portion of the gathered real-world data is overlaid with at least a portion of the virtual-world data.

2. The system of claim 1, wherein the query component is configured to establish the query based at least in part on contextual factors including activity context, user context, environment context, temporal context and geo-location context.

3. The system of claim 2, wherein the aggregated data includes contextual data related to temporal context, the temporal context including at least one of a time of day and a day of week.

4. The system of claim 2, wherein the aggregated data includes contextual data related to geo-location context, the geo-location context including at least one of the at least one real-world object and a user.

5. The system of claim 2, wherein the information gathering component is configured to determine whether to monitor at least one real-world object of a plurality of real-world objects based at least in part on the contextual factors.

6. The system of claim 5, wherein the aggregated data includes contextual data related to a subset of the plurality of real-world objects.

7. The system of claim 1, wherein the reality monitor component employs a sensor component to gather context-related information.

8. The system of claim 7, wherein the sensor component is at least one of an environmental or physiological sensor.

9. The system of claim 1, wherein the consolidation component includes:

an analysis component that analyzes the gathered real-world data and the virtual-world data; and

an aggregation component that combines the gathered real-world data with a subset of the virtual-world data based at least in part on contextual data.

10. The system of claim 1, further comprising a rendering component that personalizes the augmented-reality experience based at least in part on one of user preference, user policy, or context.

11. The system of claim 10, further comprising a configuration component that organizes the augmented-reality experience based at least in part on at least one of user preference, user policy, or user context.

12. The system of claim 10, further comprising a filter component that filters virtual data for inclusion within the augmented-reality experience based at least in part on at least one of user preference, user policy, or user context.

13. The system of claim 1, further comprising a machine learning and reasoning component that employs at least one of a probabilistic and a statistical-based analysis that infers an action that a user desires to be automatically performed.

14. A computer-implemented method of augmenting reality, comprising:

monitoring a plurality of real-world objects;

gathering real-world data related to the monitored plurality of real-world objects;

establishing a query related to at least one of the plurality of real-world objects;

obtaining, via a communications network, virtual-world information related to at least one of the plurality of real-world objects in response to the query;

aggregating the gathered real-world data and the virtual-world information into aggregated data; and

establishing an augmented-reality experience in which at least a portion of the gathered real-world data is overlaid with at least a portion of the virtual-world information based at least in part on the aggregated data.

15. The method of claim 14, wherein the establishing an augmented-reality experience in which at least a portion of the gathered real-world data is overlaid with at least a portion of the virtual-world information based at least in part on the aggregated data includes rendering at least a portion of the aggregated data to a user based at least in part on one of a context, policy or preference.

16. The method of claim 15, wherein the context is at least one of activity context, user context or environment context.

17. The method of claim 14, further comprising:

filtering an augmented-reality based at least in part on a context; and

wherein the establishing an augmented-reality experience includes rendering the filtered augmented-reality.

18. A computer-executable system comprising:

means for machine learning and reasoning to infer an action that a user desires to be automatically performed;

means for monitoring a plurality of real-world objects related to a user;

means for establishing a query related to at least one of the plurality of real-world objects;

means for obtaining, via a communications network, virtual-world information related to at least one of the plurality of real-world objects in response to the query related to at least one of the plurality of real-world objects;

means for aggregating at least a portion of the gathered real-world data and with at least a portion of the virtual-world information into aggregated data; and

means for rendering the aggregation to a user in accordance with the inferred action, wherein at least a portion of the real-world data is overlaid with at least a portion of the virtual-world data.

19. The computer-executable system of claim 18, further comprising means for filtering the virtual-world information based at least in part on a user preference.

20. The computer-executable system of claim 18, wherein the means for obtaining virtual-world information related to at least one of the plurality of real-world objects in response to the query related to at least one of the plurality of real-world objects obtains the virtual-world information, from a remote source, over an internet connection.