Augmented reality spaces with adaptive rules

• US 10,062,213 B2
• Filed: 08/11/2016
• Issued: 08/28/2018
• Est. Priority Date: 11/03/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mount, Brian J., Scott, Jason, Hastings, Ryan L., Bennett, Darren, Latta, Stephen G., McCulloch, Daniel J., Geisner, Kevin A., Steed, Jonathan T., Scavezze, Michael J.
• Primary Examiner(s): Ge, Jin
• Application Number:
  US15/234,607
• Publication Number:
  US 20160350978A1
• Time in Patent Office: 747 Days
• Field of Search: 345633
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/32 using local area network [LAN] connections
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/30 characterized by output arrangements for receiving control signals generated by the game device
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2219/2016 : Rotation, translation, scaling
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/60 : Analysis of geometric attributes

Abstract

A system for generating a virtual gaming environment based on features identified within a real-world environment, and adapting the virtual gaming environment over time as the features identified within the real-world environment change is described. Utilizing the technology described, a person wearing a head-mounted display device (HMD) may walk around a real-world environment and play a virtual game that is adapted to that real-world environment. For example, the HMD may identify environmental features within a real-world environment such as five grassy areas and two cars, and then spawn virtual monsters based on the location and type of the environmental features identified. The location and type of the environmental features identified may vary depending on the particular real-world environment in which the HMD exists and therefore each virtual game may look different depending on the particular real-world environment.

Claims

1. A method for generating an augmented reality environment, comprising:

displaying one or more images corresponding with a first virtual object within the augmented reality environment using a mobile device;

detecting a particular sound while displaying the one or more images;

determining a distance between the first virtual object within the augmented reality environment and the mobile device displaying the one or more images;

setting a degree of transparency for the first virtual object based on the distance between the first virtual object within the augmented reality environment and the mobile device in response to detecting the particular sound;

generating one or more new images corresponding with the first virtual object based on the degree of transparency; and

displaying the one or more new images corresponding with the first virtual object within the augmented reality environment using the mobile device in response to detecting the particular sound.

2. The method of claim 1, further comprising:

identifying one or more real-world objects within the augmented reality environment that are of a particular type;

acquiring a computing application requirement that a particular number of real-world objects of the particular type be identified within the augmented reality environment from the computing application;

determining that a number of the one or more real-world objects identified within the augmented reality environment of the particular type is not equal to the particular number of real-world objects required by the computing application; and

adjusting a speed associated with the first virtual object in response to determining that the number of the one or more real-world objects identified within the augmented reality environment of the particular type is not equal to the particular number of real-world objects required by the computing application.

3. The method of claim 2, wherein:

the adjusting the speed associated with the first virtual object comprises increasing the speed of the first virtual object within the augmented reality environment.

4. The method of claim 2, wherein:

the augmented reality environment comprises an emerging environment in which locations corresponding with the one or more real-world objects are not pre-defined; and

the particular sound corresponds with a siren sound.

5. The method of claim 1, wherein:

the one or more new images are displayed using the mobile device such that the first virtual object is perceived to be partially transparent within the augmented reality environment.

6. The method of claim 1, further comprising:

determining a geographic location of the mobile device; and

modifying the first virtual object based on the geographic location.

7. The method of claim 6, wherein:

the modifying the first virtual object comprises scaling a size of the first virtual object.

8. The method of claim 1, wherein:

the mobile device comprises a head-mounted display device.

9. A mobile device for generating an augmented reality environment, comprising:

a see-through display configured to display one or more images corresponding with a first virtual object within the augmented reality environment; and

one or more processors configured to detect a particular sound while the one or more images are displayed and determine a distance between the first virtual object within the augmented reality environment and the mobile device, the one or more processors configured to set a degree of transparency for the first virtual object based on the distance between the first virtual object within the augmented reality environment and the mobile device in response to detecting the particular sound, the one or more processors configured to generate one or more new images corresponding with the first virtual object based on the degree of transparency, the see-through display configured to display the one or more new images corresponding with the first virtual object within the augmented reality environment.

10. The mobile device of claim 9, wherein:

the one or more processors configured to identify one or more real-world objects within the augmented reality environment that are of a particular type and acquire a computing application requirement that a particular number of real-world objects of the particular type be identified within the augmented reality environment from the computing application, the one or more processors configured to determine that a number of the one or more real-world objects identified within the augmented reality environment of the particular type is not equal to the particular number of real-world objects required by the computing application and adjust a speed associated with the first virtual object in response to determining that the number of the one or more real-world objects identified within the augmented reality environment of the particular type is not equal to the particular number of real-world objects required by the computing application.

11. The mobile device of claim 10, wherein:

the one or more processors configured to increase the speed of the first virtual object within the augmented reality environment.

12. The mobile device of claim 10, wherein:

the augmented reality environment comprises an emerging environment in which locations corresponding with the one or more real-world objects are not pre-defined; and

the particular sound corresponds with a siren sound.

13. The mobile device of claim 9, wherein:

the one or more new images are displayed using the see-through display such that the first virtual object is perceived to be partially transparent within the augmented reality environment.

14. The mobile device of claim 9, wherein:

the one or more processors configured to determine a geographic location of the mobile device and modify the first virtual object based on the geographic location.

15. The mobile device of claim 14, wherein:

the one or more processors configured to adjust a size of the first virtual object.

16. The mobile device of claim 15, wherein:

the one or more processors configured to reduce a size of the first virtual object.

17. The mobile device of claim 9, wherein:

the mobile device comprises a head-mounted display device.

18. One or more storage devices containing processor readable code for programming one or more processors to perform a method for generating an augmented reality environment comprising the steps of:

displaying one or more images corresponding with a first virtual object within the augmented reality environment using a mobile device;

detecting a particular sound while displaying the one or more images using the mobile device;

determining a distance between the first virtual object within the augmented reality environment and the mobile device displaying the one or more images;

setting a degree of transparency for the first virtual object based on the distance between the first virtual object within the augmented reality environment and the mobile device in response to detecting the particular sound;

generating one or more new images corresponding with the first virtual object based on the degree of transparency; and

displaying the one or more new images corresponding with the first virtual object within the augmented reality environment using the mobile device in response to detecting the particular sound.

19. The one or more storage devices of claim 18, further comprising:

identifying one or more real-world objects within the augmented reality environment that are of a particular type;

acquiring a computing application requirement that a particular number of real-world objects of the particular type be identified within the augmented reality environment;

determining that a number of the one or more real-world objects identified within the augmented reality environment of the particular type is not equal to the particular number of real-world objects required by the computing application; and

increasing a speed associated with the first virtual object in response to determining that the number of the one or more real-world objects identified within the augmented reality environment of the particular type is not equal to the particular number of real-world objects required by the computing application.

20. The one or more storage devices of claim 18, wherein:

the particular sound corresponds with a siren sound; and

the mobile device comprises a head-mounted display device.