Augmented reality playspaces with adaptive game rules

• US 9,454,849 B2
• Filed: 11/29/2012
• Issued: 09/27/2016
• Est. Priority Date: 11/03/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mount, Brian J., Scott, Jason, Hastings, Ryan L., Bennett, Darren, Latta, Stephen G., McCulloch, Daniel J., Geisner, Kevin A., Steed, Jonathan T., Scavezze, Michael J.
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): Ge, Jin
• Application Number:
  US13/689,378
• Publication Number:
  US 20130328927A1
• Time in Patent Office: 1,398 Days
• Field of Search: 345/633, 345/632
• US Class Current: 1/1
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/32 using local area network [LAN] connections
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/30 characterized by output arrangements for receiving control signals generated by the game device
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2219/2016 : Rotation, translation, scaling
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/60 : Analysis of geometric attributes

Abstract

A system for generating a virtual gaming environment based on features identified within a real-world environment, and adapting the virtual gaming environment over time as the features identified within the real-world environment change is described. Utilizing the technology described, a person wearing a head-mounted display device (HMD) may walk around a real-world environment and play a virtual game that is adapted to that real-world environment. For example, the HMD may identify environmental features within a real-world environment such as five grassy areas and two cars, and then spawn virtual monsters based on the location and type of the environmental features identified. The location and type of the environmental features identified may vary depending on the particular real-world environment in which the HMD exists and therefore each virtual game may look different depending on the particular real-world environment.

Claims

1. A method for generating an augmented reality environment, comprising:

generating one or more virtual objects associated with a computing application using a mobile device, the one or more virtual objects include a first virtual object;

identifying one or more real-world objects within a first real-world environment that are of a particular type;

acquiring a computing application requirement that a particular number of real-world objects of the particular type be identified within the first real-world environment from the computing application;

determining that a number of the one or more real-world objects identified within the first real-world environment of the particular type is not equal to the particular number of real-world objects required by the computing application;

adjusting the one or more virtual objects in response to determining that the number of the one or more real-world objects identified within the first real-world environment of the particular type is not equal to the particular number of real-world objects required by the computing application, the one or more virtual objects include a first virtual object that moves within the augmented reality environment;

detecting a particular sound using the mobile device;

determining a distance between the first virtual object within the augmented reality environment and the mobile device;

setting a degree of transparency for the first virtual object based on the distance between the first virtual object within the augmented reality environment and the mobile device in response to detecting the particular sound;

generating one or more images associated with the first virtual object based on the degree of transparency; and

displaying on the mobile device the one or more images, the one or more images are displayed such that the first virtual object is perceived to exist within the augmented reality environment.

2. The method of claim 1,

wherein:

the adjusting the one or more virtual objects includes increasing a speed associated with the first virtual object.

3. The method of claim 2, wherein:

the adjusting the one or more virtual objects includes scaling a size of the first virtual object.

4. The method of claim 1, wherein

the one or more images are displayed using the mobile device such that the first virtual object is perceived to be partially transparent.

5. The method of claim 2, further comprising:

determining one or more personal characteristics associated with an end user of the mobile device, the one or more personal characteristics include a height of the end user;

determining one or more locations within the first real-world environment associated with the one or more virtual objects; and

modifying the one or more locations associated with the one or more virtual objects based on the one or more personal characteristics prior to the displaying.

6. The method of claim 1, further comprising:

determining a geographic location associated with the mobile device; and

modifying the one or more virtual objects based on the geographic location prior to the displaying.

7. The method of claim 1, further comprising:

acquiring environmental information associated with a second real-world environment;

generating one or more other virtual objects based on the environmental information;

adjusting the one or more other virtual objects; and

displaying on the mobile device one or more other images associated with the one or more other virtual objects, the one or more other images are displayed such that the one or more other virtual objects are perceived to exist within the augmented reality environment.

8. The method of claim 2, wherein:

the first real-world environment comprises an emerging environment in which locations corresponding with the one or more real-world objects are not pre-defined;

the particular sound corresponds with a siren sound; and

the mobile device comprises a see-through HMD.

9. One or more storage devices containing processor readable code for programming one or more processors to perform a method for generating an augmented reality environment comprising the steps of:

identifying a first set of real-world objects within a first real-world environment;

generating one or more virtual objects associated with a computing application, the generating one or more virtual objects includes generating the one or more virtual objects based on the first set of real-world objects;

identifying a second set of real-world objects within a second real-world environment different from the first real-world environment that are of a particular type, the identifying a second set of real-world objects is performed subsequent to the generating one or more virtual objects;

acquiring a computing application requirement that a particular number of real-world objects of the particular type be identified within the second real-world environment from the computing application;

determining that a number of the second set of real-world objects identified within the second real-world environment of the particular type is not equal to the particular number of real-world objects required by the computing application;

adjusting the one or more virtual objects in response to the determining that the number of the second set of real-world objects identified within the second real-world environment of the particular type is not equal to the particular number of real-world objects required by the computing application, the one or more virtual objects include a first virtual object that moves within the augmented reality environment;

displaying on a mobile device one or more images associated with the one or more virtual objects, the one or more images are displayed such that the one or more virtual objects are perceived to exist within the second real-world environment;

detecting a particular sound within the second real-world environment;

determining a distance between the first virtual object within the augmented reality environment and the mobile device;

setting a degree of transparency for the first virtual object based on the distance between the first virtual object within the augmented reality environment and the mobile device in response to detecting the particular sound;

generating one or more new images associated with the first virtual object based on the degree of transparency; and

displaying on the mobile device the one or more new images such that the first virtual object is perceived to be partially transparent within the second real-world environment in response to detecting the particular sound.

10. The one or more storage devices of claim 9, wherein:

the second set of real-world objects of the particular type comprises one of trees, cars, or people and wherein;

the adjusting to one or more virtual objects includes increasing a speed associated with the first virtual object.

11. The one or more storage devices of claim 9, wherein:

the adjusting the one or more virtual objects includes scaling a size of the first virtual object.

12. The one or more storage devices of claim 9, wherein:

the particular sound corresponds with a siren sound.

13. The one or more storage devices of claim 9, further comprising:

determining one or more personal characteristics associated with an end user of the mobile device, the one or more personal characteristics include a height of the end user;

registering the one or more virtual objects, the registering determines one or more locations within the first real-world environment associated with the one or more virtual objects; and

modifying the one or more locations associated with the one or more virtual objects based on the one or more personal characteristics prior to the displaying.

14. The one or more storage devices of claim 9, further comprising:

acquiring landmark information associated with the second real-world environment;

generating one or more other virtual objects based on the landmark information; and

displaying on the mobile device one or more other images associated with the one or more other virtual objects, the one or more other images are displayed such that the one or more other virtual objects are perceived to exist within the first real-world environment.

15. An electronic device for generating an augmented reality environment, comprising:

one or more processors configured to identify one or more real-world objects within a first real-world environment and generate one or more virtual objects associated with a computing application based on the one or more real-world objects,

the one or more processors configured to identify one or more other real-world objects within a second real-world environment different from the first real-world environment that are of a particular type and acquire a computing application requirement that a particular number of real-world objects of the particular type be identified within the second real-world environment from the computing application,

the one or more processors configured to determine that a number of the one or more other real-world objects identified within the second real-world environment that are of the particular type is not equal to the particular number of real-world objects required by the computing application,

the one or more processors configured to adjust the one or more virtual objects in response to determining that the number of the one or more other real-world objects identified within the second real-world environment that are of the particular type is not equal to the particular number of real-world objects required by the computing application, the one or more virtual objects include a first virtual object that moves within the augmented reality environment, and detect a particular sound within the second real-world environment,

the one or more processors configured to determine a distance between the first virtual object within the augmented reality environment and the electronic device and set a degree of transparency for the first virtual object based on the distance between the first virtual object within the augmented reality environment and the electronic device in response to detecting the particular sound,

the one or more processors configured to generate one or more images associated with the first virtual object based on the degree of transparency; and

a display configured to display the one or more images, the one or more images are displayed such that the first virtual object is perceived to exist within the augmented reality environment.

16. The electronic device of claim 15, wherein:

the one or more other real-world objects of the particular type comprise one of trees, cars, or people and wherein;

the one or more processors configured to adjust the one or more virtual objects by increasing a speed associated with the first virtual object.

17. The electronic device of claim 15, wherein:

the particular sound corresponds with a siren sound.

18. The electronic device of claim 15, wherein:

the one or more processors configured to determine a height of the end user;

the one or more processors configured to register the one or more virtual objects; and

the one or more processors configured to modify one or more locations associated with the one or more virtual objects based on the height of the end user.

19. The electronic device of claim 17, wherein:

the one or more processors configured to acquire landmark information associated with a second real-world environment;

the one or more processors configured to generate one or more other virtual objects based on the landmark information; and

the display configured to display one or more other images associated with the one or more other virtual objects, the one or more other images are displayed such that the one or more other virtual objects are perceived to exist within the second real-world environment.

20. The electronic device of claim 15, wherein:

the first real-world environment comprises an emerging environment in which locations corresponding with the one or more other real-world objects are not pre-defined; and

the electronic device comprises a see-through HMD.