Personal audio/visual system for providing an adaptable augmented reality environment

• US 9,285,871 B2
• Filed: 03/27/2012
• Issued: 03/15/2016
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Geisner, Kevin A., Latta, Stephen G., Sugden, Ben J., Vaught, Benjamin I., Kipman, Alex Aben-Athar, Perez, Kathryn Stone, Hastings, Ryan L., Bennett, Darren, McCulloch, Daniel J., Clavin, John, Scott, Jason
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Nguyen, Vu
• Application Number:
  US13/431,135
• Publication Number:
  US 20130083062A1
• Time in Patent Office: 1,449 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2380/00 Specific applications
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

A system for generating an augmented reality environment in association with one or more attractions or exhibits is described. In some cases, a see-through head-mounted display device (HMD) may acquire one or more virtual objects from a supplemental information provider associated with a particular attraction. The one or more virtual objects may be based on whether an end user of the HMD is waiting in line for the particular attraction or is on (or in) the particular attraction. The supplemental information provider may vary the one or more virtual objects based on the end user'"'"'s previous experiences with the particular attraction. The HMD may adapt the one or more virtual objects based on physiological feedback from the end user (e.g., if a child is scared). The supplemental information provider may also provide and automatically update a task list associated with the particular attraction.

Claims

1. A method for generating an augmented reality environment using a mobile device, comprising:

detecting a user of the mobile device within a particular area of an attraction;

identifying an age associated with the user;

acquiring at the mobile device a set of virtual objects associated with the attraction based on the age of the user;

detecting that a heart rate of the user is above a particular threshold while the user is within the particular area;

identifying one or more virtual objects of the set of virtual objects that were viewed by the user within a particular period of time prior to detecting that the heart rate of the user is above the particular threshold;

adjusting an object speed associated with each of the one or more virtual objects;

detecting that the heart rate of the user has not changed subsequent to adjusting the object speed associated with each of the one or more virtual objects;

replacing the one or more virtual objects with one or more other virtual objects different from the one or more virtual objects in response to detecting that the heart rate of the user has not changed subsequent to adjusting the object speed associated with each of the one or more virtual objects; and

generating and displaying on the mobile device one or more images associated with the one or more other virtual objects in response to replacing the one or more virtual objects with the one or more other virtual objects, the one or more images are displayed such that the one or more other virtual objects are perceived to exist within the particular area.

2. The method of claim 1, further comprising:

detecting that the user has exited the particular area; and

disabling the one or more other virtual objects from being displayed on the mobile device in response to detecting that the user has exited the particular area.

3. The method of claim 2, further comprising:

acquiring an attraction placement test, the attraction placement test includes one or more placement test virtual objects; and

generating and displaying the one or more other placement test virtual objects while the user is within the particular area.

4. The method of claim 2, wherein:

the disabling of the one or more other virtual objects prevents the one or more other virtual objects from being displayed on the mobile device.

5. The method of claim 1, further comprising:

detecting that the user is at the front of a line associated with the attraction; and

disabling the one or more other virtual objects from being displayed on the mobile device in response to detecting that the user is at the front of the line.

6. The method of claim 1, wherein:

the attraction comprises a ride at an amusement park.

7. The method of claim 1, wherein:

the mobile device comprises a head-mounted display device.

8. One or more storage devices containing processor readable code for programming one or more processors to perform a method for generating an augmented reality environment using a mobile device comprising the steps of:

detecting that the mobile device is within a particular area of an attraction;

acquiring at the mobile device a set of virtual objects based on an age of the user;

detecting that a heart rate of the user is above a particular threshold while the user is within the particular area;

identifying one or more virtual objects of the set of virtual objects that were viewed by the user within a particular period of time prior to detecting that the heart rate of the user is above the particular threshold;

adjusting an object speed associated with each of the one or more virtual objects, the adjusting an object speed comprises slowing down a movement of each of the one or more virtual objects;

detecting that the heart rate of the user has not changed subsequent to adjusting the object speed;

replacing the one or more virtual objects with one or more other virtual objects different from the one or more virtual objects in response to detecting that the heart rate of the user has not changed subsequent to adjusting the object speed; and

generating and displaying on the mobile device one or more images associated with the one or more other virtual objects, the one or more images are displayed such that the one or more other virtual objects are perceived to exist within the particular area.

9. The one or more storage devices of claim 8, further comprising:

automatically detecting that the user is at the front of a line associated with the attraction; and

disabling the one or more other virtual objects in response to detecting that the user is at the front of the line.

10. The one or more storage devices of claim 9, wherein:

the disabling the one or more virtual other objects prevents the one or more other virtual objects from being displayed on the mobile device.

11. The one or more storage devices of claim 9, wherein:

the attraction comprises a ride at an amusement park.

12. The one or more storage devices of claim 8, wherein:

the detecting that the mobile device is within the particular area includes verifying that a connection between the mobile device and a supplemental information provider has persisted for at least a predefined amount of time.

13. The one or more storage devices of claim 8, wherein:

the mobile device comprises a head-mounted display device.

14. An electronic device for generating an augmented reality environment, comprising:

one or more processors configured to detect a user of the electronic device within a particular area of an attraction and acquire a set of virtual objects associated with the attraction based on an age of the user, the one or more processors configured to detect that a heart rate of the user is above a particular threshold while the user is within the particular area and identify one or more virtual objects of the set of virtual objects that were viewed by the user within a particular period of time prior to detecting that the heart rate of the user is above the particular threshold, the one or more processors configured to adjust an object speed associated with each of the one or more virtual objects and detect that the heart rate of the user has not changed subsequent to adjusting the object speed, the one or more processors configured to replace the one or more virtual objects with one or more other virtual objects different from the one or more virtual objects in response to detecting that the heart rate of the user has not changed subsequent to adjusting the object speed, the one or more processors configured to generate one or more images associated with the one or more other virtual objects; and

a see-through display configured to display the one or more images such that the one or more other virtual objects are perceived to exist within the particular area.

15. The electronic device of claim 14, wherein:

the one or more processors configured to detect the user exiting the particular area and disable the one or more other virtual objects in response to detecting the user exiting the particular area.

16. The electronic device of claim 14, wherein:

the attraction comprises a ride at an amusement park.

17. The electronic device of claim 14, wherein:

the attraction comprises a museum exhibit.

18. The electronic device of claim 14, wherein:

the electronic device comprises a head-mounted display device.