PERSONAL AUDIO/VISUAL SYSTEM WITH HOLOGRAPHIC OBJECTS

• US 20130083018A1
• Filed: 03/27/2012
• Published: 04/04/2013
• Est. Priority Date: 09/30/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Geisner, Kevin A., Latta, Stephen G., Sugden, Ben J., Vaught, Benjamin I., Kipman, Alex Aben-Athar, Perez, Kathryn Stone
• Application Number:
  US13/430,972
• Publication Number:
  US 20130083018A1
• Time in Patent Office: Days
• US Class Current: 345/420
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A system for generating an augmented reality environment using state-based virtual objects is described. A state-based virtual object may be associated with a plurality of different states. Each state of the plurality of different states may correspond with a unique set of triggering events different from those of any other state. The set of triggering events associated with a particular state may be used to determine when a state change from the particular state is required. In some cases, each state of the plurality of different states may be associated with a different 3-D model or shape. The plurality of different states may be defined using a predetermined and standardized file format that supports state-based virtual objects. In some embodiments, one or more potential state changes from a particular state may be predicted based on one or more triggering probabilities associated with the set of triggering events.

Claims

1. A method for generating an augmented reality environment using a mobile device, comprising:

acquiring a particular file of a predetermined file format, the particular file includes information associated with one or more virtual objects, the particular file includes state information for each virtual object of the one or more virtual objects, the one or more virtual objects include a first virtual object, the first virtual object is associated with a first state and a second state different from the first state, the first state is associated with one or more triggering events, a first triggering event of the one or more triggering events is associated with the second state;

setting the first virtual object into the first state;

detecting the first triggering event;

setting the first virtual object into the second state in response to the detecting the first triggering event, the setting the first virtual object into the second state includes acquiring one or more new triggering events different from the one or more triggering events; and

generating and displaying on the mobile device one or more images associated with the first virtual object in the second state, the one or more images are displayed such that the first virtual object in the second state is perceived to exist within a real-world environment.

2. The method of claim 1, wherein:

the first state is associated with a first 3-D model of the first virtual object; and

the second state is associated with a second 3-D model of the first virtual object different from the first 3-D model, the one or more images comprise rendered versions of the second 3-D model.

3. The method of claim 2, further comprising:

displaying on the mobile device one or more other images associated with the first virtual object in the first state, the one or more other images are displayed such that the first virtual object in the first state is perceived to exist within the real-world environment, the displaying on the mobile device one or more other images associated with the first virtual object in the first state is performed prior to the detecting the first triggering event, the one or more other images comprise rendered versions of the first 3-D model.

4. The method of claim 1, wherein:

the first triggering event includes the performance of a particular hand gesture simultaneous with an eye gaze towards the first virtual object; and

the mobile device comprises a see-through HMD.

5. The method of claim 1, wherein:

the second state is associated with the one or more new triggering events different from the one or more triggering events.

6. The method of claim 1, further comprising:

predicting the second state prior to the setting the first virtual object into the second state; and

acquiring one or more secondary virtual objects in response to the predicting the second state prior to the setting the first virtual object into the second state.

7. The method of claim 6, wherein:

the predicting the second state includes determining one or more triggering probabilities associated with each of the one or more triggering events.

8. The method of claim 1, further comprising:

identifying a supplemental information provider associated with the real-world environment; and

negotiating an information transfer with the supplemental information provider, the acquiring one or more virtual objects includes acquiring the one or more virtual objects from the supplemental information provider.

9. The method of claim 8, wherein:

the negotiating an information transfer includes receiving a response from the supplemental information provider as to whether the particular file format is supported by the supplemental information provider.

10. One or more storage devices containing processor readable code for programming one or more processors to perform a method for generating an augmented reality environment comprising the steps of:

acquiring one or more virtual objects from a supplemental information provider, the one or more virtual objects include a first virtual object, the first virtual object is associated with a first state and a second state different from the first state, the first state is associated with a first 3-D model, the second state is associated with a second 3-D model different from the first 3-D model;

setting the first virtual object into the first state, the first state is associated with one or more triggering events;

predicting the second state, the predicting the second state includes determining one or more triggering probabilities associated with each of the one or more triggering events;

acquiring one or more secondary virtual objects in response to the predicting the second state;

detecting a first triggering event of the one or more triggering events associated with the second state;

setting the first virtual object into the second state in response to the detecting a first triggering event; and

generating and displaying on a mobile device one or more images associated with the first virtual object in the second state, the one or more images are displayed such that the first virtual object in the second state is perceived to exist within a real-world environment.

11. The one or more storage devices of claim 10, wherein:

the one or more images comprise rendered versions of the second 3-D model.

12. The one or more storage devices of claim 10, wherein:

the second 3-D model comprises a deformed version of the first virtual object.

13. The one or more storage devices of claim 10, further comprising:

displaying on the mobile device one or more other images associated with the first virtual object in the first state, the one or more other images are displayed such that the first virtual object in the first state is perceived to exist within the real-world environment, the displaying on the mobile device one or more other images associated with the first virtual object in the first state is performed prior to the detecting a first triggering event, the one or more other images comprise rendered versions of the first 3-D model.

14. The one or more storage devices of claim 10, wherein:

the first triggering event includes at least one of the performance of a particular physical gesture, the performance of an eye gaze towards the first virtual object for at least a particular period of time, or the performance of a particular voice command; and

the mobile device comprises a see-through HMD.

15. The one or more storage devices of claim 10, wherein:

the second state is associated with one or more new triggering events different from the one or more triggering events.

16. The one or more storage devices of claim 10, further comprising:

identifying the supplemental information provider; and

negotiating an information transfer with the supplemental information provider, the negotiating an information transfer includes receiving a response from the supplemental information provider as to whether a particular holographic file format is supported by the supplemental information provider.

17. An electronic device for generating an augmented reality environment, comprising:

one or more processors, the one or more processors establish a connection with a supplemental information provider, the one or more processors transmit a particular identity associated with one or more virtual objects to the supplemental information provider, the one or more processors receive virtual object information associated with the one or more virtual objects based on the particular identity, the virtual object information is embedded within a particular file of a particular holographic file format, the particular holographic file format comprises a predetermined structure, the one or more virtual objects include a first virtual object, the one or more processors determine a pose associated with the electronic device, the one or more processors generate one or more images associated with the first virtual object based on the pose; and

a see-through display, the see-through display displays the one or more images associated with the first virtual object, the one or more images are displayed such that the first virtual object is perceived to exist within a real-world environment in which the electronic device exists.

18. The electronic device of claim 17, wherein:

the first virtual object is associated with a first state and a second state different from the first state, the first state is associated with one or more triggering events, a first triggering event of the one or more triggering events is associated with the second state, the one or more processors set the first virtual object into the first state, the one or more processors detect the first triggering event, the one or more processors set the first virtual object into the second state in response to the detection of the first triggering event, the one or more processors acquire one or more new triggering events from the supplemental information provider different from the one or more triggering events in response to the detection of the first triggering event, the one or more images are associated with the first virtual object in the second state, the one or more images are displayed such that the first virtual object in the second state is perceived to exist within the real-world environment.

19. The electronic device of claim 18, wherein:

the first state is associated with a first 3-D model of the first virtual object; and

the second state is associated with a second 3-D model of the first virtual object different from the first 3-D model.

20. The electronic device of claim 18, wherein:

the first triggering event includes the performance of a particular hand gesture simultaneous with an eye gaze towards the first virtual object; and

the electronic device comprises a see-through HMD.