EXECUTABLE VIRTUAL OBJECTS ASSOCIATED WITH REAL OBJECTS

• US 20130194164A1
• Filed: 01/27/2012
• Published: 08/01/2013
• Est. Priority Date: 01/27/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sugden, Ben, Clavin, John, Vaught, Ben, Latta, Stephen, Perez, Kathryn Stone, McCulloch, Daniel, Scott, Jason, Zhang, Wei, Bennett, Darren, Hastings, Ryan, Tomlin, Arthur, Geisner, Kevin
• Granted Patent: US 8,963,805 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/1407 General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation

Abstract

Embodiments for interacting with an executable virtual object associated with a real object are disclosed. In one example, a method for interacting with an executable virtual object associated with a real object includes receiving sensor input from one or more sensors attached to the portable see-through display device, and obtaining information regarding a location of the user based on the sensor input. The method also includes, if the location includes a real object comprising an associated executable virtual object, then determining an intent of the user to interact with the executable virtual object, and if the intent to interact is determined, then interacting with the executable object.

Claims

1. On a portable see-through display device, a method for interacting with an executable virtual object associated with a real object, the method comprising:

receiving sensor input from one or more sensors attached to the portable see-through display device;

obtaining information regarding a location of the user based on the sensor input;

if the location includes a real object comprising an associated executable virtual object, then determining an intent of the user to interact with the executable virtual object; and

if the intent to interact is determined, then interacting with the executable object.

2. The method of claim 1, wherein interacting with the executable object further comprises retrieving the executable virtual object from a remote location and launching the executable object on the see-through display device.

3. The method of claim 1, wherein interacting with the executable object further comprises sending a request to a remote location to launch the executable virtual object.

4. The method of claim 1, wherein determining the intent of the user to interact with the executable object comprises determining that a gaze of the user is directed at the real object for a threshold amount of time.

5. The method of claim 1, wherein determining the intent of the user to interact with the executable object comprises detecting one or more of a voice command of the user and a gesture of the user indicating the intent of the user to interact with the executable virtual object.

6. The method of claim 1, wherein obtaining information regarding the location of the user based on the sensor input comprises obtaining information regarding a field of view of the user, and

if the field of view includes the real object comprising the associated executable virtual object, then determining the intent of the user to interact with the executable virtual object.

7. The method of claim 1, further comprising receiving contextual information regarding the real object, and wherein interacting with the virtual object comprises receiving information regarding a nearby real object that is associated with a second executable virtual object.

8. The method of claim 1, wherein the virtual object is a private virtual object, and further comprising receiving information authenticating the user to interact with the virtual object.

9. The method of claim 1, wherein the virtual object is a public virtual object.

10. The method of claim 1, wherein interacting with the executable virtual object includes receiving an invitation to join an activity occurring at the location of the user.

11. A portable see-through display device, comprising:

one or more sensors;

a logic subsystem; and

a data-holding subsystem holding instructions executable by the logic subsystem to;

receive sensor input from the one or more sensors;

obtain information regarding a location and a field of view of a user based on the sensor input;

if the field of view includes a real object comprising an associated executable virtual object, then determine if a gaze location and duration of the user; and

if the gaze of the user is directed at the real object for a threshold amount of time, then launch the executable object.

12. The portable see-through display device of claim 11, where the executable virtual object is executable to display an image on the portable see-through display device.

13. The portable see-through display device of claim 11, where the executable virtual object includes an audio content item.

14. The portable see-through display device of claim 11, where the executable virtual object includes an invitation to join an activity occurring at the location of the user.

15. The portable see-through display device of claim 11, wherein the one or more sensors comprise outward facing image sensors, inward facing image sensors, microphones, and/or motion sensors.

16. A method for associating a user-specified executable virtual object with a selected real object, the method comprising:

receiving an input of an identity of the selected real object based on one or more of input received from one or more sensors of a see-through display device and a selection of a location on a map;

receiving a request to link the user-specified executable virtual object with the selected real object such that the virtual object is executable by a selected user in proximity to the real object;

linking the virtual object with the selected real object; and

sending information regarding the virtual object and the linked real object to a remote service.

17. The method of claim 16, wherein receiving the request to link the user-specified executable virtual object with the selected real object further comprises receiving a voice command from the user.

18. The method of claim 16, wherein receiving the input of the identity of the real object comprises receiving image data of a background scene from an image sensor and determining which real object from a plurality of real objects in the background scene is the real object.

19. The method of claim 18, wherein determining which object from a plurality of real objects in the background scene is the selected real object further comprises determining which object from a plurality of real objects in the background scene is the selected real object based on one or more of a voice command of the user and a gaze location of the user.

20. The method of claim 16, wherein sending information regarding the virtual object and the linked real object to a remote service further comprises sending the virtual object to the remote service.