Systems and methods for utilizing anchor graphs in mixed reality environments

• US 10,217,231 B2
• Filed: 05/31/2016
• Issued: 02/26/2019
• Est. Priority Date: 05/31/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ard, Tracy K., Takacs, Gabriel, Gedye, David M., Fletcher, James, Ashman, Matthew Sammis
• Primary Examiner(s): Mushambo, Martin
• Application Number:
  US15/169,045
• Publication Number:
  US 20170345167A1
• Time in Patent Office: 1,001 Days
• Field of Search: None
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G03H 1/08 Synthesising holograms, i.e. holograms synthesized from objects or objects from hologramsusing electric digital computers G06F; G06T
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds

Abstract

Mixed-reality systems are provided for using anchor graphs within a mixed-reality environment. These systems utilize anchor vertexes that comprise at least one first key frame, a first mixed-reality element, and at least one first transform connecting the at least one first key frame to the first mixed-reality element. Anchor edges comprising transformations connect the anchor vertexes.

Claims

1. A computer system for generating an anchor graph for a mixed-reality environment, comprising:

one or more processors; and

one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following;

identify a first anchor vertex, the first anchor vertex including;

a first set of key frames, a first mixed-reality element, and at least one first transform connecting at least one key frame of the first set of key frames to the first mixed-reality element;

identify a second anchor vertex, the second anchor vertex including;

a second set of key frames, a second mixed-reality element, and at least one second transform connecting at least one key frame of the second set of key frames to the second mixed-reality element;

create a first anchor edge between the first anchor vertex and the second anchor vertex wherein the first anchor edge omits one or more separate transforms between each of the key frames in the first anchor vertex and the second anchor vertex, and wherein the first anchor edge comprises a single transform that connects the first anchor vertex and the second anchor vertex; and

save the first anchor vertex, the second anchor vertex, and the anchor edge within an anchor graph data structure.

2. The computer system as recited in claim 1, wherein the executable instructions include instructions that when executed configure the computer system to:

identify a third anchor vertex, the third anchor vertex including;

a third set of one or more key frames, a third mixed-reality element, and at least one third transform connecting at least one key frame of the third set of the one or more key frames to the third mixed-reality element;

create a second anchor edge between the second anchor vertex and the third anchor vertex; and

create a third anchor edge between the third anchor vertex and the first anchor vertex.

3. The computer system as recited in claim 2, wherein the executable instructions are configured, when executed, to cause the computer system to tune the first anchor edge based upon values associated with the third anchor edge.

4. The computer system as recited in claim 2, wherein:

the second anchor edge is created based upon one or more sensor readings gathered along a pathway that links the second anchor vertex and the third anchor vertex; and

the third anchor edge is created based upon a combination of the first anchor edge and the second anchor edge.

5. The computer system as recited in claim 2, wherein:

the second anchor edge is created based upon one or more sensor readings gathered along a pathway that links the second anchor vertex and the third anchor vertex; and

the third anchor edge is created based upon one or more key frames that link the first anchor edge and the second anchor edge.

6. The computer system as recited in claim 1, wherein creating the first anchor edge comprises calculating a direct transformation between the first anchor vertex and the second anchor vertex.

7. The computer system as recited in claim 6, wherein the first anchor edge omits key frames from the first set of key frames.

8. The computer system as recited in claim 1, wherein identifying a first anchor vertex comprises creating the first anchor vertex.

9. The computer system as recited in claim 8, wherein creating the first anchor vertex comprises creating the at least one first transform.

10. The computer system as recited in claim 9, wherein the at least one first transform is based upon an interpolation of a coordinate location associated with the first mixed-reality element, which coordinate location is derived from coordinates associated with each of the key frames in the first set of key frames.

11. The computer system as recited in claim 9, wherein the at least one first transform includes a plurality of transforms that connect a plurality of respective key frames to the first mixed-reality element.

12. The computer system as recited in claim 1, wherein the first mixed-reality element comprises a hologram.

13. A method for using an anchor graph within a mixed-reality environment, comprising:

identifying a first device with a stored first anchor graph;

detecting that the first device is within a predetermined proximity to a first physical location; and

in response to detecting that the first device is within the predetermined proximity, transmitting a part of a first anchor vertex along with a first anchor edge to the first device, wherein;

the first anchor vertex comprises;

a plurality of key frames including at least a first key frame, a first mixed-reality element, and at least one first transform connecting at least the first key frame to the first mixed-reality element, wherein the transmitting comprises transmitting the first key frame without transmitting all of the plurality of key frames from the first anchor vertex, and

the first anchor edge comprises a single transformation connecting the first anchor vertex to a second anchor vertex, wherein the second anchor vertex comprises at least one second key frame, a second mixed-reality element, and at least one second transform connecting the at least one second key frame to the second mixed-reality element, and wherein the first anchor edge omits one or more separate transforms between each of the key frames in the first anchor vertex and the second anchor vertex.

14. The method as recited in claim 13, further comprising:

receiving a new anchor vertex from the first device, wherein the new anchor vertex comprises at least one new key frame, a new mixed-reality element, and at least one new transform connecting the at least one new key frame to the new mixed-reality element;

receiving a new physical location that is associated with the physical location of the new anchor vertex; and

receiving a new anchor edge from the first device, wherein the new anchor edge comprises a transformation between the first anchor vertex and the new anchor vertex.

15. The method as recited in claim 14, further comprising:

identifying a second device with a stored second anchor graph;

detect that the second device is within a predetermined proximity to the second physical location; and

in response to detecting that the second device is within the predetermined proximity, transmit the second anchor vertex and the second anchor edge to the second device.

16. The method as recited in claim 13, wherein the first anchor edge omits key frames from the first plurality of key frames.

17. A method for using an anchor graph within a mixed-reality environment, comprising:

communicating, to a server, location data associated with a location of a first device; and

in response to communicating the location data to the server, receiving at least part of a first anchor vertex that is within a predetermined proximity to the first device and a first anchor edge, wherein;

the first anchor vertex comprises;

a first plurality of key frames including at least a first key frame, a first mixed-reality element, and at least one first transform connecting at least the first key frame to the first mixed-reality element, and

the first anchor edge comprises a transformation connecting the first anchor vertex to a second anchor vertex comprising a second plurality of key frames including at least a second key frame, a second mixed-reality element, and at least one second transform connecting at least the second key frame to the second mixed-reality element, and wherein the first anchor edge omits one or more separate transforms between each of the key frames in the first anchor vertex and the second anchor vertex.

18. The method as recited in claim 17, further comprising:

associating a second mixed-reality element with a second physical location;

generating at least one second key frame that is associated with the second-mixed-reality element;

determining at least one second transform that connects the at least one second key frame to the second mixed-reality element

generating a second anchor vertex comprising the second mixed-reality element, the at least one second key frame, and the at least one second transform;

generating a second anchor edge that comprises a transformation connecting the first anchor vertex to the second anchor vertex; and

communicating, to the server, the second anchor vertex and the second anchor edge.

19. The method as recited in claim 18, wherein the second mixed-reality element comprises a hologram.

20. The method as recited in claim 17, wherein the first anchor edge omits all of the first plurality of key frames.