Hologram anchor prioritization

• US 10,379,606 B2
• Filed: 06/05/2017
• Issued: 08/13/2019
• Est. Priority Date: 03/30/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Eade, Ethan, Vanturennout, Jeroen, Kamath, Neena, Turner, Alex Christopher, Fields, David, Hildebrandt, Jonathan David, Grabner, Michael, Lazarow, Gavin Dean, Bhatnagar, Tushar Cyril
• Primary Examiner(s): Wu, Xiao M
• Assistant Examiner(s): Elbinger, Steven Z
• Application Number:
  US15/614,408
• Publication Number:
  US 20180286128A1
• Time in Patent Office: 799 Days
• Field of Search: None
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G03H 1/08 Synthesising holograms, i.e. holograms synthesized from objects or objects from hologramsusing electric digital computers G06F; G06T
• CPC Class Codes: G03H 1/0808 Methods of numerical synthesis, e.g. coherent ray tracing [CRT], diffraction specific
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A computing device is provided, including a display configured to display a plurality of holograms superimposed upon a physical environment. The computing device may further comprise a processor configured to store in non-volatile memory a representation of the physical environment, including a plurality of hologram anchors indicating locations at which the holograms are displayed. The processor may store a priority level of each hologram anchor, wherein each priority level is selected from a plurality of priority levels including a high priority level and a low priority level, and wherein at least one hologram anchor has the low priority level. The processor may determine that a total size of the plurality of hologram anchors exceeds a predetermined size threshold. The processor may, for at least one hologram anchor assigned the low priority level, delete that hologram anchor from the representation of the physical environment.

Claims

1. A computing device, comprising:

a display configured to display a plurality of holograms superimposed upon a physical environment;

non-volatile memory storing;

a representation of the physical environment including a plurality of hologram anchors indicating locations in the physical environment at which the holograms are displayed; and

a priority level of each hologram anchor, wherein each priority level is selected from a plurality of priority levels including at least a high priority level and a low priority level, and wherein at least one hologram anchor of the plurality of hologram anchors has the low priority level; and

a processor configured to;

determine that a total file size of the plurality of hologram anchors stored in the non-volatile memory exceeds a predetermined file size threshold; and

for at least one hologram anchor that was assigned the low priority level of the plurality of hologram anchors that exceeds the predetermined file size threshold, delete that hologram anchor from the representation of the physical environment stored in the non-volatile memory.

2. The computing device of claim 1, wherein at least one hologram anchor of the plurality of hologram anchors indicates a location of a neighborhood in which a plurality of holograms are displayed in the physical environment.

3. The computing device of claim 2, wherein the processor is configured to determine an area of overlap between two or more neighborhoods, and, in response to the determination of the area of overlap, de-associate at least one hologram displayed in the area of overlap from one or more of the two or more neighborhoods.

4. The computing device of claim 1, wherein a device viewing area extends to a predetermined distance from the computing device, and wherein each hologram is displayed when the hologram anchor of that hologram is within the device viewing area.

5. The computing device of claim 1, wherein, responsive to the determination that the total size of the plurality of hologram anchors exceeds the predetermined size threshold, the processor is configured to:

convey to the display an output including an option to delete one or more hologram anchors; and

responsive to selection of the option to delete the one or more hologram anchors, delete the one or more hologram anchors from the representation of the physical environment.

6. The computing device of claim 1, wherein the processor is further configured to:

receive instructions to add an additional hologram anchor to the representation of the physical environment; and

determine that the total size of the plurality of hologram anchors plus the additional hologram anchor exceeds the predetermined size threshold.

7. The computing device of claim 6, wherein the processor is further configured to prevent the additional hologram anchor from being added to the representation of the physical environment.

8. The computing device of claim 6, wherein the processor is further configured to:

determine that the additional hologram anchor has the high priority level; and

for at least one hologram anchor of the plurality of hologram anchors that was assigned the low priority level, delete data associated with that hologram anchor from the representation of the physical environment.

9. The computing device of claim 1, wherein:

the representation of the physical environment includes a plurality of surface hologram anchors that indicate a surface of an object in the physical environment; and

each surface hologram anchor has the low priority level.

10. The computing device of claim 1, wherein:

the representation of the physical environment includes a plurality of key frames, wherein each hologram anchor is associated with a key frame and each key frame includes image feature data; and

the processor is configured to;

for each key frame, determine at least one spatially adjacent key frame of the plurality of key frames;

produce a graph including each key frame of the plurality of key frames, wherein each key frame is connected by a pose graph to the at least one spatially adjacent key frame for that key frame; and

for each key frame, determine a distance on the graph between that key frame and each key frame associated with a hologram anchor that has the high priority value.

11. The computing device of claim 10, wherein the processor is configured to, for each key frame:

determine a discardability factor based at least in part on the distance on the graph between that key frame and each key frame associated with a hologram anchor that has the high priority value;

responsive to a determination that the total size exceeds the predetermined size threshold, delete the image feature data of the key frame with the highest discardability factor.

12. A method for use with a computing device including a display and a processor, comprising:

at the processor, computing a plurality of holograms for display;

at the display, displaying at least one hologram of the plurality of holograms superimposed upon a physical environment, wherein the displayed hologram is determined based at least in part on a location of the computing device in the physical environment;

at the processor;

storing in non-volatile memory;

a representation of the physical environment including a plurality of hologram anchors indicating locations in the physical environment at which the holograms are displayed; and

a priority level of each hologram anchor, wherein each priority level is selected from a plurality of priority levels including at least a high priority level and a low priority level, and wherein at least one hologram anchor of the plurality of hologram anchors has the low priority level;

determining that a total file size of the plurality of hologram anchors stored in the non-volatile memory exceeds a predetermined file size threshold; and

for at least one hologram anchor that was assigned the low priority level of the plurality of hologram anchors that exceeds the predetermined file size threshold, deleting that hologram anchor from the representation of the physical environment stored in the non-volatile memory.

13. The method of claim 12, wherein at least one hologram anchor of the plurality of hologram anchors indicates a location of a neighborhood in which a plurality of holograms are displayed in the physical environment.

14. The method of claim 12, wherein a device viewing area extends to a predetermined distance from the computing device, and wherein each hologram is displayed when the hologram anchor of that hologram is within the device viewing area.

15. The method of claim 12, further comprising, responsive to the determination that the total size of the plurality of hologram anchors exceeds the predetermined size threshold:

conveying to the display an output including an option to delete one or more hologram anchors; and

responsive to selection of the option to delete the one or more hologram anchors, deleting the one or more hologram anchors from the representation of the physical environment.

16. The method of claim 12, further comprising:

receiving instructions to add an additional hologram anchor to the representation of the physical environment; and

determining that the total size of the plurality of hologram anchors plus the additional hologram anchor exceeds the predetermined size threshold.

17. The method of claim 16, further comprising preventing the additional hologram anchor from being added to the representation of the physical environment.

18. The method of claim 16, further comprising:

determining that the additional hologram anchor has the high priority level; and

for at least one hologram anchor of the plurality of hologram anchors that was assigned the low priority level, deleting data associated with that hologram anchor from the representation of the physical environment.

19. The method of claim 12, wherein:

the representation of the physical environment includes a plurality of key frames, wherein each hologram anchor is associated with a key frame and each key frame includes image feature data; and

the method further includes;

for each key frame, determining at least one spatially adjacent key frame of the plurality of key frames;

producing a graph including each key frame of the plurality of key frames, wherein each key frame is connected by a pose graph to the at least one spatially adjacent key frame for that key frame; and

for each key frame, determining a distance on the graph between that key frame and each key frame associated with a hologram anchor that has the high priority value.

20. A computing device, comprising:

non-volatile memory storing;

a representation of a physical environment including a plurality of location anchors indicating locations in the physical environment; and

a priority level of each location anchor, wherein each priority level is selected from a plurality of priority levels including at least a high priority level and a low priority level, and wherein at least one location anchor of the plurality of location anchors has the low priority level; and

a processor configured to;

determine that a total file size of the plurality of location anchors stored in the non-volatile memory exceeds a predetermined file size threshold; and

for at least one location anchor that was assigned the low priority level of the plurality of location anchors that exceeds the predetermined file size threshold, delete that location anchor from the representation of the physical environment stored in the non-volatile memory.