Determination of environmental augmentation allocation data

• US 10,429,653 B2
• Filed: 06/28/2016
• Issued: 10/01/2019
• Est. Priority Date: 07/02/2015
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Eronen, Antti, Leppanen, Jussi, Lehtiniemi, Arto
• Primary Examiner(s): Sheng, Tom V
• Application Number:
  US15/739,203
• Publication Number:
  US 20180210206A1
• Time in Patent Office: 1,190 Days
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking

Abstract

A method including receiving augmentation allocation property data that includes data that correlates a virtual information region with at least one surface property, receiving information indicative of at least one physical object that is in a physical environment proximate to the head mounted display, determining that the physical object has a surface that conforms with the surface property, determining environmental augmentation allocation data that includes data that correlates the virtual information region with the surface of the physical object such that the environmental augmentation allocation data designates the surface of the physical object for display of a representation of information allocated to the virtual information region, determining that at least part of the surface of the physical object is at least partially within a field of view of the head mounted display, and causing display of at least part of information that is allocated to the virtual information region.

Claims

1. A method comprising:

receiving, by a head mounted display, augmentation allocation property data that comprises data that correlates a virtual information region with at least one surface property;

receiving information indicative of at least one physical object that is in a physical environment proximate to the head mounted display;

determining that the physical object has a surface that conforms with the surface property;

determining environmental augmentation allocation data that comprises data that correlates the virtual information region with the surface of the physical object such that the environmental augmentation allocation data designates the surface of the physical object for display of a representation of information allocated to the virtual information region, the determination of the environmental augmentation allocation data being based, at least in part, on the determination that the physical object has the surface that conforms with the surface property;

determining that at least part of the surface of the physical object is at least partially within a field of view of the head mounted display; and

causing display of at least part of information that is allocated to the virtual information region based, at least in part, on the environmental augmentation allocation data and the determination that at least part of the surface of the physical object is at least partially within the field of view of the head mounted display.

2. The method of claim 1, wherein the surface property comprises a surface orientation property.

3. The method of claim 2, wherein the determination that the physical object has the surface that conforms with the surface property comprises determination that an orientation of the surface of the physical object is substantially similar to the surface orientation property.

4. The method of claim 1, wherein the surface property comprises a surface location property.

5. The method of claim 4, wherein the surface location property indicates a location relative to a reference location of the physical environment.

6. The method of claim 5, further comprising receipt of information indicative of a reference location designation input that designates the reference location, wherein determination that the physical object has the surface that conforms with the surface property comprises determination that the surface of the object is substantially proximate to the surface location property relative to the reference location.

7. The method of claim 4, wherein the determination that the physical object has the surface that conforms with the surface property comprises determination that a location of the surface of the physical object is substantially similar to the surface location property.

8. The method of claim 1, wherein the augmentation allocation property data comprises data that correlates the virtual information region with at least one object classification property, and further comprising determination that the physical object conforms with the object classification property.

9. The method of claim 8, further comprising:

receiving an object designation input that identifies a designated object for display of a representation of information allocated to the virtual information region;

identifying at least one property of the designated object; and

determining the augmentation allocation property data to comprise data that correlates the virtual information region with the property of the designated object.

10. The method of claim 1, further comprising:

receiving information indicative of a virtual information region movement input; and

changing a location of the virtual information region to a different location in response to the virtual information region movement input.

11. The method of claim 1, further comprising changing of the environmental augmentation allocation data to designate the different location for display of a representation of information allocated to the virtual information region.

12. The method of claim 1, further comprising:

receiving a surface designation input that identifies a designated surface for display of a representation of information allocated to the virtual information region;

identifying at least one property of the designated surface; and

determining the augmentation allocation property data to comprise data that correlates the virtual information region with the property of the designated surface.

13. An apparatus comprising

at least one processor;

and

at least one memory including computer program code;

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform;

receiving, by a head mounted display, augmentation allocation property data that comprises data that correlates a virtual information region with at least one surface property;

receiving information indicative of at least one physical object that is in a physical environment proximate to the head mounted display;

determining that the physical object has a surface that conforms with the surface property;

determining environmental augmentation allocation data that comprises data that correlates the virtual information region with the surface of the physical object such that the environmental augmentation allocation data designates the surface of the physical object for display of a representation of information allocated to the virtual information region, the determination of the environmental augmentation allocation data being based, at least in part, on the determination that the physical object has the surface that conforms with the surface property;

determining that at least part of the surface of the physical object is at least partially within a field of view of the head mounted display; and

causing display of at least part of information that is allocated to the virtual information region based, at least in part, on the environmental augmentation allocation data and the determination that at least part of the surface of the physical object is at least partially within the field of view of the head mounted display.

14. The apparatus of claim 13, wherein the means for performing the method comprise at least one processor and at least one memory, the memory comprising machine-readable instructions, that when executed cause the apparatus to perform the method.

15. A non-transitory computer readable medium comprising program instructions stored thereon for performing at least the following:

receiving, by a head mounted display, augmentation allocation property data that comprises data that correlates a virtual information region with at least one surface property;

receiving information indicative of at least one physical object that is in a physical environment proximate to the head mounted display;

determining that the physical object has a surface that conforms with the surface property;

determining environmental augmentation allocation data that comprises data that correlates the virtual information region with the surface of the physical object such that the environmental augmentation allocation data designates the surface of the physical object for display of a representation of information allocated to the virtual information region, the determination of the environmental augmentation allocation data being based, at least in part, on the determination that the physical object has the surface that conforms with the surface property;

determining that at least part of the surface of the physical object is at least partially within a field of view of the head mounted display; and

causing display of at least part of information that is allocated to the virtual information region based, at least in part, on the environmental augmentation allocation data and the determination that at least part of the surface of the physical object is at least partially within the field of view of the head mounted display.