Geometrically and semanitcally aware proxy for content placement

• US 9,978,170 B2
• Filed: 05/08/2017
• Issued: 05/22/2018
• Est. Priority Date: 12/27/2011
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: Velkavrh, Urban, Aanjaneya, Mridul, Pylvninen, Timo Pekka, Grzeszczuk, Radek, Vedantham, Ramakrishna
• Primary Examiner(s): Tseng, Charles
• Application Number:
  US15/589,550
• Publication Number:
  US 20170243392A1
• Time in Patent Office: 379 Days
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/40 Hidden part removal
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An approach is provided for providing perspective-based content placement. A content placement platform processes and/or facilitates a processing of one or more models of one or more objects associated with a geographical area to cause, at least in part, a decomposition of the one or more models into one or more simplified surfaces. The content placement platform further causes, at least in part, a selection of one or more portions of the one or more simplified surfaces as one or more content placement layers based, at least in part, on one or more viewpoints, with the one or more content placement layers supporting a perspective-based rendering of one or more content items associated with the one or more objects.

Claims

1. A method comprising:

processing, by an apparatus, one or more models of one or more objects associated with a geographical area to initiate a decomposition of the one or more models into one or more simplified surfaces based on a mesh segmentation;

analyzing, by the apparatus, one or more error metrics as clusters are merged during the decomposition;

stopping, by the apparatus, the decomposition at a threshold number of the clusters based on results of the one or more error metrics;

selecting, by the apparatus, one or more non-occluded simplified surfaces with respect to various viewpoints among the one or more simplified surfaces as one or more content placement layers based, at least in part, on the threshold number of the clusters and a threshold of occlusions; and

initiating, by the apparatus, a perspective-based rendering of an image of the one or more objects augmented with one or more content items associated with the one or more objects to display an augmented reality on at least one of the content placement layers.

2. A method of claim 1, wherein the mesh segmentation is based, at least in part, on applying one or more mesh segmentation algorithms to provide one or more surface primitives, and

wherein the image is a live image, a live view finder image, or a pre-recorded panoramic image.

3. A method of claim 2, wherein the one or more surface primitives include, at least in part, one or more planes.

4. A method of claim 2, further comprising:

determining the one or more content placement layers based, at least in part, on a clustering of the one or more surface primitives, the one or more simplified surfaces, or a combination thereof into the clusters.

5. A method of claim 1, further comprising:

processing the one or more models, the one or more objects, or a combination thereof to determine one or more representational locations associated with the one or more simplified surfaces, wherein the one or more content placement layers are further selected based, at least in part, on the one or more representational locations.

6. A method of claim 1, further comprising:

processing the one or more simplified surfaces with respect to the various viewpoints to determine visibility statistics with respect to the one or more simplified surfaces, wherein the visibility statistics rank visibility of the one or more simplified surfaces based on the various viewpoints.

7. A method of claim 6, wherein the one or more content placement layers are further selected based, at least in part, on the visibility statistics of the various viewpoints of the one or more simplified surfaces.

8. A method of claim 1, further comprising:

processing the one or more simplified surfaces with respect to the various viewpoints per model to determine the one or more non-occluded simplified surfaces of each respective model, wherein the threshold of occlusions is specified such that every model is associated with at least one content placement layer.

9. A method of claim 8, further comprising:

determining one or more occluding objects associated with the one or more non-occluded simplified surfaces with respect to the various viewpoints;

initiating a segmentation of one or more of the one or more simplified surfaces into one or more segmented surfaces based, at least in part, on the one or more occluding objects,

wherein the one or more content placement layers are further selected based, at least in part, on the one or more segmented surfaces.

10. A method of claim 1, wherein the one or more content items rendered within the one or more content placement layers maintain the perspectives of the one or more content placement layers.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

process one or more models of one or more objects associated with a geographical area to initiate a decomposition of the one or more models into one or more simplified surfaces based on a mesh segmentation;

analyze one or more error metrics as clusters are merged during the decomposition;

stop the decomposition at a threshold number of the clusters based on results of the one or more error metrics;

select one or more non-occluded simplified surfaces with respect to various viewpoints among the one or more simplified surfaces as one or more content placement layers based, at least in part, on the threshold number of the clusters and a threshold of occlusions; and

initiate a perspective-based rendering of an image of the one or more objects augmented with one or more content items associated with the one or more objects to display an augmented reality on at least one of the content placement layers.

12. An apparatus of claim 11, wherein the mesh segmentation is based, at least in part, on applying one or more mesh segmentation algorithms to provide one or more surface primitives, and wherein the image is a live image, a live view finder image, or a pre-recorded panoramic image.

13. An apparatus of claim 12, wherein the one or more surface primitives include, at least in part, one or more planes.

14. An apparatus of claim 12, wherein the apparatus is further caused to:

determine the one or more content placement layers based, at least in part, on a clustering of the one or more surface primitives, the one or more simplified surfaces, or a combination thereof into the clusters.

15. An apparatus of claim 11, wherein the apparatus is further caused to:

process the one or more models, the one or more objects, or a combination thereof to determine one or more representational locations associated with the one or more simplified surfaces, wherein the one or more content placement layers are further selected based, at least in part, on the one or more representational locations.

16. An apparatus of claim 11, wherein the apparatus is further caused to:

process the one or more simplified surfaces with respect to the various viewpoints to determine visibility statistics with respect to the one or more simplified surfaces, wherein the visibility statistics rank visibility of the one or more simplified surfaces based on the various viewpoints.

17. An apparatus of claim 16, wherein the one or more content placement layers are further selected based, at least in part, on the visibility statistics of the various viewpoints of the one or more simplified surfaces.

18. An apparatus of claim 11, wherein the apparatus is further caused to:

process the one or more simplified surfaces with respect to the various viewpoints per model to determine the one or more non-occluded simplified surfaces of each respective model, wherein the threshold of occlusions is specified such that every model is associated with at least one content placement layer.

19. An apparatus of claim 18, wherein the apparatus is further caused to:

determine one or more occluding objects associated with the one or more non-occluded simplified surfaces with respect to the various viewpoints;

initiate a segmentation of one or more of the one or more simplified surfaces into one or more segmented surfaces based, at least in part, on the one or more occluding objects, wherein the one or more content placement layers are further selected based, at least in part, on the one or more segmented surfaces.

20. An apparatus of claim 11, wherein the one or more content items rendered within the one or more content placement layers maintain the perspectives of the one or more content placement layers.