METHOD AND APPARATUS FOR AN AUGMENTED REALITY X-RAY

• US 20110287811A1
• Filed: 05/21/2010
• Published: 11/24/2011
• Est. Priority Date: 05/21/2010
• Status: Abandoned Application

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Mattila, Ville-Veikko, Cunningham, Andrew, Sandor, Christian
• Application Number:
  US12/785,170
• Publication Number:
  US 20110287811A1
• Time in Patent Office: Days
• US Class Current: 455/566
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

An approach is provided for generating an augmented reality X-Ray composite image. A visual saliency is determined of one or more features of a first image, a second image, or a combination thereof. The one or more features of the first image occlude, at least in part, one or more features of the second image. The first image and the second image are composited based, at least in part, on the visual saliency.

Claims

1. A method comprising:

determining a visual saliency of one or more features of a first image, a second image, or a combination thereof, wherein the one or more features of the first image occlude, at least in part, one or more features of the second image; and

causing, at least in part, compositing of the first image and the second image based, at least in part, on the visual saliency.

2. A method of claim 1, further comprising:

determining one or more locations in the first image and the second image where the one or more features of the first image occlude, at least in part, the one or more features of the second image; and

for the one or more locations, determining which of the respective one or more features of the first image or the second image to preserve during the compositing based, at least in part, on one or more criteria.

3. A method of claim 2, wherein preserving the respective one or more features comprises causing, at least in part, rendering of the respective one or more features as substantially opaque, and wherein not preserving the respective one or more features comprises causing, at least in part, rendering of the respective one or more features as substantially transparent.

4. A method of claim 1, further comprising:

generating a first saliency map of the respective one or more features of the first image and a second saliency map of the respective one or more features of the second image,

wherein the compositing is further based on the first saliency map, the second saliency map, or a combination thereof.

5. A method of claim 1, further comprising:

generating a edge map of the respective one or more features of the first image,

wherein the compositing is further based on the edge map.

6. A method of claim 1, wherein the visual saliency is determined at a device, the method further comprising:

determining a location of the device;

causing, at least in part, transmission of a request for the second image, based, at least in part, on the location, to a server;

receiving the second image from the server; and

receiving the first image from an image capture device.

7. A method of claim 1, further comprising:

determining a visual saliency of one or more respective features of a third image; and

causing, at least in part, compositing of the third image with the first image and the second image based, at least in part, on the visual saliency of the third image.

8. A method of claim 1, wherein the visual saliency is based, at least in part, on a color hue, a shape, a intensity, a motion, a luminosity, density, size, curvature, three dimensional depth cues, or a combination thereof.

9. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

determine a visual saliency of one or more features of a first image, a second image, or a combination thereof, wherein the one or more features of the first image occlude, at least in part, one or more features of the second image; and

cause, at least in part, compositing of the first image and the second image based, at least in part, on the visual saliency.

10. An apparatus of claim 9, wherein the apparatus is further caused to:

determine one or more locations in the first image and the second image where the one or more features of the first image occlude, at least in part, the one or more features of the second image; and

for the one or more locations, determine which of the respective one or more features of the first image or the second image to preserve during the compositing based, at least in part, on one or more criteria.

11. An apparatus of claim 10, wherein preserving the respective one or more features comprises causing, at least in part, rendering of the respective one or more features as substantially opaque, and wherein not preserving the respective one or more features comprises causing, at least in part, rendering of the respective one or more features as substantially transparent.

12. An apparatus of claim 9, wherein the apparatus is further caused to:

generate a first saliency map of the respective one or more features of the first image and a second saliency map of the respective one or more features of the second image,

wherein the compositing is further based on the first saliency map, the second saliency map, or a combination thereof.

13. An apparatus of claim 9, wherein the apparatus is further caused to:

generate a edge map of the respective one or more features of the first image,

wherein the compositing is further based on the edge map.

14. An apparatus of claim 9, wherein the apparatus is further caused to:

determine a location of the apparatus;

cause, at least in part, transmission of a request for the second image, based, at least in part, on the location, to a server;

receive the second image from the server; and

receive the first image from an image capture device.

15. An apparatus of claim 9, wherein the apparatus is further caused to:

determine a visual saliency of one or more respective features of a third image; and

cause, at least in part, compositing of the third image with the first image and the second image based, at least in part, on the visual saliency of the third image.

16. An apparatus of claim 9, wherein the visual saliency is based, at least in part, on a color hue, a shape, a intensity, a motion, a luminosity, density, size, curvature, three dimensional depth cues, or a combination thereof.

17. An apparatus of claim 9, wherein the apparatus is a mobile phone further comprising:

user interface circuitry and user interface software configured to facilitate user control of at least some functions of the mobile phone through use of a display and configured to respond to user input; and

a display and display circuitry configured to display at least a portion of a user interface of the mobile phone, the display and display circuitry configured to facilitate user control of at least some functions of the mobile phone.

18. A computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to at least perform the following steps:

determining a visual saliency of one or more features of a first image, a second image, or a combination thereof, wherein the one or more features of the first image occlude, at least in part, one or more features of the second image; and

causing, at least in part, compositing of the first image and the second image based, at least in part, on the visual saliency.

19. A computer-readable storage medium of claim 18, wherein the apparatus is caused to further perform:

determining one or more locations in the first image and the second image where the one or more features of the first image occlude, at least in part, the one or more features of the second image; and

for the one or more locations, determining which of the respective one or more features of the first image or the second image to preserve during the compositing based, at least in part, on one or more criteria.

20. A computer-readable storage medium of claim 19, wherein preserving the respective one or more features comprises causing, at least in part, rendering of the respective one or more features as substantially opaque, and wherein not preserving the respective one or more features comprises causing, at least in part, rendering of the respective one or more features as substantially transparent.

21-52. -52. (canceled)