METHOD AND APPARATUS FOR DETERMINING CAMERA LOCATION INFORMATION AND/OR CAMERA POSE INFORMATION ACCORDING TO A GLOBAL COORDINATE SYSTEM

• US 20140300775A1
• Filed: 08/30/2013
• Published: 10/09/2014
• Est. Priority Date: 04/05/2013
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Nokia Corporation
• Inventors: FAN, Lixin, Roimela, Kimmo Tapio, FU, Junsheng, YOU, Yu
• Granted Patent: US 9,699,375 B2
• Time in Patent Office: Days
• US Class Current: 348/231.3
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/95 Computational photography systems, e.g. light-field imaging systems

Abstract

An approach is provided for processing and/or facilitating a processing of one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a global coordinate system. The approach involves causing, at least in part, an association of the camera location information, the camera pose information, or a combination thereof with the one or more images as meta-data information.

Claims

1. A method comprising facilitating a processing of and/or processing (1) data and/or (2) information and/or (3) at least one signal, the (1) data and/or (2) information and/or (3) at least one signal based, at least in part, on the following:

a processing of one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a global coordinate system; and

an association of the camera location information, the camera pose information, or a combination thereof with the one or more images as meta-data information.

2. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a processing of the one or more images to determine a local point cloud of one or more objects depicted in the one or more images;

a matching of the local three-dimensional point cloud to one or more reference point clouds; and

a definition of the camera location information, the camera pose information, or a combination thereof according to the global coordinate system based, at least in part, on the matching.

3. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination of the one or more reference point clouds based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera.

4. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination of the overlay information based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

a rendering of the overlay information in the one or more images.

5. A method of claim 1, wherein the one or more images represent one or more viewfinder images of the at least one camera, and wherein (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination of popularity information associated with the camera location information, the camera pose information, or a combination thereof; and

a presentation of one or more representations of the popularity information in at least one user interface of the at least one camera.

6. A method of claim 5, wherein (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination of one or more representative images associated with the camera location, the camera pose information, or a combination thereof; and

a presentation of (a) the one or more representative images;

(b) guidance information for navigating to one or more locations, for pointing the at least one camera to one or more targets, or a combination thereof associated with the one or more representative images;

(c) one or more indicators for confirming that the at least one camera is at or near the one or more locations associated with the one or more representative images, pointing to the one or more targets, or a combination thereof;

(d) or a combination thereof.

7. A method of claim 1, wherein the one or more images are part of at least one video sequence and wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a selection of the one or more images from among one or more frames of the at least one video,

wherein the one or more images represent, at least in part, a subset of the one or more frames.

8. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination of the one or more panoramic images based, at least in part, on the camera location information, the camera pose information, or a combination thereof;

at least one determination of the field of view information for the one or more images based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

an extension of the one or more images with at least a portion of the one or more panoramic images based, at least in part, on the field of view information.

9. A method of claim 1, wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination of the one or more other images associated with other camera location information, other camera pose information, or a combination thereof based, at least in part, on the camera location information, the camera pose information, or a combination; and

an association of the one or more other images with the one or more images for interaction by at least one user during playback of the one or more images.

10. A method of claim 1, wherein the one or more other images are further determined based, at least in part, on one or more contextual criteria, and wherein one or more contextual criteria include, at least in part, one or more temporal criteria.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

process and/or facilitate a processing of one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a global coordinate system; and

cause, at least in part, an association of the camera location information, the camera pose information, or a combination thereof with the one or more images as meta-data information.

12. An apparatus of claim 11, wherein the apparatus is further caused to:

process and/or facilitate a processing of the one or more images to determine a local point cloud of one or more objects depicted in the one or more images;

cause, at least in part, a matching of the local three-dimensional point cloud to one or more reference point clouds; and

cause, at least in part, a definition of the camera location information, the camera pose information, or a combination thereof according to the global coordinate system based, at least in part, on the matching.

13. An apparatus of claim 11, wherein the apparatus is further caused to:

determine the one or more reference point clouds based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera.

14. An apparatus of claim 11, wherein the apparatus is further caused to:

determine overlay information based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

cause, at least in part, a rendering of the overlay information in the one or more images.

15. An apparatus of claim 11, wherein the one or more images represent one or more viewfinder images of the at least one camera, and wherein the apparatus is further caused to:

determine popularity information associated with the camera location information, the camera pose information, or a combination thereof; and

cause, at least in part, a presentation of one or more representations of the popularity information in at least one user interface of the at least one camera.

16. An apparatus of claim 15, wherein the apparatus is further caused to:

determine one or more representative images associated with the camera location, the camera pose information, or a combination thereof; and

cause, at least in part, a presentation of (a) the one or more representative images;

(b) guidance information for navigating to one or more locations, for pointing the at least one camera to one or more targets, or a combination thereof associated with the one or more representative images;

(c) one or more indicators for confirming that the at least one camera is at or near the one or more locations associated with the one or more representative images, pointing to the one or more targets, or a combination thereof;

(d) or a combination thereof.

17. An apparatus of claim 11 wherein the one or more images are part of at least one video sequence and wherein the apparatus is further caused to:

cause, at least in part, a selection of the one or more images from among one or more frames of the at least one video,

wherein the one or more images represent, at least in part, a subset of the one or more frames.

18. An apparatus of claim 11, wherein the apparatus is further caused to:

determine one or more panoramic images based, at least in part, on the camera location information, the camera pose information, or a combination thereof;

determine field of view information for the one or more images based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

cause, at least in part, an extension of the one or more images with at least a portion of the one or more panoramic images based, at least in part, on the field of view information.

19. An apparatus of claim 11, wherein the apparatus is further caused to:

determine one or more other images associated with other camera location information, other camera pose information, or a combination thereof based, at least in part, on the camera location information, the camera pose information, or a combination; and

cause, at least in part, an association of the one or more other images with the one or more images for interaction by at least one user during playback of the one or more images.

20. An apparatus of claim 11, wherein the one or more other images are further determined based, at least in part, on one or more contextual criteria, and wherein one or more contextual criteria include, at least in part, one or more temporal criteria.

21-52. -52. (canceled)