Online reference generation and tracking for multi-user augmented reality
First Claim
1. A method comprising:
- receiving a first image of a planar object captured by a first mobile platform from the first mobile platform;
receiving a second image of the planar object captured by a second mobile platform from the second mobile platform;
determining a first pose, comprising a position and an orientation, of the first mobile platform with respect to the planar object using the first and second images;
warping the first image of the planar object using the orientation of the first mobile platform with respect to the planar object to produce a reference image of a front view of the planar object; and
transmitting the reference image to the first and second platforms, wherein the second platform determines a second pose of the second platform with respect to the planar object based at least in part on the reference image and the second image captured by the second platform, wherein determining the first pose, comprising the position and the orientation, of the first mobile platform with respect to the planar object comprises;
extracting features from the first and second images;
computing a 2D-2D homography between the first and second images using the features;
computing 3D coordinates for the features to define a 3D plane that corresponds to a plane of the planar object;
computing a 3D-2D homography between the 3D plane and one of the first and second images using the features and the 3D plane; and
using the 3D-2D homography to determine the first pose.
1 Assignment
0 Petitions
Accused Products
Abstract
A multi-user augmented reality (AR) system operates without a previously acquired common reference by generating a reference image on the fly. The reference image is produced by capturing at least two images of a planar object and using the images to determine a pose (position and orientation) of a first mobile platform with respect to the planar object. Based on the orientation of the mobile platform, an image of the planar object, which may be one of the initial images or a subsequently captured image, is warped to produce the reference image of a front view of the planar object. The reference image may be produced by the mobile platform or by, e.g., a server. Other mobile platforms may determine their pose with respect to the planar object using the reference image to perform a multi-user augmented reality application.
-
Citations
26 Claims
-
1. A method comprising:
-
receiving a first image of a planar object captured by a first mobile platform from the first mobile platform; receiving a second image of the planar object captured by a second mobile platform from the second mobile platform; determining a first pose, comprising a position and an orientation, of the first mobile platform with respect to the planar object using the first and second images; warping the first image of the planar object using the orientation of the first mobile platform with respect to the planar object to produce a reference image of a front view of the planar object; and transmitting the reference image to the first and second platforms, wherein the second platform determines a second pose of the second platform with respect to the planar object based at least in part on the reference image and the second image captured by the second platform, wherein determining the first pose, comprising the position and the orientation, of the first mobile platform with respect to the planar object comprises; extracting features from the first and second images; computing a 2D-2D homography between the first and second images using the features; computing 3D coordinates for the features to define a 3D plane that corresponds to a plane of the planar object; computing a 3D-2D homography between the 3D plane and one of the first and second images using the features and the 3D plane; and using the 3D-2D homography to determine the first pose. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. An apparatus comprising:
-
means for receiving a first image of a planar object captured by a first mobile platform from the first mobile platform; means for receiving a second image of the planar object captured by a second mobile platform from the second mobile platform; means for determining a first pose, comprising a position and an orientation, of the first mobile platform with respect to the planar object using the first and second images; means for warping the first image of the planar object using the orientation of the first mobile platform with respect to the planar object to produce a reference image of a front view of the planar object; and means for transmitting the reference image to the first and second platforms, wherein the second platform determines a second pose of the second platform with respect to the planar object based at least in part on the reference image and the second image captured by the second platform, wherein the means for determining the first pose, comprising the position and the orientation, of the first mobile platform with respect to the planar object comprises; means for extracting features from the first and second images; means for computing a 2D-2D homography between the first and second images using the features; means for computing 3D coordinates for the features to define a 3D plane that corresponds to a plane of the planar object; means for computing a 3D-2D homography between the 3D plane and one of the first and second images using the features and the 3D plane; and means for using the 3D-2D homography to determine the first pose. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A server apparatus comprising:
-
a memory; and a processor coupled to the memory, the processor to; receive a first image of a planar object captured by a first mobile platform from the first mobile platform; receive a second image of the planar object captured by a second mobile platform from the second mobile platform; determine a first pose, comprising a position and an orientation, of the first mobile platform with respect to the planar object using the first and second images; warp the first image of the planar object using the orientation of the first mobile platform with respect to the planar object to produce a reference image of a front view of the planar object; and transmit the reference image to the first and second platforms, wherein the second platform determines a second pose of the second platform with respect to the planar object based at least in part on the reference image and the second image captured by the second platform, wherein to determine the first pose, comprising the position and the orientation, of the first mobile platform with respect to the planar object, the processor is to; extract features from the first and second images; compute a 2D-2D homography between the first and second images using the features; compute 3D coordinates for the features to define a 3D plane that corresponds to a plane of the planar object; compute a 3D-2D homography between the 3D plane and one of the first and second images using the features and the 3D plane; and use the 3D-2D homography to determine the first pose. - View Dependent Claims (16, 17, 18, 19, 20)
-
-
21. A non-transitory computer-readable medium comprising code which, when executed by a processor, causes the processor to perform a method comprising:
-
receiving a first image of a planar object captured by a first mobile platform from the first mobile platform; receiving a second image of the planar object captured by a second mobile platform from the second mobile platform; determining a first pose, comprising a position and an orientation, of the first mobile platform with respect to the planar object using the first and second images; warping the first image of the planar object using the orientation of the first mobile platform with respect to the planar object to produce a reference image of a front view of the planar object; and transmitting the reference image to the first and second platforms, wherein the second platform determines a second pose of the second platform with respect to the planar object based at least in part on the reference image and the second image captured by the second platform, wherein the code for determining the first pose, comprising the position and the orientation, of the first mobile platform with respect to the planar object comprises code for; extracting features from the first and second images; computing a 2D-2D homography between the first and second images using the features; computing 3D coordinates for the features to define a 3D plane that corresponds to a plane of the planar object; computing a 3D-2D homography between the 3D plane and one of the first and second images using the features and the 3D plane; and using the 3D-2D homography to determine the first pose. - View Dependent Claims (22, 23, 24, 25, 26)
-
Specification