Network-based real time registered augmented reality for mobile devices
First Claim
1. A method of operating a mobile device to provide a display of supplementary information aligned with a view of a scene, the mobile device including a camera, a display, at least one processor and at least one position sensor, the mobile device being in networked communication via a mobile network with a remote server, the method comprising the steps of:
- (a) obtaining a first image from the camera;
(b) uploading at least said first image via the mobile network to the remote server together with corresponding data from the at least one position sensor;
(c) performing image processing to track image motion between said first image and subsequent images obtained from the camera, and hence determining a mapping between features of a reference image and features of a current image obtained from the camera, said reference image being selected from the group consisting of;
said first image; and
a second image for which transformation parameters were uploaded to the server corresponding to a mapping between said first image and said second image, said processing being performed by the at least one processor of the mobile device;
(d) receiving via the mobile network data indicative of a pixel location for display of supplementary information within said reference image, said receiving including receiving data indicative of a pixel location for display of supplementary information relating to an object obscured from view;
(e) employing said mapping to determine a corresponding pixel location for display of the supplementary information within the current image; and
(f) displaying the supplementary information on the display correctly aligned with the view of the scene.
1 Assignment
0 Petitions
Accused Products
Abstract
A method of operating a mobile device with a camera, a display and a position sensor to provide a display of supplementary information aligned with a view of a scene. One or more image obtained from the camera is uploaded to a remote server together with corresponding data from the position sensor. Image processing is then performed to track image motion between that image and subsequent images obtained from the camera, determining a mapping between the uploaded image and a current image. Data is then received via the network indicative of a pixel location for display of supplementary information within the reference image. The mapping is used to determine a corresponding pixel location for display of the supplementary information within the current image, and the supplementary information is displayed on the display correctly aligned with the view of the scene.
60 Citations
19 Claims
-
1. A method of operating a mobile device to provide a display of supplementary information aligned with a view of a scene, the mobile device including a camera, a display, at least one processor and at least one position sensor, the mobile device being in networked communication via a mobile network with a remote server, the method comprising the steps of:
-
(a) obtaining a first image from the camera; (b) uploading at least said first image via the mobile network to the remote server together with corresponding data from the at least one position sensor; (c) performing image processing to track image motion between said first image and subsequent images obtained from the camera, and hence determining a mapping between features of a reference image and features of a current image obtained from the camera, said reference image being selected from the group consisting of;
said first image; and
a second image for which transformation parameters were uploaded to the server corresponding to a mapping between said first image and said second image, said processing being performed by the at least one processor of the mobile device;(d) receiving via the mobile network data indicative of a pixel location for display of supplementary information within said reference image, said receiving including receiving data indicative of a pixel location for display of supplementary information relating to an object obscured from view; (e) employing said mapping to determine a corresponding pixel location for display of the supplementary information within the current image; and (f) displaying the supplementary information on the display correctly aligned with the view of the scene. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A method of operating a mobile device to provide a display of supplementary information aligned with a view of a scene, the mobile device including a camera, a display, at least one processor and at least one position sensor, the mobile device being in networked communication via a mobile network with a remote server, the method comprising the steps of:
-
(a) obtaining a first image from the camera; (b) uploading at least said first image via the mobile network to the remote server together with corresponding data from the at least one position sensor; (c) performing image processing to track image motion between said first image and subsequent images obtained from the camera, and hence determining a mapping between features of a reference image and features of a current image obtained from the camera, said reference image being selected from the group consisting of;
said first image; and
a second image for which transformation parameters were uploaded to the server corresponding to a mapping between said first image and said second image, said processing being performed by the at least one processor of the mobile device;(d) receiving via the mobile network data indicative of a pixel location for display of supplementary information within said reference image; (e) employing said mapping to determine a corresponding pixel location for display of the supplementary information within the current image; (f) displaying the supplementary information on the display correctly aligned with the view of the scene; (g) evaluating a tracking reliability indicator based upon at least one parameter selected from the group consisting of;
elapsed time since sampling of said first image;
degree of overlap between the current image and said first image; and
number of tracked features in common between the current image and said first image; and(h) when said tracking reliability indicator passes a threshold, uploading a new reference image obtained from the camera to the server. - View Dependent Claims (17, 18, 19)
-
Specification