NETWORK-BASED REAL TIME REGISTERED AUGMENTED REALITY FOR MOBILE DEVICES
First Claim
1. A method of operating a mobile device to provide a display of supplementary information aligned with a view of a scene, the mobile device including a camera, a display and at least one position sensor, the mobile device being in networked communication to a remote server, the method comprising the steps of:
- (a) obtaining a first image from the camera;
(b) uploading at least said first image to the remote server together with corresponding data from the at least one position sensor;
(c) performing image processing to track image motion between said first image and subsequent images obtained from the camera, and hence determining a mapping between features of a reference image and features of a current image obtained from the camera, said reference image being selected from the group consisting of;
said first image; and
a second image for which transformation parameters were uploaded to the server corresponding to a mapping between said first image and said second image;
(d) receiving via the network data indicative of a pixel location for display of supplementary information within said reference image;
(e) employing said mapping to determine a corresponding pixel location for display of the supplementary information within the current image; and
(f) displaying the supplementary information on the display correctly aligned with the view of the scene.
1 Assignment
0 Petitions
Accused Products
Abstract
A method of operating a mobile device with a camera, a display and a position sensor to provide a display of supplementary information aligned with a view of a scene. One or more image obtained from the camera is uploaded to a remote server together with corresponding data from the position sensor. Image processing is then performed to track image motion between that image and subsequent images obtained from the camera, determining a mapping between the uploaded image and a current image. Data is then received via the network indicative of a pixel location for display of supplementary information within the reference image. The mapping is used to determine a corresponding pixel location for display of the supplementary information within the current image, and the supplementary information is displayed on the display correctly aligned with the view of the scene.
126 Citations
19 Claims
-
1. A method of operating a mobile device to provide a display of supplementary information aligned with a view of a scene, the mobile device including a camera, a display and at least one position sensor, the mobile device being in networked communication to a remote server, the method comprising the steps of:
-
(a) obtaining a first image from the camera; (b) uploading at least said first image to the remote server together with corresponding data from the at least one position sensor; (c) performing image processing to track image motion between said first image and subsequent images obtained from the camera, and hence determining a mapping between features of a reference image and features of a current image obtained from the camera, said reference image being selected from the group consisting of;
said first image; and
a second image for which transformation parameters were uploaded to the server corresponding to a mapping between said first image and said second image;(d) receiving via the network data indicative of a pixel location for display of supplementary information within said reference image; (e) employing said mapping to determine a corresponding pixel location for display of the supplementary information within the current image; and (f) displaying the supplementary information on the display correctly aligned with the view of the scene. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A method for supporting display on a mobile device of supplementary information aligned with a view of a scene, the method comprising the steps of:
-
(a) receiving from the mobile device a first image obtained from a camera of the mobile device together with data indicative of a position of the mobile device when said first image was obtained; (b) generating an estimate of a field of view of the camera of the mobile device corresponding to said first image; (c) retrieving from a database at least part of a geographically anchored three-dimensional model corresponding to said estimate of said field of view; (d) co-processing said first image with said at least part of the three-dimensional model to derive a registration mapping for mapping pixels of said first image to said three-dimensional model; (e) determining a pixel location within said first image for display of at least one item of supplementary information within said first image; and (f) transmitting to the mobile device data indicative of said pixel location for display of the supplementary information. - View Dependent Claims (18, 19)
-
Specification