NETWORK-BASED REAL TIME REGISTERED AUGMENTED REALITY FOR MOBILE DEVICES
First Claim
1. A method of operating a mobile device to provide a display of supplementary information aligned with a view of a scene, the mobile device including a camera, a display and at least one position sensor, the mobile device being in networked communication to a remote server, the method comprising the steps of:
- (a) obtaining a first image from the camera;
(b) uploading at least said first image to the remote server together with corresponding data from the at least one position sensor;
(c) performing image processing to track image motion between said first image and subsequent images obtained from the camera, and hence determining a mapping between features of a reference image and features of a current image obtained from the camera, said reference image being selected from the group consisting of;
said first image; and
a second image for which transformation parameters were uploaded to the server corresponding to a mapping between said first image and said second image;
(d) receiving via the network data indicative of a pixel location for display of supplementary information within said reference image;
(e) employing said mapping to determine a corresponding pixel location for display of the supplementary information within the current image; and
(f) displaying the supplementary information on the display correctly aligned with the view of the scene.
1 Assignment
0 Petitions
Accused Products
Abstract
A method of operating a mobile device with a camera, a display and a position sensor to provide a display of supplementary information aligned with a view of a scene. One or more image obtained from the camera is uploaded to a remote server together with corresponding data from the position sensor. Image processing is then performed to track image motion between that image and subsequent images obtained from the camera, determining a mapping between the uploaded image and a current image. Data is then received via the network indicative of a pixel location for display of supplementary information within the reference image. The mapping is used to determine a corresponding pixel location for display of the supplementary information within the current image, and the supplementary information is displayed on the display correctly aligned with the view of the scene.
-
Citations
19 Claims
-
1. A method of operating a mobile device to provide a display of supplementary information aligned with a view of a scene, the mobile device including a camera, a display and at least one position sensor, the mobile device being in networked communication to a remote server, the method comprising the steps of:
-
(a) obtaining a first image from the camera; (b) uploading at least said first image to the remote server together with corresponding data from the at least one position sensor; (c) performing image processing to track image motion between said first image and subsequent images obtained from the camera, and hence determining a mapping between features of a reference image and features of a current image obtained from the camera, said reference image being selected from the group consisting of;
said first image; and
a second image for which transformation parameters were uploaded to the server corresponding to a mapping between said first image and said second image;(d) receiving via the network data indicative of a pixel location for display of supplementary information within said reference image; (e) employing said mapping to determine a corresponding pixel location for display of the supplementary information within the current image; and (f) displaying the supplementary information on the display correctly aligned with the view of the scene. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A method for supporting display on a mobile device of supplementary information aligned with a view of a scene, the method comprising the steps of:
-
(a) receiving from the mobile device a first image obtained from a camera of the mobile device together with data indicative of a position of the mobile device when said first image was obtained; (b) generating an estimate of a field of view of the camera of the mobile device corresponding to said first image; (c) retrieving from a database at least part of a geographically anchored three-dimensional model corresponding to said estimate of said field of view; (d) co-processing said first image with said at least part of the three-dimensional model to derive a registration mapping for mapping pixels of said first image to said three-dimensional model; (e) determining a pixel location within said first image for display of at least one item of supplementary information within said first image; and (f) transmitting to the mobile device data indicative of said pixel location for display of the supplementary information. - View Dependent Claims (18, 19)
-
Specification