Natural user interfaces for mobile image viewing
First Claim
1. A computer-implemented process for viewing imagery on a mobile device, comprising:
- tracking a mobile device'"'"'s orientation and position with a gyroscope;
using a front-facing camera and viewer tracker on the mobile device to track a position of a user'"'"'s face looking at a screen on the mobile device;
correcting drift of the gyroscope based on the tracked position of the user'"'"'s face to provide a drift-corrected orientation and position of the mobile device;
computing image transformations for an image, a portion of which is displayed in two dimensions on the screen of the mobile device at any one time, using the tracked mobile device'"'"'s drift-corrected orientation and position and the tracked position of the user'"'"'s face;
affecting a dynamic change of the perspective of the displayed portion of the image, comprising changing the portion of the image displayed on the screen of the mobile device based on a change in the mobile device'"'"'s orientation or position or both relative to the tracked position user'"'"'s face, wherein the perspective of the image displayed can be changed even when the user'"'"'s face is outside of the front-facing camera'"'"'s field of view.
2 Assignments
0 Petitions
Accused Products
Abstract
The mobile image viewing technique described herein provides a hands-free interface for viewing large imagery (e.g., 360 degree panoramas, parallax image sequences, and long multi-perspective panoramas) on mobile devices. The technique controls the imagery displayed on a display of a mobile device by movement of the mobile device. The technique uses sensors to track the mobile device'"'"'s orientation and position, and front facing camera to track the user'"'"'s viewing distance and viewing angle. The technique adjusts the view of a rendered imagery on the mobile device'"'"'s display according to the tracked data. In one embodiment the technique can employ a sensor fusion methodology that combines viewer tracking using a front facing camera with gyroscope data from the mobile device to produce a robust signal that defines the viewer'"'"'s 3D position relative to the display.
32 Citations
18 Claims
-
1. A computer-implemented process for viewing imagery on a mobile device, comprising:
-
tracking a mobile device'"'"'s orientation and position with a gyroscope; using a front-facing camera and viewer tracker on the mobile device to track a position of a user'"'"'s face looking at a screen on the mobile device; correcting drift of the gyroscope based on the tracked position of the user'"'"'s face to provide a drift-corrected orientation and position of the mobile device; computing image transformations for an image, a portion of which is displayed in two dimensions on the screen of the mobile device at any one time, using the tracked mobile device'"'"'s drift-corrected orientation and position and the tracked position of the user'"'"'s face; affecting a dynamic change of the perspective of the displayed portion of the image, comprising changing the portion of the image displayed on the screen of the mobile device based on a change in the mobile device'"'"'s orientation or position or both relative to the tracked position user'"'"'s face, wherein the perspective of the image displayed can be changed even when the user'"'"'s face is outside of the front-facing camera'"'"'s field of view. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computer-implemented process for viewing imagery on a mobile device, comprising:
-
tracking a mobile device'"'"'s orientation and position using a gyroscope on the mobile device; using a front-facing camera and viewer tracker on the mobile device to track a position of a user'"'"'s face looking at a screen on the mobile device; correcting drift of the gyroscope based on the tracked position of the user'"'"'s face to provide a drift-corrected orientation and position of the mobile device; using the mobile device'"'"'s drift-corrected orientation and position from the gyroscope and the position of the user'"'"'s face from the front-facing camera to determine a combined position and rate control for viewing imagery in three dimensions on the screen of the mobile device; using the combined position and rate control to compute image transformations of the imagery rendered on the screen of the mobile device to allow the dynamic display of different viewpoints in three dimensions of the rendered imagery; and dynamically displaying the image transformations of the imagery on the screen of the mobile device. - View Dependent Claims (11, 12, 13, 14)
-
-
15. A system for viewing imagery, comprising:
-
a computing device; a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to, track a mobile device'"'"'s orientation and position with a gyroscope; use a camera and viewer tracker to track a position of a face of a user viewing imagery rendered on the screen of the mobile device, correct drift of the gyroscope based on the tracked position of the face of the user to provide a drift-corrected orientation and position of the mobile device; compute image transformations of the imagery rendered on the screen of the mobile device that allow for dynamic display of different viewpoints of the rendered imagery in both depth and position based on the drift-corrected orientation and position of the mobile device even when the face of the user is outside of the field of view of the camera by moving the mobile device relative to the user'"'"'s face. - View Dependent Claims (16, 17, 18)
-
Specification