DYNAMIC VIEWING OF WIDE ANGLE IMAGES

• US 20080144968A1
• Filed: 04/23/2007
• Published: 06/19/2008
• Est. Priority Date: 12/15/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kopf, Johannes, Uyttendaele, Matthew T., Cohen, Michael F.
• Granted Patent: US 8,224,122 B2
• Time in Patent Office: Days
• US Class Current: 382/276
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations

Abstract

A dynamic wide angle image viewing technique is presented which provides a way to view a wide-angle image while zooming between a wide angle view and a narrower angle view that employs both perspective and non-perspective projection models. In general, this involves first establishing the field of view for a view of the wide angle image that is to be displayed. The view is then rendered and displayed based on the established field of view, such that the projection transitions between a perspective projection associated with narrower angle views and a non-perspective projection (e.g., cylindrical, spherical or some other parameterization) associated with wider-angle views.

Claims

1. A computer-implemented process for dynamically viewing a wide angle image while zooming between a wide angle view and a narrower angle view, the process comprising using a computer to perform the following process actions for each view of the wide angle image to be presented to a user:

establishing a field of view desired for the view to be presented; and

rendering and displaying the view based on the established field of view, such that the projection transitions between a perspective projection associated with narrower angle views and a non-perspective projection associated with wider-angle views.

2. The process of claim 1, wherein the process action of rendering and displaying the view based on the established field of view, comprises the actions of:

rendering the view using the perspective projection whenever the established field of view is below a prescribed zoom level; and

rendering the view using the non-perspective projection whenever the established field of view is above the prescribed zoom level.

3. The process of claim 1, wherein the process action of rendering and displaying the view based on the established field of view, comprises the actions of:

rendering the view using the perspective projection whenever the established field of view is equal to or below a first prescribed zoom level;

rendering the view using the non-perspective projection whenever the established field of view is equal to or above a second, higher prescribed zoom level; and

rendering the view using a blend of the perspective and non-perspective projections whenever the established field of view falls between the first and second prescribed zoom levels.

4. The process of claim 1, wherein the process action of rendering and displaying the view based on the established field of view, comprises an action of rendering the view using a projection that is between the perspective and non-perspective projections by warping the perspective projection toward the non-perspective projection by an amount that is proportional to the established field of view, with the amount of warping being greater for wider fields of view.

5. The process of claim 1, wherein the non-perspective projection is a cylindrical projection.

6. The process of claim 1, wherein the non-perspective projection is a spherical projection.

7. A computer-implemented process for dynamically viewing a wide angle image on a display screen while zooming between a wide angle view and a narrower angle view, the process comprising using a computer to perform the following process actions:

establishing a pair of screen coordinates for each pixel or vertex location of the display screen;

establishing a field of view and viewing direction which are used to generate a view of the wide angle image that is to be displayed on the display screen;

for each pair of screen coordinates,

computing a ray defined by two angles representing the direction from a viewer through the pair of screen coordinates under consideration, wherein said view comprises a projection that is between a perspective projection and a non-perspective projection which is generated by warping the perspective projection toward the non-perspective projection by an amount that is proportional to the established field of view, with the amount of warping being greater for wider fields of view;

rendering the view of the wide angle image based on the computed rays; and

displaying the rendered view on the display screen.

8. The process of claim 7, wherein the process action of rendering the view of the wide angle image, comprises the actions of:

identifying texture coordinates in a stored mapping of the wide angle image that correspond to the computed ray;

obtaining the texture data corresponding to the identified texture coordinates from the mapping; and

rendering the portion of the view using the obtained texture data.

9. The process of claim 8, wherein the stored mapping of the wide angle image employs a projection that is not spherical, and wherein prior to performing the process action of identifying texture coordinates in the stored mapping that correspond to the computed ray, performing an action of converting the direction represented by the computed ray to the projection model employed by the stored mapping.

10. The process of claim 7, wherein the wide angle image is a virtual image, and wherein the process action of rendering the view of the wide angle image, comprises the actions of:

generating the texture data corresponding to the identified texture coordinates; and

rendering the portion of the view using the generated texture data.

11. The process of claim 7, wherein the process action of computing the ray, comprises the actions of:

computing r_c=1/(2*sin (α

)), wherein α

is a first bending parameter;

computing x_int=sin (x_screen/r_c)*r_c, wherein x_screen is one of the screen coordinates;

computing z_int=cos (x_screen/r_c)*r_c+(1−

r_c);

computing x_s=x_int−

sin (x_screen/r_c);

computing z_s=z_int−

cos (x_screen/r_c);

computing x_n=x_int−

x_s;

computing z_n=z_int−

z_s;

computing r=1/(2*sin (β

)), wherein β

is a second bending parameter;

computing y=sin (y_screen/r)*r wherein y_screen is the other of the screen coordinates;

computing x=x_s+x_n*(cos (y_screen/r)+(1−

r));

computing z=z_s+z_n*(cos (y_screen/r)+(1−

r));

computing the first angle θ

of the ray as θ

=a tan 2(x, z); and

computing the second angle φ

of the ray as φ

=a tan 2(y, sqrt(x²+z²)).

12. The process of claim 11, wherein the process action of computing the first angle θ

of the ray, comprises an action of computing the first bending parameter α

as α

=α

_max/(1+exp((zoom-mid)*scale)) wherein α

_max is arc sin (0.5)=30 degrees, zoom is the zoom level associated with the established field of view, and mid and scale are scaling parameters.

13. The process of claim 12, wherein the process action of computing the second angle φ

of the ray, comprises an action of computing the second bending parameter β

as β

=β

_max/(1+exp((zoom-mid)*scale)), wherein β

_max=α

_max.

14. The process of claim 13, wherein the scaling parameter mid ranges between about 50 and about 130 degrees, and the scaling parameter scale ranges between about 2 and about 10.

15. The process of claim 13, wherein the scaling parameter mid is set to deg2rad(100) and the scaling parameter scale is set to 6.

16. A computer-implemented process for dynamically viewing a wide angle image on a display screen while zooming between a wide angle view and a narrower angle view, the process comprising using a computer to perform the following process actions:

establishing a field of view and a central viewing direction desired for a view of the wide angle image to be displayed on the display screen;

establishing for each of a set of viewing directions of interest within the established field of view surrounding the established central viewing direction, a pair of screen coordinates for a perspective projection of a view being rendered;

establishing for each of the set of viewing directions of interest within the established field of view surrounding the established central viewing direction, a pair of screen coordinates for a non-perspective projection of a view being rendered;

for each of the set of viewing directions of interest,

identifying the pair of screen coordinates associated with the direction under consideration for the perspective projection of the view whenever the field of view is equal to or below a prescribed minimum field of view,

identifying the pair of screen coordinates associated with the direction under consideration for the non-perspective projection of the view whenever the field of view is equal to or above a prescribed maximum field of view, and

identifying the pair of screen coordinates associated with the direction under consideration for both the perspective and non-perspective projections of the view whenever the field of view is between the prescribed minimum and maximum fields of view, and computing an interpolated pair of screen coordinates based on a blending of the identified screen coordinates associated with the perspective and non-perspective projections;

rendering the view using each pair of the identified screen coordinates, or each pair of the blended screen coordinates whenever they have been computed; and

displaying the view on the display screen.

17. The process of claim 16, wherein the process action of rendering the view, comprises the actions of:

identifying a pair of texture coordinates in a stored mapping of the wide angle image that correspond to each pair of the identified screen coordinates, or to each pair of the blended screen coordinates whenever they have been computed;

obtaining the texture data corresponding to each pair of the identified texture coordinates from the mapping; and

rendering the view using the obtained texture data.

18. The process of claim 17, wherein the stored mapping of the wide angle image employs a projection that is not the same as the projection used to obtain the screen coordinate pairs, and wherein prior to performing the process action of identifying the pair of texture coordinates in the stored mapping of the wide angle image that correspond to each pair of the identified screen coordinates, or to each pair of the blended screen coordinates whenever they have been computed, performing an action of converting the screen coordinate pairs to the projection employed by the stored mapping.

19. The process of claim 16, wherein the wide angle image is a virtual image, and wherein the process action of rendering the view, comprises the actions of:

generating the texture data corresponding to each pair of the identified screen coordinates, or to each pair of the blended screen coordinates whenever they have been computed; and

rendering the view using the generated texture data.

20. The process of claim 16, wherein the process action of computing the interpolated pair of screen coordinates based on the blending of identified screen coordinates associated with the perspective and non-perspective projections, comprises the actions of:

computing a blending factor as the difference between the prescribed maximum field of view and the established field of view, divided by the difference between the prescribed maximum field of view and the prescribed minimum field of view; and

computing the interpolated pair of screen coordinates as the sum of the identified screen coordinates associated with the perspective projection multiplied by the blending factor and the identified screen coordinates associated with the non-perspective projection multiplied by one minus the blending factor.