NAVIGATION MODEL TO RENDER CENTERED OBJECTS USING IMAGES

• US 20120314040A1
• Filed: 06/09/2011
• Published: 12/13/2012
• Est. Priority Date: 06/09/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kopf, Johannes, Stollnitz, Eric, Sinha, Sudipta, Szeliski, Rick
• Granted Patent: US 8,675,049 B2
• Time in Patent Office: Days
• US Class Current: 348/50
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/264 using the relative movement of objects in two video frames or fields
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

A 3D model of an object is rendered using centered images of the object. An algorithm executed locally or in a distributed manner calculates camera positions for the images and determines a virtual camera path based on the camera positions. The application adjusts the images to fit the plane of the virtual camera path and fills in the gaps between the images using transition renderings. To improve user experience, the application also calculates resting positions for navigation stop points using a spring system. Upon constructing the 3D model, the application can transmit the 3D model to a variety of user devices including the network connected device having a camera module that captured the images.

Claims

1. A method executed at least in part by a computing device for employing a navigation model to reconstruct an object using images, the method comprising:

receiving images of an object captured by a camera;

determining camera positions for the images;

determining a virtual camera path based on the camera positions;

determining resting positions for navigation stop points; and

constructing and transmitting a three-dimensional (3D) model of the object navigable following the virtual camera path.

2. The method of claim 1, wherein the virtual camera path has a trajectory approximating an arc around the object.

3. The method of claim 1, wherein the virtual camera path is navigable left-to-right, and right-to-left.

4. The method of claim 1, wherein the images are captured in a circular shape around the object while the camera is pointed towards the object.

5. The method of claim 1, further comprising:

associating the camera positions with orthonormal up vectors, right vectors, and front vectors.

6. The method of claim 5, further comprising:

estimating a location of the object to be a point x near optical axes of multiple cameras using a least squares computation;

defining the point x near the optical axes using x−

(p_i+t·

f_i),f_i=0, in which p_i is a camera position and f_i is the camera'"'"'s forward-facing direction;

defining a point closest to x on a ray defined by the optical axes by p_i+t·

f_i; and

determining t by

7. The method of claim 6, further comprising:

determining a point closest to several optical axes by minimizing an energy term;

8. The method of claim 6, further comprising:

computing x by solving a linear system of equations derived in a least squares sense.

9. The method of claim 1, further comprising:

smoothing a motion of the camera by attaching resting values corresponding to the camera positions to current values of the current camera position.

10. The method of claim 1, in which,

fitting a plane to the camera positions using a least squares computation;

and

embedding the virtual camera path into the plane.

11. A computing device capable of employing a navigation model to reconstruct an object using images, the computing device comprising:

a memory;

a processor coupled to the memory, the processor executing an application in conjunction with instructions stored in the memory, wherein the application is configured to;

receive the images of the object captured by a camera;

determine camera positions for the images by;

fitting a plane to the camera positions; and

determining a virtual camera path based on the camera positions by embedding the virtual camera path into the plane;

determine resting positions for navigation stop points; and

construct and transmit a three-dimensional (3D) model of the object navigable following the virtual camera path.

12. The computing device of claim 11, wherein the application is further configured to:

choosing arbitrary orthonormal vectors in the plane to define a two-dimensional (2D)

Euclidean coordinate system with a projected location of the object as an origin.

13. The computing device of claim 11, wherein the application is further configured to:

snap viewpoints using the resting positions by;

parameterizing at least one of the camera positions by an angle φ

_c; and

using a predefined distance d_c to offset a distance of the camera to the virtual camera path and a predefined height h_c to offset a height of the camera along a direction normal to the virtual camera path.

14. The computing device of claim 13, wherein the application is further configured to:

employ a spring system to smooth a motion of the camera by attaching resting values φ

_r, d_r, h_r corresponding to the camera positions to current values of a current camera position φ

_c, d_c, h_c.

15. The computing device of claim 14, wherein the spring system brings the current values to the resting values using φ

_c←

φ

_r+(φ

_c−

φ

_r)e<sup>(Δ

t·

k)</sup>.

16. The computing device of claim 15, wherein Δ

t is a time since a last update and k is a spring constant controlling a stiffness property of the spring system.

17. The computing device of claim 11, wherein the computing device is a camera-equipped smart phone.

18. A computer-readable memory device with instructions stored thereon for employing a navigation model to reconstruct an object using images, the instructions comprising:

receiving the images of the object captured by a camera;

determining camera positions for the images by;

fitting a plane to the camera positions using a least squares computation; and

determining a virtual camera path based on the camera positions by embedding the virtual camera path into the plane;

determining resting positions for navigation stop points using a spring system; and

constructing and transmitting a three-dimensional (3D) model of the object navigable around the virtual camera path.

19. The computer-readable memory device of claim 18, wherein the camera positions are further determined through one of Global Positioning Satellite (GPS) coordinates and wireless signal triangulation.

20. The computer-readable memory device of claim 18, wherein the images are captured in about 10 degree angular increments around the object.