Method and Apparatus for Centering Swivel Views

• US 20150163402A1
• Filed: 12/10/2013
• Published: 06/11/2015
• Est. Priority Date: 12/10/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Frueh, Christian, Conley, Ken, Jain, Sumit
• Granted Patent: US 9,449,426 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: H04N 5/2627 for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80

Abstract

Methods and an apparatus for centering swivel views are disclosed. An example method involves a computing device identifying movement of a pixel location of a 3D object within a sequence of images. Each image of the sequence of images may correspond to a view of the 3D object from a different angular orientation. Based on the identified movement of the pixel location of the 3D object, the computing device may estimate movement parameters of at least one function that describes a location of the 3D object in an individual image. The computing device may also determine for one or more images of the sequence of images a respective modification to the image using the estimated parameters of the at least one function. And the computing device may adjust the pixel location of the 3D object within the one or more images based on the respective modification for the image.

Claims

1. A method comprising:

a computing device receiving a sequence of images of a three-dimensional (3D) object, wherein images of the sequence of images correspond to views of the 3D object from different angular orientations about the 3D object;

the computing device identifying movement of a pixel location of the 3D object within the sequence of images;

based on the identified movement of the pixel location of the 3D object within the sequence of images, the computing device estimating movement parameters of at least one function that describes a location of the 3D object in an individual image given a position of the individual image within the sequence of images;

the computing device determining for one or more images of the sequence of images a respective modification to the image using the estimated parameters of the at least one function; and

the computing device adjusting the pixel location of the 3D object within the one or more images of the sequence of images based on the respective modification for the image.

2. The method of claim 1, further comprising outputting a modified sequence of images corresponding to the sequence of images.

3. The method of claim 2, wherein outputting a modified sequence of images corresponding to the sequence of images comprises outputting a video.

4. The method of claim 1, wherein identifying the movement of the pixel location of the 3D object within the sequence of images comprises identifying for images of the sequence of images a center of the 3D object with respect to at least one axis of the image.

5. The method of claim 4, wherein the at least one axis of the image comprises a horizontal axis or a vertical axis.

6. The method of claim 4, wherein identifying for images of the sequence of images a center of the 3D object comprises:

identifying for images of the sequence of images pixels that represent the 3D object; and

determining for images of the sequence of images a center of pixel mass of the 3D object within the image based on the pixels that represent the 3D object.

7. The method of claim 1, wherein identifying the movement of the pixel location of the 3D object within the sequence of images comprises identifying for images of the sequence of images a change in the pixel location of the 3D object within the images with respect to a pixel location of the 3D object within another image.

8. The method of claim 7, wherein the other image comprises a first image of the sequence of images.

9. The method of claim 7, wherein the other image comprises an adjacent image to the image in the sequence, the method further comprising:

identifying for images of the sequence of images a location of one or more feature points of the 3D object within the images and a location of the one or more feature points of the 3D object within the adjacent image to the images; and

determining for images of the sequence of images the change in the pixel location of the 3D object within the images based on a movement of the one or more feature points between the images and the adjacent image to the image.

10. The method of claim 1, wherein the sequence of images of the 3D object comprises images of the 3D object captured as the 3D object rotates on a turntable.

11. The method of claim 1, wherein the sequence of images of the 3D object comprise images of the 3D object captured from a sequence of positions along a curved path in at least one plane perpendicular to an axis that passes through the 3D object.

12. The method of claim 1, wherein the at least one function comprises a trigonometric function that defines a location of the 3D object in an individual image with respect to an axis of the image given a position of the individual image within the sequence of images.

13. The method of claim 12, wherein the movement parameters of the at least one function comprise an amplitude and a phase of the trigonometric function.

14. The method of claim 1, wherein the at least one function comprises a horizontal movement trigonometric function and a vertical movement trigonometric function.

15. The method of claim 14, wherein the at least one function further comprises a scale function that describes a change in a size of the 3D object in an individual image given a position of the individual image within the sequence of images.

16. The method of claim 1, wherein the respective modification for each of the one or more images comprises one or any combination of a horizontal shift of the image, a vertical shift of the image, and a scaling of the image.

17. The method of claim 1, wherein adjusting the pixel location of the 3D object within the one or more images of the sequence of images based on the respective modification for the image comprises shifting pixels of the image, scaling the image, or both.

18. The method of claim 1, further comprising cropping each image of the sequence of images after adjusting the pixel location of the 3D object within the one or more images of the sequence of images based on the respective modification for the image.

19. A non-transitory computer-readable medium having stored therein instructions, that when executed by a computing device, cause the computing device to perform functions comprising:

identifying movement of a pixel location of a three-dimensional (3D) object within a sequence of images, wherein images of the sequence of images correspond to views of the 3D object from different angular orientations about the 3D object;

based on the identified movement of the pixel location of the 3D object within the sequence of images, estimating movement parameters of at least one function that describes a location of the 3D object in an individual image given a position of the individual image within the sequence of images;

determining for one or more images of the sequence of images a respective modification to the image using the estimated parameters of the at least one function; and

adjusting the pixel location of the 3D object within the one or more images of the sequence of images based on the respective modification for the image.

20. The non-transitory computer-readable medium of claim 19, wherein identifying the movement of the pixel location of the 3D object within the sequence of images comprises identifying for images of the sequence of images a center of the 3D object with respect to at least one axis of the images.

21. The non-transitory computer-readable medium of claim 19, wherein identifying the movement of the pixel location of the 3D object within the sequence of images comprises identifying for images of the sequence of images a change in the pixel location of the 3D object within the images with respect to a pixel location of the 3D object within another image.

22. The non-transitory computer-readable medium of claim 19, wherein adjusting the pixel location of the 3D object within the one or more images of the sequence of images based on the respective modification for the image comprises shifting pixels of the image, scaling the image, or both.

23. A computing device, comprising:

a processor; and

a computer-readable medium, configured to store instructions, that when executed by the processor, cause the computing device to perform functions comprising;

identifying movement of a pixel location of a three-dimensional (3D) object within a sequence of images, wherein images of the sequence of images correspond to views of the 3D object from different angular orientations about the 3D object,

based on the identified movement of the pixel location of the 3D object within the sequence of images, estimating movement parameters of at least one function that describes a location of the 3D object in an individual image given a position of the individual image within the sequence of images,

determining for one or more images of the sequence of images a respective modification to the image using the estimated parameters of the at least one function, and

adjusting the pixel location of the 3D object within the one or more images of the sequence of images based on the respective modification for the image.

24. The computing device of claim 23, wherein identifying the movement of the pixel location of the 3D object within the sequence of images comprises identifying for images of the sequence of images a center of the 3D object with respect to at least one axis of the images.

25. The computing device of claim 23, wherein identifying the movement of the location of the 3D object within the sequence of images comprises identifying for images of the sequence of images a change in the pixel location of the 3D object within the images with respect to a pixel location of the 3D object within another image.

26. The computing device of claim 23:

wherein the instructions further comprise instructions that cause the computing device to receive camera calibration information and turntable geometry information associated with the sequence of images,

wherein the movement parameters of the at least one function are estimated based on the identified movement of the pixel location of the 3D object within the sequence of images and the received camera calibration information and turntable geometry information.