Analysis and manipulation of panoramic surround views

• US 10,521,954 B2
• Filed: 12/31/2018
• Issued: 12/31/2019
• Est. Priority Date: 11/12/2013
• Status: Active Grant

• Current Assignee: Fyusion, Inc. (Cox Enterprises Incorporated)
• Original Assignee: Fyusion, Inc. (Cox Enterprises Incorporated)
• Inventors: Holzer, Stefan Johannes Josef, Miller, Stephen David, Rusu, Radu Bogdan
• Primary Examiner(s): Williams, Jeffery A
• Application Number:
  US16/237,516
• Publication Number:
  US 20190156559A1
• Time in Patent Office: 365 Days
• Field of Search: 348 36
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2215/12 Shadow map, environment map
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/265 Mixing

Abstract

Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a surround view. In particular embodiments, a surround view can be generated by combining a panoramic view of an object with a panoramic view of a distant scene, such that the object panorama is placed in a foreground position relative to the distant scene panorama. Such combined panoramas can enhance the interactive and immersive viewing experience of the surround view.

Claims

1. A method comprising:

obtaining a first surround view, wherein the first surround view includes a 360 degree panoramic multi-view interactive digital media representation of a first object, wherein the first surround view is an object panorama;

obtaining a second surround view, wherein the second surround view includes a partial panoramic view of a second object; and

generating a third surround view, wherein generating the third surround view includes combining the second surround view with the first surround view, wherein the first and second surround views are obtained using different capture motions.

2. The method of claim 1, wherein the first object is a car.

3. The method of claim 1, wherein the first object is dynamic, and wherein the third surround view is a dynamic panorama.

4. The method of claim 1, wherein the second object is dynamic, and wherein the third surround view is a dynamic panorama.

5. The method of claim 1, wherein the third surround view includes dynamic content that is animated during user navigation.

6. The method of claim 1, wherein the first object is a vehicle.

7. The method of claim 1, wherein one of the objects moves less than the other object when the third surround view is navigated.

8. The method of claim 7, wherein the first object appears three-dimensional when the third surround view is navigated.

9. The method of claim 1, wherein the third surround view is displayed on a device, wherein the viewing angle of the third surround view is manipulated by tilting and rotating the device.

10. The method of claim 1, wherein the viewing angle of the third surround view is manipulated by rotating the device along an axis corresponding to a same axis of the third surround view.

11. A non-transitory computer readable medium storing instructions for executing a method, the method comprising:

obtaining a first surround view, wherein the first surround view includes a 360 degree panoramic multi-view interactive digital media representation of a first object, wherein the first surround view is an object panorama;

obtaining a second surround view, wherein the second surround view includes a partial panoramic view of a second object; and

generating a third surround view, wherein generating the third surround view includes combining the second surround view with the first surround view, wherein the first and second surround views are obtained using different capture motions.

12. The non-transitory computer readable medium of claim 11, wherein the first object is a car.

13. The non-transitory computer readable medium of claim 11, wherein the first object is dynamic, and wherein the third surround view is a dynamic panorama.

14. The non-transitory computer readable medium of claim 11, wherein the second object is dynamic, and wherein the third surround view is a dynamic panorama.

15. The non-transitory computer readable medium of claim 11, wherein the third surround view includes dynamic content that is animated during user navigation.

16. The non-transitory computer readable medium of claim 11, wherein the first object is a vehicle.

17. The non-transitory computer readable medium of claim 11, wherein one of the objects moves less than the other object when the third surround view is navigated.

18. The non-transitory computer readable medium of claim 17, wherein the first object appears three-dimensional when the third surround view is navigated.

19. A system comprising:

a processor; and

memory, wherein the memory includes instructions for;

obtaining a first surround view, wherein the first surround view includes a 360 degree panoramic multi-view interactive digital media representation of a first object, wherein the first surround view is an object panorama;

obtaining a second surround view, wherein the second surround view includes a partial panoramic view of a second object; and

generating a third surround view, wherein generating the third surround view includes combining the second surround view with the first surround view, wherein the first and second surround views are obtained using different capture motions.

20. The system of claim 19, wherein the first object is a car.