TECHNIQUES FOR CAPTURING AND DISPLAYING PARTIAL MOTION IN VIRTUAL OR AUGMENTED REALITY SCENES

• US 20150243085A1
• Filed: 02/20/2015
• Published: 08/27/2015
• Est. Priority Date: 02/21/2014
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Newhouse, Benjamin Zeis, McArdle, Terrence Edward
• Granted Patent: US 10,977,864 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

The present disclosure relates to techniques for capturing and displaying partial motion in VAR scenes. VAR scenes can include a plurality of images combined and oriented over any suitable geometry. Although VAR scenes may provide an immersive view of a static scene, current systems do not generally support VAR scenes that include dynamic content (e.g., content that varies over time). Embodiments of the present invention can capture, generate, and/or share VAR scenes. This immersive, yet static, view of the VAR scene lacks dynamic content (e.g., content which varies over time). Embodiments of the present invention can efficiently add dynamic content to the VAR scene, allowing VAR scenes including dynamic content to be uploaded, shared, or otherwise transmitted without prohibitive resource requirements. Dynamic content can be captured by device and combined with a preexisting or simultaneously captured VAR scene, and the dynamic content may be played back upon selection.

Claims

1. A method comprising:

capturing a first plurality of images associated with a virtual or augmented reality (VAR) scene;

capturing a second plurality of images associated with a highlight of the VAR scene;

determining differential data between the first plurality of images and the second plurality of images;

detecting location and orientation data of a viewer device;

rendering the VAR scene based on the location and orientation of the viewer device;

receiving a selection of the highlight; and

rendering the highlight within the VAR scene using the differential data.

2. The method of claim 1, wherein the VAR scene comprises a static base VAR scene and the highlight of the VAR scene which represents a portion of the VAR scene and varies with time.

3. The method of claim 1, wherein capturing a first plurality of images further comprises:

detecting location and orientation data of a capture device;

associating each image of the first plurality of images with the location and orientation data detected when that image is captured.

4. The method of claim 3, wherein capturing a second plurality of images further comprises:

receiving a selection to capture the second plurality of images with the device in a starting location and orientation; and

detecting location and orientation data of a capture device;

associating the location and orientation data of the capture device over time during capture of the second plurality of images.

5. The method of claim 4, wherein rendering the VAR scene further comprises:

assembling the first plurality of images into a spherical VAR scene based on the detected location and orientation data associated with each image of the first plurality of images; and

overlaying at least one image from the second plurality of images on the spherical VAR scene based on the location and orientation data associated with the at least one image from the second plurality of images.

6. The method of claim 5, wherein overlaying at least one image from the second plurality of images on the spherical VAR scene further comprises:

blending the at least one image from the second plurality of images with the spherical VAR scene.

7. The method of claim 6, wherein blending includes one or more of direct placement, blending using a Gaussian mask, poisson blending, and seaming.

8. A system comprising:

a capture device configured to;

capture a first plurality of images associated with a virtual or augmented reality (VAR) scene;

capture a second plurality of images associated with a highlight of the VAR scene; and

determine differential data between the first plurality of images and the second plurality of images; and

a viewer device configured to;

detect location and orientation data of the viewer device;

render the VAR scene based on the location and orientation of the viewer device;

receive a selection of the highlight; and

render the highlight within the VAR scene using the differential data.

9. The system of claim 8, wherein the VAR scene comprises a static base VAR scene and the highlight of the VAR scene which represents a portion of the VAR scene and varies with time.

10. The system of claim 8, wherein capturing a first plurality of images further comprises:

detecting location and orientation data of a capture device;

associating each image of the first plurality of images with the location and orientation data detected when that image is captured.

11. The system of claim 10, wherein capturing a second plurality of images further comprises:

receiving a selection to capture the second plurality of images with the device in a starting location and orientation; and

detecting location and orientation data of a capture device;

associating the location and orientation data of the capture device over time during capture of the second plurality of images.

12. The system of claim 11, wherein rendering the VAR scene further comprises:

assembling the first plurality of images into a spherical VAR scene based on the detected location and orientation data associated with each image of the first plurality of images; and

overlaying at least one image from the second plurality of images on the spherical VAR scene based on the location and orientation data associated with the at least one image from the second plurality of images.

13. The system of claim 13, wherein overlaying at least one image from the second plurality of images on the spherical VAR scene further comprises:

blending the at least one image from the second plurality of images with the spherical VAR scene.

14. The system of claim 13, wherein blending includes one or more of direct placement, blending using a Gaussian mask, poisson blending, and seaming.

15. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor, causes the processor to perform a method comprising:

capturing a first plurality of images associated with a virtual or augmented reality (VAR) scene;

capturing a second plurality of images associated with a highlight of the VAR scene;

determining differential data between the first plurality of images and the second plurality of images;

detecting location and orientation data of a viewer device;

rendering the VAR scene based on the location and orientation of the viewer device;

receiving a selection of the highlight; and

rendering the highlight within the VAR scene using the differential data.

16. The non-transitory computer readable storage medium of claim 15, wherein the VAR scene comprises a static base VAR scene and the highlight of the VAR scene which represents a portion of the VAR scene and varies with time.

17. The non-transitory computer readable storage medium of claim 15, wherein capturing a first plurality of images further comprises:

detecting location and orientation data of a capture device;

associating each image of the first plurality of images with the location and orientation data detected when that image is captured.

18. The non-transitory computer readable storage medium of claim 17, wherein capturing a second plurality of images further comprises:

receiving a selection to capture the second plurality of images with the device in a starting location and orientation; and

detecting location and orientation data of a capture device;

associating the location and orientation data of the capture device over time during capture of the second plurality of images.

19. The non-transitory computer readable storage medium of claim 18, wherein rendering the VAR scene further comprises:

assembling the first plurality of images into a spherical VAR scene based on the detected location and orientation data associated with each image of the first plurality of images; and

overlaying at least one image from the second plurality of images on the spherical VAR scene based on the location and orientation data associated with the at least one image from the second plurality of images.

20. The non-transitory computer readable storage medium of claim 19, wherein overlaying at least one image from the second plurality of images on the spherical VAR scene further comprises:

blending the at least one image from the second plurality of images with the spherical VAR scene, wherein blending includes one or more of direct placement, blending using a Gaussian mask, poisson blending, and seaming.