System and method for acquiring virtual and augmented reality scenes by a user

• US 10,462,383 B2
• Filed: 07/03/2017
• Issued: 10/29/2019
• Est. Priority Date: 11/24/2010
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Newhouse, Benjamin Zeis, McArdle, Terrence Edward, Akbarzadeh, Amir
• Primary Examiner(s): Elisca, Pierre E
• Application Number:
  US15/640,858
• Publication Number:
  US 20170366758A1
• Time in Patent Office: 848 Days
• Field of Search: 463 31
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/2408 Monitoring of the upstream path of the transmission network, e.g. client requests monitoring data switching networks utilization H04L43/0876; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/6547 comprising parameters, e.g. for client setup
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

A preferred method of acquiring virtual or augmented reality (VAR) scenes can include at a plurality of locations of interest, providing one or more users with a predetermined pattern for image acquisition with an image capture device and for each of the one or more users, in response to a user input, acquiring at least one image at the location of interest. The method of the preferred embodiment can also include for each of the one or more users, in response to the acquisition of at least one image, providing the user with feedback to ensure a complete acquisition of the virtual or augmented reality scene; and receiving at a remote database, from each of the one or more users, one or more VAR scenes. One variation of the method of the preferred embodiment can include providing game mechanics to promote proper image acquisition and promote competition between users.

Claims

1. A method comprising:

receiving, from a first client device;

a plurality of images of a location, and

device orientation information for the plurality of images;

compositing, using the device orientation information, the plurality of images into a spherical spatial image scene by spatially organizing the plurality of images into the spherical spatial image scene based on the device orientation information; and

providing the spherical spatial image scene to a second client device, wherein the spherical spatial image scene is explorable by rotating or changing a direction of the second client device.

2. The method of claim 1, wherein compositing the plurality of images into the spherical spatial image scene comprises compositing the plurality of images surrounding a nodal point.

3. The method of claim 2, wherein:

the nodal point represents a position of a user of the first client device while the first client device captures the plurality of images; and

the device orientation information comprises a detected distance between the first client device and the user of the first client device while the first client device captures the plurality of images.

4. The method of claim 2, wherein compositing the plurality of images surrounding the nodal point comprises compositing the plurality of images that include a complete view of space surrounding the nodal point.

5. The method of claim 2, wherein the nodal point represents a position of the first client device while the first client device captures the plurality of images.

6. The method of claim 1, wherein the device orientation information for the plurality of images comprises gyroscope data from a gyroscope sensor of the first client device.

7. The method of claim 1, further comprising:

determining parallax properties associated with each of the plurality of images; and

weighting different portions of each of the plurality of images based on the parallax properties to represent different viewing angles for objects within the plurality of images.

8. A non-transitory computer readable medium storing instruction thereon that, when executed by at least one processor, cause a system to:

receive, from a first client device;

a plurality of images of a location, and

device orientation information for the plurality of images;

composite, using the device orientation information, the plurality of images into a spherical spatial image scene by spatially organizing the plurality of images into the spherical spatial image scene based on the device orientation information; and

provide the spherical spatial image scene to a second client device, wherein the spherical spatial image scene is explorable by rotating or changing a direction of the second client device.

9. The non-transitory computer readable medium of claim 8, wherein a viewable portion of the spherical spatial image scene changes according to a rotation of the second client device.

10. The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the system to composite the plurality of images into the spherical spatial image scene in part by compositing the plurality of images surrounding a nodal point.

11. The non-transitory computer readable medium of claim 10, wherein the nodal point represents a position of the first client device while the first client device captures the plurality of images.

12. The non-transitory computer readable medium of claim 10, further comprising instructions that, when executed by the at least one processor, cause the system to composite the plurality of images surrounding the nodal point in part by compositing the plurality of images that include a complete view of space surrounding the nodal point.

13. The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the system to composite the plurality of images into the spherical spatial image scene in part by stitching the plurality of images together using an ordering of the plurality of images as consecutively captured by the first client device.

14. The non-transitory computer readable medium of claim 8, wherein the device orientation information for the plurality of images comprises gyroscope data from a gyroscope sensor of the first client device.

15. A system comprising:

at least one processor; and

at least one non-transitory computer readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the system to;

receive, from a first client device;

a plurality of images of a location, and

device orientation information for the plurality of images;

composite, using the device orientation information, the plurality of images into a spherical spatial image scene by spatially organizing the plurality of images into the spherical spatial image scene based on the device orientation information; and

provide the spherical spatial image scene to a second client device, wherein the spherical spatial image scene is explorable by rotating or changing a direction of the second client device.

16. The system of claim 15, further comprising instructions that, when executed by the at least one processor, cause the system to composite the plurality of images into the spherical spatial image scene in part by compositing the plurality of images surrounding a nodal point.

17. The system of claim 16, wherein:

the nodal point represents a position of a user of the first client device while the first client device captures the plurality of images; and

the device orientation information comprises a detected distance between the first client device and the user of the first client device while the first client device captures the plurality of images.

18. The system of claim 16, further comprising instructions that, when executed by the at least one processor, cause the system to composite the plurality of images surrounding the nodal point in part by compositing the plurality of images that include a complete view of space surrounding the nodal point.

19. The system of claim 15, further comprising instructions that, when executed by the at least one processor, cause the system to composite the plurality of images into the spherical spatial image scene in part by stitching the plurality of images together using an ordering of the plurality of images as consecutively captured by the first client device.

20. The system of claim 15, further comprising instructions that, when executed by the at least one processor, cause the system to:

determine parallax properties associated with each of the plurality of images; and

weight different portions of each of the plurality of images based on the parallax properties to represent different viewing angles for objects within the plurality of images.