Methods and systems for generating and providing virtual reality data that accounts for level of detail

• US 10,347,037 B2
• Filed: 10/31/2017
• Issued: 07/09/2019
• Est. Priority Date: 05/31/2017
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Castaneda, Oliver S., Lodato, Michael
• Primary Examiner(s): Mushambo, Martin
• Application Number:
  US15/799,731
• Publication Number:
  US 20180350135A1
• Time in Patent Office: 616 Days
• Field of Search: None
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/22 Cropping
• CPC Class Codes: G06T 2210/36 Level of detail
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

An exemplary virtual reality provider system accesses surface data representative of a virtual three-dimensional (“3D”) space of a virtual scene. Based on the accessed surface data, the system orthographically projects a respective plurality of adjacent surface data slices of the virtual 3D space along each of three orthogonal axes in a coordinate system associated with the virtual 3D space. The system generates virtual reality data that represents the virtual 3D space and accounts for level of detail of the surfaces included within the virtual 3D space with respect to a particular vantage point. The system provides this virtual reality data to a media player device associated with the particular vantage point for processing by the media player device to present, to a user, virtual reality content that is based on the virtual 3D space and is tailored to the particular vantage point.

Claims

1. A method comprising:

accessing, by a virtual reality provider system, surface data representative of a virtual three-dimensional (“

3D”

) space of a virtual scene, the surface data including color and depth data for surfaces included within the virtual 3D space;

orthographically projecting, by the virtual reality provider system based on the accessed surface data, a respective plurality of adjacent surface data slices of the virtual 3D space along each of three orthogonal axes in a coordinate system associated with the virtual 3D space;

generating, by the virtual reality provider system, virtual reality data that represents the virtual 3D space and accounts for level of detail of the surfaces included within the virtual 3D space with respect to a particular vantage point, the generating of the virtual reality data comprising applying an image transform operation to a surface data slice in one of the respective pluralities of adjacent surface data slices along the three orthogonal axes; and

providing, by the virtual reality provider system to a media player device associated with the particular vantage point, the virtual reality data that represents the virtual 3D space for processing by the media player device to present, to a user of the media player device, virtual reality content that is based on the virtual 3D space and is tailored to the particular vantage point.

2. The method of claim 1, wherein the orthographically projected pluralities of adjacent surface data slices of the virtual 3D space along each of the three orthogonal axes comprise:

a plurality of adjacent surface data slices orthographically projected in a positive direction along an x-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a negative direction along the x-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a positive direction along a y-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a negative direction along the y-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a positive direction along a z-axis of the coordinate system; and

a plurality of adjacent surface data slices orthographically projected in a negative direction along the z-axis of the coordinate system.

3. The method of claim 1, wherein the image transform operation applied to the surface data slice comprises at least one of:

a cropping image transform operation in which two areas of the surface data slice are differentiated so as to be processed separately;

a down-sampling image transform operation in which a quality level of the surface data slice is reduced so that the surface data slice is represented using less data than an amount of data used to represent the surface data slice at an original quality level; and

a layering image transform operation in which the surface data slice is combined with another surface data slice such that the surface data slice and the other surface data slice are represented using less data than an amount of data used to represent the surface data slice and the other surface data slice separately.

4. The method of claim 1, wherein the generating of the virtual reality data that represents the virtual 3D space and accounts for the level of detail of the surfaces further comprises:

determining a target level of detail for the surface data slice based on a proximity of the surface data slice to the particular vantage point;

determining, based on the target level of detail, that the surface data slice is to be transformed to account for level of detail with respect to the particular vantage point; and

selecting, based on the determining that the surface data slice is to be transformed to account for level of detail with respect to the particular vantage point, the image transform operation to be applied from a plurality of available image transform operations.

5. The method of claim 1, further comprising:

arranging, by the virtual reality provider system, each surface data slice in the pluralities of adjacent surface data slices to form a slice atlas data structure representative of the surface data slices in the pluralities of adjacent surface data slices; and

wherein the generating of the virtual reality data is performed using the slice atlas data structure representative of the surface data slices in the pluralities of adjacent surface data slices.

6. The method of claim 5, wherein:

the slice atlas data structure representative of the surface data slices in the pluralities of adjacent surface data slices includes metadata descriptive of the surface data slices arranged in the slice atlas data structure; and

the generating of the virtual reality data is performed using the slice atlas data structure in accordance with the metadata.

7. The method of claim 1, wherein the virtual scene is a virtual representation of a real-world scene and at least some of the color and depth data included in the surface data is captured by a plurality of capture devices disposed at different positions with respect to the real-world scene.

8. The method of claim 1, wherein at least some of the color and depth data included in the surface data is 3D model data representative of a computer-generated virtual object.

9. The method of claim 1, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

10. A system comprising:

at least one physical computing device that

accesses surface data representative of a virtual three-dimensional (“

3D”

) space of a virtual scene, the surface data including color and depth data for surfaces included within the virtual 3D space;

orthographically projects, based on the accessed surface data, a respective plurality of adjacent surface data slices of the virtual 3D space along each of three orthogonal axes in a coordinate system associated with the virtual 3D space;

generates virtual reality data that represents the virtual 3D space and accounts for level of detail of the surfaces included within the virtual 3D space with respect to a particular vantage point, the generation of the virtual reality data comprising applying an image transform operation to a surface data slice in one of the respective pluralities of adjacent surface data slices along the three orthogonal axes; and

provides, to a media player device associated with the particular vantage point, the virtual reality data that represents the virtual 3D space for processing by the media player device to present, to a user of the media player device, virtual reality content that is based on the virtual 3D space and is tailored to the particular vantage point.

11. The system of claim 10, wherein the orthographically projected pluralities of adjacent surface data slices of the virtual 3D space along each of the three orthogonal axes comprise:

a plurality of adjacent surface data slices orthographically projected in a positive direction along an x-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a negative direction along the x-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a positive direction along a y-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a negative direction along the y-axis of the coordinate system;

a plurality of adjacent surface data slices orthographically projected in a positive direction along a z-axis of the coordinate system; and

a plurality of adjacent surface data slices orthographically projected in a negative direction along the z-axis of the coordinate system.

12. The system of claim 10, wherein the image transform operation applied to the surface data slice comprises at least one of:

a cropping image transform operation in which two areas of the surface data slice are differentiated so as to be processed separately;

a down-sampling image transform operation in which a quality level of the surface data slice is reduced so that the surface data slice is represented using less data than an amount of data used to represent the surface data slice at an original quality level; and

a layering image transform operation in which the surface data slice is combined with another surface data slice such that the surface data slice and the other surface data slice are represented using less data than an amount of data used to represent the surface data slice and the other surface data slice separately.

13. The system of claim 10, wherein the generation of the virtual reality data that represents the virtual 3D space and accounts for the level of detail of the surfaces further comprises:

determining a target level of detail for the surface data slice based on a proximity of the surface data slice to the particular vantage point;

determining, based on the target level of detail, that the surface data slice is to be transformed to account for level of detail with respect to the particular vantage point; and

selecting, based on the determining that the surface data slice is to be transformed to account for level of detail with respect to the particular vantage point, the image transform operation to be applied from a plurality of available image transform operations.

14. The system of claim 10, wherein the at least one physical computing device further:

arranges each surface data slice in the pluralities of adjacent surface data slices to form a slice atlas data structure representative of the surface data slices in the pluralities of adjacent surface data slices; and

wherein the generation of the virtual reality data is performed using the slice atlas data structure representative of the surface data slices in the pluralities of adjacent surface data slices.

15. The system of claim 14, wherein:

the slice atlas data structure representative of the surface data slices in the pluralities of adjacent surface data slices includes metadata descriptive of the surface data slices arranged in the slice atlas data structure; and

the generation of the virtual reality data is performed using the slice atlas data structure in accordance with the metadata.

16. The system of claim 10, wherein the virtual scene is a virtual representation of a real-world scene and at least some of the color and depth data included in the surface data is captured by a plurality of capture devices disposed at different positions with respect to the real-world scene.

17. The system of claim 10, wherein at least some of the color and depth data included in the surface data is 3D model data representative of a computer-generated virtual object.

18. A system comprising:

a set of detail customization engines implemented by at least one physical computing device, each detail customization engine in the set of detail customization engines associated with a different respective vantage point within a virtual three-dimensional (“

3D”

) space of a virtual scene and configured to

access a respective plurality of adjacent surface data slices of the virtual 3D space orthographically projected in each direction along each of three orthogonal axes in a coordinate system associated with the virtual 3D space, each of the pluralities of surface data slices based on at least one of color data and depth data for surfaces included within the virtual 3D space;

generate a respective instance of virtual reality data that represents the virtual 3D space and accounts for level of detail of the surfaces included within the virtual 3D space with respect to the respective vantage point with which each detail customization engine is associated, the generation of the respective instance of virtual reality data comprising applying an image transform operation to a surface data slice in one of the pluralities of adjacent surface data slices; and

provide, to a media player device associated with the respective vantage point and included within a plurality of media player devices associated with the different respective vantage points, the respective instance of virtual reality data for processing by the media player device to present, to a user of the media player device, virtual reality content that is based on the virtual 3D space and is tailored to the respective vantage point;

wherein the detail customization engines in the set of detail customization engines concurrently perform the respective providing of the respective instances of virtual reality data for processing by different media player devices within the plurality of media player devices.

19. The system of claim 18, wherein:

each detail customization engine in the set of detail customization engines is configured to access the pluralities of adjacent surface data slices of the virtual 3D space by accessing a slice atlas data structure representative of each of the surface data slices in the pluralities of adjacent surface data slices; and

each detail customization engine in the set of detail customization engines generates the respective instance of virtual reality data using the slice atlas data structure representative of each of the surface data slices in the pluralities of adjacent surface data slices.

20. The system of claim 19, wherein:

the slice atlas data structure representative of each of the surface data slices in the pluralities of adjacent surface data slices includes metadata descriptive of the surface data slices in the slice atlas data structure; and

each detail customization engine in the set of detail customization engines generates the respective instance of virtual reality data using the slice atlas data structure in accordance with the metadata.