Systems and Methods for Rendering Optical Distortion Effects

• US 20190318526A1
• Filed: 04/16/2018
• Published: 10/17/2019
• Est. Priority Date: 04/16/2018
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Oculus VR, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Hunt, Warren Andrew, Kaplanyan, Anton S., Mara, Michael, Nankervis, Alexander
• Granted Patent: US 10,529,117 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 3/14 of variable focal length
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 2219/2021 Shape modification

Abstract

In one embodiment, a computing system may receive a focal surface map, which may be specified by an application. The system may determine an orientation in a 3D space based on sensor data generated by a virtual reality device. The system may generate first coordinates in the 3D space based on the determined orientation and generate second coordinates using the first coordinates and the focal surface map. Each of the first coordinates is associated with one of the second coordinates. For each of the first coordinates, the system may determine visibility of one or more objects defined within the 3D space by projecting a ray from the first coordinate through the associated second coordinate to test for intersection with the one or more objects. The system may generate an image of the one or more objected based on the determined visibility of the one or more objects.

Claims

1. A method comprising, by a computing system:

receiving a focal surface map specified by an application through an application programming interface (API);

determining an orientation in a three-dimensional (3D) space, wherein the orientation is based on sensor data generated by a virtual reality device;

generating a plurality of first coordinates in the 3D space based on the determined orientation;

generating a plurality of second coordinates in the 3D space using the plurality of first coordinates and the focal surface map, wherein each of the plurality of first coordinates is associated with one of the plurality of second coordinates;

for each of the plurality of first coordinates, determining visibility of one or more objects defined within the 3D space by projecting a ray from the first coordinate through the associated second coordinate to test for intersection with the one or more objects; and

generating an image of the one or more objected based on the determined visibility of the one or more objects.

2. The method of claim 1, wherein the focal surface map is a grid of coordinates, wherein the generating of the plurality of second coordinates comprises:

for each of the plurality of first coordinates, identifying a coordinate in the focal surface map that is mapped to the first coordinate and generating the associated second coordinate based on the identified coordinate in the focal surface map.

3. The method of claim 1, wherein the focal surface map is specified by an algorithm, wherein the generating of the plurality of second coordinates comprises:

for each of the plurality of first coordinates, generating the associated second coordinate by processing the first coordinate in accordance with the algorithm.

4. The method of claim 1, wherein the focal surface map is associated with a first primary color, wherein the generating of the image is further based on a second focal surface map associated with a second primary color and a third focal surface map associated with a third primary color.

5. The method of claim 1, wherein a smallest angle between the ray projected from one of the plurality of first coordinates and the ray projected from another one of the plurality of first coordinates is more than 90 degrees.

6. The method of claim 1, wherein the plurality of second coordinates fit on a curved surface defined by the focal surface map.

7. The method of claim 1, wherein the focal surface map is defined by a Catmull-Rom surface.

8. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to perform operations comprising:

receiving a focal surface map specified by an application through an application programming interface (API);

determining an orientation in a three-dimensional (3D) space, wherein the orientation is based on sensor data generated by a virtual reality device;

generating a plurality of first coordinates in the 3D space based on the determined orientation;

generating a plurality of second coordinates in the 3D space using the plurality of first coordinates and the focal surface map, wherein each of the plurality of first coordinates is associated with one of the plurality of second coordinates;

for each of the plurality of first coordinates, determining visibility of one or more objects defined within the 3D space by projecting a ray from the first coordinate through the associated second coordinate to test for intersection with the one or more objects; and

generating an image of the one or more objected based on the determined visibility of the one or more objects.

9. The media of claim 8, wherein the focal surface map is a grid of coordinates, wherein the generating of the plurality of second coordinates comprises:

for each of the plurality of first coordinates, identifying a coordinate in the focal surface map that is mapped to the first coordinate and generating the associated second coordinate based on the identified coordinate in the focal surface map.

10. The media of claim 8, wherein the focal surface map is specified by an algorithm, wherein the generating of the plurality of second coordinates comprises:

for each of the plurality of first coordinates, generating the associated second coordinate by processing the first coordinate in accordance with the algorithm.

11. The media of claim 8, wherein the focal surface map is associated with a first primary color, wherein the generating of the image is further based on a second focal surface map associated with a second primary color and a third focal surface map associated with a third primary color.

12. The media of claim 8, wherein a smallest angle between the ray projected from one of the plurality of first coordinates and the ray projected from another one of the plurality of first coordinates is more than 90 degrees.

13. The media of claim 8, wherein the plurality of second coordinates fit on a curved surface defined by the focal surface map.

14. The media of claim 8, wherein the focal surface map is defined by a Catmull-Rom surface.

15. A system comprising:

one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising;

receiving a focal surface map specified by an application through an application programming interface (API);

determining an orientation in a three-dimensional (3D) space, wherein the orientation is based on sensor data generated by a virtual reality device;

generating a plurality of first coordinates in the 3D space based on the determined orientation;

generating a plurality of second coordinates in the 3D space using the plurality of first coordinates and the focal surface map, wherein each of the plurality of first coordinates is associated with one of the plurality of second coordinates;

for each of the plurality of first coordinates, determining visibility of one or more objects defined within the 3D space by projecting a ray from the first coordinate through the associated second coordinate to test for intersection with the one or more objects; and

generating an image of the one or more objected based on the determined visibility of the one or more objects.

16. The system of claim 15, wherein the focal surface map is a grid of coordinates, wherein the generating of the plurality of second coordinates comprises:

for each of the plurality of first coordinates, identifying a coordinate in the focal surface map that is mapped to the first coordinate and generating the associated second coordinate based on the identified coordinate in the focal surface map.

17. The system of claim 15, wherein the focal surface map is specified by an algorithm, wherein the generating of the plurality of second coordinates comprises:

for each of the plurality of first coordinates, generating the associated second coordinate by processing the first coordinate in accordance with the algorithm.

18. The system of claim 15, wherein the focal surface map is associated with a first primary color, wherein the generating of the image is further based on a second focal surface map associated with a second primary color and a third focal surface map associated with a third primary color.

19. The system of claim 15, wherein a smallest angle between the ray projected from one of the plurality of first coordinates and the ray projected from another one of the plurality of first coordinates is more than 90 degrees.

20. The system of claim 15, wherein the plurality of second coordinates fit on a curved surface defined by the focal surface map.