Generating content using a virtual environment

• US 10,460,497 B1
• Filed: 05/11/2017
• Issued: 10/29/2019
• Est. Priority Date: 05/13/2016
• Status: Active Grant

• Current Assignee: Pixar (The Walt Disney Company)
• Original Assignee: Pixar (The Walt Disney Company)
• Inventors: Steinbach, Stephan, Bickley, Max, Minor, Joshua, Del Carmen, Ronnie
• Primary Examiner(s): Lhymn, Sarah
• Application Number:
  US15/593,068
• Time in Patent Office: 901 Days
• Field of Search: 345418, 345419, 345632, 345633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

Embodiments can generate content (e.g., a feature film, virtual reality experience) in a virtual environment (e.g., a VR environment). Specifically, they allow fast prototyping and development of a virtual reality experience by allowing virtual assets to be quickly imported into a virtual environment. The virtual assets can be used to help visualize or “storyboard” an item of content during early stages of development. In doing so, the content can be rapidly iterated upon without requiring use of more substantial assets, which can be time consuming and resource intensive.

Claims

1. A method for generating content in a virtual reality environment, the method being generated by one or more computer systems including a computer server, the method comprising:

receiving, at the computer server, set information regarding a virtual set of a three-dimensional animated scene, wherein the set information includes virtual asset information representing a first virtual asset placed in the virtual set;

receiving, at the computer server from a first user, view information regarding camera views into the virtual set, the view information being generated based on sensors associated with a first display device associated with the first user;

rendering, at the computer server, a virtual reality environment including the virtual set based on the camera views;

sending, from the computer server, the rendered virtual reality environment to the first display device associated with the first user for presenting the rendered virtual reality environment on the first display device;

sending, from the computer server at a same time as the rendered virtual reality environment is sent to the first display device, image data of the rendered virtual reality environment perceived by the first user to a computing device for presenting the rendered virtual reality environment perceived by the first user on the computing device so that the first display device and the computing device are presenting a same virtual reality environment perceived by the first user in real time;

receiving, at the computer server, input data indicating a first movement of the first virtual asset in the virtual set from the computing device;

re-rendering, at the computer server, the virtual reality environment based on the input data indicating the first movement of the first virtual asset;

sending, from the computer server, the re-rendered virtual reality environment to the first display device for presenting the first movement of the first virtual asset in the virtual reality environment to the first user; and

replaying, from the computer server, the re-rendered virtual reality environment on a second display device by replacing the first virtual asset with a virtual object corresponding to the first virtual asset in the virtual set and simulating the first movement for the virtual object.

2. The method of claim 1, wherein the computing device is a first computing device, and the method further comprises:

receiving, at the computer server, input data indicating a second movement of a second virtual asset in the virtual set from a second computing device.

3. The method of claim 1, further comprising

recording, at the computer server, the first movement of the first virtual asset in the virtual set such that the first movement is available for replay;

receiving, at the computer server, a replay request to replay the first movement of the first virtual asset in the virtual set on a display device; and

wherein the replaying is in response to the replay request.

4. The method of claim 3, further comprising receiving, at the computer server, a record request from an input device associated with the first user to record the first movement of the first virtual asset, and wherein the first movement of the first virtual asset is recorded after the record request is received.

5. The method of claim 1, wherein the computing device is a first computing device, and the set information includes virtual asset information representing a second virtual asset placed in the virtual set, the method further comprising:

presenting, from the computer server, the virtual set to a second computing device;

receiving, at the computer server, a second movement of the second virtual asset in the virtual set from the second computing device, wherein the second movement of the second virtual asset takes place when the first movement of the first virtual asset is taking place;

simulating, at the computer server, the second movement of the second virtual asset in the virtual set such that the second movement of the second virtual asset is presented in the virtual set to the first user through the first display device when the first movement of the first virtual asset is taking place; and

recording, at the computer server, the second movement of the second virtual asset in the virtual set for replay.

6. The method of claim 1, further comprising:

receiving, at the computer server, a voice input through the computing device, the voice input being associated with first movement of the first virtual asset; and

recording, at the computer server, the voice input with the first movement of the first virtual asset for replay.

7. The method of claim 6, further comprising:

receiving, at the computer server, changes in camera views as the first user navigates through the virtual set using an input device associated with the first user; and

, wherein recording the first movement of the first virtual asset includes recording the first movement of the first virtual asset with a camera view selected by the first user when the first movement of the first virtual asset takes place.

8. The method of claim 1, wherein the set information includes virtual asset information representing a second virtual asset placed in the virtual set, and the method further comprises:

presenting, from the computer server, the virtual set to the computing device;

receiving, at the computer server, a second movement of the second virtual asset in the virtual set from the computing device, wherein the second movement of the second virtual asset takes place after the first movement of the first virtual asset;

simulating, at the computer server, the second movement of the second virtual asset in the virtual set such that the second movement of the second virtual asset is presented in the virtual set to the first user through the first display device after the first movement of the first virtual asset; and

recording, at the computer server, the second movement of the second virtual asset in the virtual set for replay.

9. The method of claim 1, wherein the input data indicating the first movement of the first virtual asset in the virtual set received from the computing device includes information indicating an old position and a new position of the first virtual asset in the virtual set.

10. A system for generating content in a virtual reality environment, the system comprising a computer server configured to perform:

receiving set information regarding a virtual set of a three-dimensional animated scene, wherein the set information includes virtual asset information representing a first virtual asset placed in the virtual set;

receiving view information regarding camera views into the virtual set, the view information being generated based on sensors associated with a first display device associated with a first user;

rendering a virtual reality environment including the virtual set based on the camera views;

sending the rendered virtual reality environment to a first display device associated with a first user for presenting the rendered virtual reality environment on the first display device;

sending image data of the rendered virtual reality environment perceived by the first user to a computing device for presenting the rendered virtual reality environment perceived by the first user on the computing device so that the first display device and the computing device are presenting a same virtual reality environment perceived by the first user in real time;

receiving input data indicating a first movement of the first virtual asset in the virtual set from the computing device;

re-rendering the virtual reality environment based on the input data indicating the first movement of the first virtual asset;

sending the re-rendered virtual reality environment the first display device for presenting the first movement of the first virtual asset in the virtual reality environment to the first user; and

replaying, from the computer server, the re-rendered virtual reality environment on a second display device by replacing the first virtual asset with a virtual object corresponding to the first virtual asset in the virtual set and simulating the first movement for the virtual object.

11. The system of claim 10, wherein the computing device is a first computing device, and the computer server is further configured to perform:

receiving, at the computer server, input data indicating a second movement of a second virtual asset in the virtual set from a second computing device.

12. The system of claim 10, wherein the computer server is further configured to perform:

recording the first movement of the first virtual asset in the virtual set such that the first movement is available for replay;

receiving a replay request to replay the first movement of the first virtual asset in the virtual set on a display device; and

wherein the replaying is in response to the replay request.

13. The system of claim 12, wherein the computer server is further configured to perform:

receiving a record request from an input device associated with the first user to record the first movement of the first virtual asset, and wherein the first movement of the first virtual asset is recorded after the record request is received.

14. The system of claim 10, wherein the computing device is a first computing device, and the set information includes virtual asset information representing a second virtual asset placed in the virtual set, the computer server being further configured to perform:

presenting the virtual set to a second computing device;

receiving a second movement of the second virtual asset in the virtual set from the second computing device, wherein the second movement of the second virtual asset takes place when the first movement of the first virtual asset is taking place;

simulating the second movement of the second virtual asset in the virtual set such that the second movement of the second virtual asset is presented in the virtual set to the first user through the display device when the first movement of the first virtual asset is taking place; and

recording the second movement of the second virtual asset in the virtual set for replay.

15. The system of claim 10, wherein the computer server is further configured to perform:

receiving a voice input through the computing device, the voice input being associated with first movement of the first virtual asset; and

recording the voice input with the first movement of the first virtual asset for replay.

16. The system of claim 15, wherein the computer server is further configured to perform:

receiving changes in camera views as the first user navigates through the virtual set using an input device associated with the first user; and

, wherein recording the first movement of the virtual asset includes recording the first movement of the first virtual asset with a camera view selected by the first user when the first movement of the first virtual asset takes place.

17. The system of claim 10, wherein the set information includes virtual asset information representing a second virtual asset placed in the virtual set, and the system being further configured to perform:

presenting the virtual set to the computing device;

receiving a second movement of the second virtual asset in the virtual set from the computing device, wherein the second movement of the second virtual asset takes place after the first movement of the first virtual asset;

simulating the second movement of the second virtual asset in the virtual set such that the second movement of the second virtual asset is presented in the virtual set to the first user through the first display device after the first movement of the first virtual asset; and

recording the second movement of the second virtual asset in the virtual set for replay.

18. The system of claim 10, wherein the input data indicating the first movement of the first virtual asset in the virtual set received from the computing device includes information indicating an old position and a new position of the first virtual asset in the virtual set.