PHOTOREALISTIC SCENE GENERATION SYSTEM AND METHOD

• US 20190066389A1
• Filed: 08/28/2018
• Published: 02/28/2019
• Est. Priority Date: 08/29/2017
• Status: Active Grant

• Current Assignee: Target Brands Inc. (Target Corporation)
• Original Assignee: Target Brands Inc. (Target Corporation)
• Inventors: JACOBSON, Tammy, MIRALLES, Alessandro, WHITACRE, Michael
• Granted Patent: US 10,643,399 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

A photorealistic scene generation system includes a virtual reality (VR) headset, a VR controller, and a VR processor. The VR headset visually presents a user with a 3D virtual environment. The VR controller receives input from the user. The VR processor communicates with the VR headset and the VR controller and instructs the VR headset to display the 3D virtual environment having an interactive vantage point based on movement of the user based on positional data received from the VR headset and instructs the VR headset to display a source zone superimposed over the 3D virtual environment. The source zone provides visual representations of physical items previously selected for use in styling the 3D virtual environment. The VR processor is programmed to direct movement of one of the visual representations from the source zone and to move the selected one of the visual representations into the 3D virtual environment.

Claims

1. A photorealistic scene generation system comprising:

a virtual reality (VR) headset configured to visually present a user with a three-dimensional (3D) virtual environment;

a VR controller separate from the VR headset and configured to receive input from the user during user interaction with the 3D virtual environment as viewed through the VR headset; and

a VR processor in communication with the VR headset and the VR controller, the VR processor being programmed to;

instruct the VR headset to display the 3D virtual environment having an interactive vantage point based on movement of the user based on positional data received from the VR headset,

instruct the VR headset to display a source zone superimposed over the 3D virtual environment to the user, wherein the source zone provides visual representations of physical items previously selected for use in styling the 3D virtual environment,

receive input from the VR controller to select and direct movement of one of the visual representations from the source zone and to move the selected one of the visual representations into the 3D virtual environment,

convert the selected one of the visual representations into a 3D model of an item corresponding to the selected one of the visual representations as the visual representation is moved from the source zone into the 3D virtual environment, and

in real time, adjust the orientation and placement of the 3D model within the 3D virtual environment to have an orientation and positional placement as directed by input from the VR controller.

2. The photorealistic scene generation system of claim 1, wherein the processor is configured to repeat receiving input from the VR controller, converting the selected one of the visual representations into the 3D model, and adjusting the orientation and placement of the 3D model for a plurality of visual representations of items previously selected for use in styling the 3D virtual environment and a plurality of 3D models corresponding to the items previously selected for use in styling the 3D virtual environment.

3. The photorealistic scene generation system of claim 1, wherein the processor is configured to adjust the orientation and placement of the 3D model as moved independently from other 3D models placed within the 3D virtual environment in real time with receiving associated input from the VR controller.

4. The photorealistic scene generation system of claim 1, wherein the processor is programmed to convert the selected one of the visual representations in a manner including automatically adjusting the scale of the 3D model, as compared to the selected one of the visual representations, to match the scale of the 3D virtual environment.

5. The photorealistic scene generation system of claim 1, further comprising:

a non-transitory computer readable storage medium for storing a photorealistic 3D model and a chunk 3D model for each one of a plurality of physical items, wherein the chunk 3D model is a less detailed representation of a corresponding one of the plurality of physical items than the photorealistic 3D model corresponding to a same one of the plurality of physical items, and the 3D model is the chunk 3D model.

6. The photorealistic scene generation system of claim 5, wherein:

the VR processor is further programmed to output a chunk scene from the 3D virtual environment including the chunk 3D model and additional chunk 3D models, and

the photorealistic scene generation system further comprises a conversion processor, in communication with the non-transitory computer readable storage medium, programmed to convert the chunk scene into a more nearly photorealistic scene by replacing individual ones of the chunk 3D model and the additional chunk 3D models with ones of the plurality of photorealistic 3D models from the non-transitory computer readable storage medium that corresponds to the same physical item as the chunk 3D model and the additional chunk 3D models.

7. The photorealistic scene generation system of claim 6, wherein the conversion processor is further programmed to add additional photorealistic features to photorealistic scene including one or more of light and shadow to the photorealistic scene.

8. The photorealistic scene generation system of claim 6, wherein the conversion processor is programmed to automatically convert the chunk scene into the more nearly photorealistic scene by replacing individual ones of the chunk 3D model and the additional chunk 3D models with ones of the plurality of photorealistic 3D models corresponding to the same physical item as the chunk 3D model and the additional chunk 3D models upon receipt of user input received via the VR controller indicating the chunk scene is complete.

9. The photorealistic scene generation system of claim 6, wherein the non-transitory computer readable storage medium is remotely located as compared to the VR processor.

10. The photorealistic scene generation system of claim 6, further comprising:

a digital warehouse interface providing visual representations of a plurality of physical items each having a corresponding chunk 3D model and a corresponding photorealistic 3D model; and

a selection processor in communication with non-transitory computer readable storage medium and for executing the digital warehouse interface in a manner allowing the user to select individual visual representations of physical items and to store the selected individual visual representations in a non-transitory database as a working board grouping including all of the selected individual visual representations;

wherein the visual representations of physical items previously selected for use in styling the 3D virtual environment shown in the source zone are the visual representations included in the working board grouping selected for design by the user.

11. A method of creating a photorealistic scene, the method comprising:

visually presenting a three-dimensional (3D) virtual environment to a user via a virtual reality (VR) headset as directed by a VR processor in communication with the VR headset, including presenting the 3D virtual environment having an interactive vantage point based on movement of a head of the user based on positional data received from the VR headset;

displaying a source zone to the user via the VR headset simultaneously with the 3D virtual environment as directed by the VR processor, wherein the source zone is displayed superimposed over the 3D virtual environment and provides visual representations of physical items previously selected for use in styling the 3D virtual environment;

receiving input from a VR controller to select and direct movement of one of the visual representations from the source zone and moving the selected one of the visual representations into the 3D virtual environment based on input received from the VR controller;

converting the selected one of the visual representations into a 3D model of an item corresponding to the selected one of the visual representations as the visual representation is moved from the source zone into the 3D virtual environment; and

in real time, adjusting the orientation and placement of the 3D model within the 3D virtual environment to have an orientation and positional placement as directed by input from the VR controller.

12. The method of claim 11, further comprising repeating receiving input from the VR controller, converting the selected one of the visual representations into the 3D model, and adjusting the orientation and placement of the 3D model for a plurality of visual representations of items previously selected for use in styling the 3D virtual environment and a plurality of 3D models corresponding to the items previously selected for use in styling the 3D virtual environment.

13. The method of claim 11, further comprising adjusting the orientation and placement of the 3D model as moved independently from other 3D models placed within the 3D virtual environment based on user input received via the VR controller.

14. The method of claim 11, wherein converting the selected one of the visual representations includes automatically adjusting the scale of the 3D model, as compared to the selected one of the visual representations, to match the scale of the 3D virtual environment.

15. The method of claim 11, storing a photorealistic 3D model and a chunk 3D model for each one of a plurality of physical items to a non-transitory computer readable storage medium, wherein the chunk 3D model is a less detailed representation of a corresponding one of the plurality of physical items than the photorealistic 3D model corresponding to a same one of the plurality of physical items, and the 3D model is the chunk 3D model.

16. The method of claim 15, further comprising:

outputting a chunk scene from the 3D virtual environment including the chunk 3D model and additional chunk 3D models, and

converting the chunk scene, using a processor in communication with the non-transitory computer readable storage medium, into a more nearly photorealistic scene by replacing individual ones of the chunk 3D model and the additional chunk 3D models with ones of the plurality of photorealistic 3D models from the non-transitory computer readable storage medium that corresponds to the same physical item as the chunk 3D model and the additional chunk 3D models.

17. The method of claim 16, further comprising adding additional photorealistic features to the photorealistic scene including adding one or more of light and shadow to the photorealistic scene.

18. The method of claim 16, automatically converting the chunk scene into the more nearly photorealistic scene by replacing individual ones of the chunk 3D model and the additional chunk 3D models with ones of the plurality of photorealistic 3D models corresponding to the same physical item as the chunk 3D model and the additional chunk 3D models upon receipt of user input received via the VR controller indicating the chunk scene is complete.

19. The method of claim 16, wherein the non-transitory computer readable storage medium is remotely located as compared to the VR processor.

20. The method of claim 16, further comprising:

accessing a digital warehouse interface providing visual representations of a plurality of physical items each having a corresponding chunk 3D model and a corresponding photorealistic 3D model; and

selecting individual visual representations of physical items and storing the selected individual visual representations in a non-transitory database as a working board grouping including all of the selected individual visual representations;

wherein the visual representations of physical items previously selected for use in styling the 3D virtual environment shown in the source zone are the visual representations included in the working board grouping selected for design by the user.

21. A non-transitory memory storing instructions for creating a photorealistic scene, the instructions comprising:

code for visually presenting a three-dimensional (3D) virtual environment to a user via a virtual reality (VR) headset as directed by a VR processor in communication with the VR headset, including presenting the 3D virtual environment having an interactive vantage point based on movement of a head of the user based on positional data received from the VR headset;

code for displaying a source zone to the user via the VR headset simultaneously with the 3D virtual environment as directed by the VR processor, wherein the source zone is displayed superimposed over the 3D virtual environment and provides visual representations of physical items previously selected for use in styling the 3D virtual environment;

code for receiving input from a VR controller to select and direct movement of one of the visual representations from the source zone and moving the selected one of the visual representations into the 3D virtual environment based on input received from the VR controller;

code for converting the selected one of the visual representations into a 3D model of an item corresponding to the selected one of the visual representations as the visual representation is moved from the source zone into the 3D virtual environment; and

code for adjusting, in real time, the orientation and placement of the 3D model within the 3D virtual environment to have an orientation and positional placement as directed by input from the VR controller.

22. The non-transitory memory of claim 21, wherein the instructions further comprise code for adjusting the orientation and placement of the 3D model as moved independently from other 3D models placed within the 3D virtual environment based on user input received via the VR controller.

23. The non-transitory memory of claim 21, wherein the code for converting the selected one of the visual representations includes code for automatically adjusting the scale of the 3D model, as compared to the selected one of the visual representations, to match the scale of the 3D virtual environment.

24. The non-transitory of claim 21, wherein the instructions further comprise code for storing a photorealistic 3D model and a chunk 3D model for each one of a plurality of physical items to a non-transitory computer readable database, wherein the chunk 3D model is a less detailed representation of a corresponding one of the plurality of physical items than the photorealistic 3D model corresponding to a same one of the plurality of physical items, and the 3D model is the chunk 3D model.

25. The non-transitory memory of claim 24, wherein the instructions further comprise:

code for outputting a chunk scene from the 3D virtual environment including the chunk 3D model and additional chunk 3D models, and

code for converting the chunk scene, using a processor in communication with the non-transitory computer readable database, into a more nearly photorealistic scene by replacing individual ones of the chunk 3D model and the additional chunk 3D models with ones of the plurality of photorealistic 3D models from the non-transitory computer readable database that corresponds to the same physical item as the chunk 3D model and the additional chunk 3D models.

26. The non-transitory memory of claim 25, wherein the instructions further comprise code for adding additional photorealistic features to the photorealistic scene including adding one or more of light and shadow to the photorealistic scene.

27. The non-transitory memory of claim 25, wherein the instructions further comprise code for automatically converting the chunk scene into the more nearly photorealistic scene by replacing individual ones of the chunk 3D model and the additional chunk 3D models with ones of the plurality of photorealistic 3D models corresponding to the same physical item as the chunk 3D model and the additional chunk 3D models upon receipt of user input received via the VR controller indicating the chunk scene is complete.

28. The non-transitory memory of claim 25, wherein the instructions further comprise:

code for accessing a digital warehouse interface providing visual representations of a plurality of physical items each having a corresponding chunk 3D model and a corresponding photorealistic 3D model; and

code for selecting individual visual representations of physical items and storing the selected individual visual representations in a non-transitory database as a working board grouping including all of the selected individual visual representations;

wherein the visual representations of physical items previously selected for use in styling the 3D virtual environment shown in the source zone are the visual representations included in the working board grouping selected for design by the user.