THREE-DIMENSIONAL MODEL CREATION AND RENDERING WITH IMPROVED VIRTUAL REALITY EXPERIENCE

• US 20170287199A1
• Filed: 01/25/2017
• Published: 10/05/2017
• Est. Priority Date: 03/31/2016
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: Umbra Software Oy (Amazon.com, Inc.)
• Inventors: MAKINEN, Otso, HATALA, Antti, SARANSAARI, Hannu, MUURIMAKI, Jarno, BUSHNAIEF, Jasin, MUSZYNSKI, Johann, PULKKI, Mikko, JABA, Niilo, LAULAJAINEN, Otto, AIJALA, Turkka, TRUONG, Vinh
• Granted Patent: US 10,672,189 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 13/63 by the player, e.g. authoring using a level editor
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/005 Tree description, e.g. octree, quadtree
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/04 : involving 3D image data
• CPC Class Codes: G06T 2200/16 : involving adaptation to the client's capabilities
• CPC Class Codes: G06T 2210/36 : Level of detail
• CPC Class Codes: G06T 2210/61 : Scene description
• CPC Class Codes: G06T 3/4092 : Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: G09G 2350/00 : Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/022 : Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2370/16 : Use of wireless transmission of display information
• CPC Class Codes: G09G 5/006 : Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 21/4312 : involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/47 : End-user applications

Abstract

A cloud network server system, a method, and a software program product for compiling and presenting a three-dimensional (3D) model are provided. An end 3D model is composed from at least two pre-existing 3D models stored in the cloud network server system by combining the pre-existing 3D models. The end 3D model is partitioned into smaller cells. The system and method allow a drawing user to view and draw the end 3D model for example of a computer game, via a drawing user terminal computer. Based on a virtual location of the drawing user in the end 3D model, parts of at least one version of the end 3D model are rendered to the drawing user. The system and method render a more lifelike virtual reality gaming experience with substantially lesser time lag, lesser memory footprint requirement, and lesser production effort.

Claims

1. A method for compiling and presenting a three-dimensional graphical computer model (1140), comprises at least one distributed cloud network server system (200) comprising a plurality of servers, and at least one user terminal computer (1100), characterised in that,

an end three-dimensional graphical computer model (1140) is composed on said cloud network server system (200) from at least two pre-existing 3D computer models (1130) stored in said cloud network server system (200) by combining said at least two pre-existing 3D computer models (1130), the end three-dimensional model (1140) being stored in the data storage (1120) on the cloud network server system (200),

the end three-dimensional graphical computer model (1140) is partitioned into smaller cells,

a drawing user connects to the cloud network (320) with the terminal computer (1100) installed with an application (1110) or a browser configured for viewing and drawing into the end three-dimensional graphical computer model (1140) from said cloud server network (320),

based on the virtual location of the user within the three-dimensional graphical computer model (1140), only the parts of the three-dimensional model (1140) visible from said virtual location are displayed.

2. A method as claimed in claim 1, characterised in that,

a viewing user connects to the cloud network (320) with a terminal computer (1150) installed with an application (1160) configured for viewing the three-dimensional graphical computer model (1140) from said cloud server network (320),

based on the virtual location of the user within the three-dimensional graphical computer model (1140), only the parts of the three-dimensional model (1140) visible from said virtual location are displayed.

3. A method as claimed in claim 1, characterised in that, additional non-spatial information is included with the spatial and visibility information of the three-dimensional model (1140) during the drawing stage.

4. A method as claimed in claim 3, characterised in that, said additional non-spatial information included with the spatial information of the three-dimensional model (1140) comprises any of the following:

triangle groups, reflectance and/or refractance attributes of light at a particular location in the three-dimensional model (1140), reflectance of sound at a particular location in the three-dimensional model (1140), smell, temperature, wind, quantity of natural light at a certain time and/or air speed.

5. A method as claimed in claim 1, characterised in that, the three-dimensional graphical computer model is a voxel grid with portal graphs while being processed to the end three-dimensional computer model or as the end three-dimensional computer model (1140).

6. A method as claimed in claim 1, characterised in that, the user terminal computer (1150) is a mobile terminal computer, such as a tablet and/or mobile phone and the viewing application (1160) is a virtual reality software application.

7. A method as claimed in claim 1, characterised in that, the information structure of the three-dimensional computer model (1140) in the model drawn and uploaded to cloud server network (320) can be transferred to and preserved in multiple different user applications (1110 and 1160) using said three-dimensional graphical model (1140).

8. A method as claimed in claim 1, characterised in that, the drawing terminal computer (1100) is a mobile terminal computer, such as a tablet and/or mobile phone or a desktop computer and the drawing takes place via an application (1110) or via a web browser.

9. A cloud server network system (200) for compiling and presenting a three-dimensional graphical computer model comprising a plurality of servers (321, 322, 323) and at least one user terminal computer (1100), characterised in that,

an end three-dimensional graphical computer model (1140) is configured to be composed on said cloud network server system (200) from at least two pre-existing 3D computer models (1130) stored in said cloud network server system (200) by combining said at least two pre-existing 3D computer models (1130), the end three-dimensional model (1140) being stored in a data storage (1120) on the cloud network server system (200),

the end three-dimensional graphical computer model (1140) is configured to be partitioned into smaller cells,

a drawing user is configured to connect to the cloud server network (320) with the terminal computer (1100) installed with an application (1110) or a browser configured for viewing and drawing into the end three-dimensional graphical computer model (1140) from said cloud server network (320),

based on a virtual location of the user within the three-dimensional graphical computer model (1140), only the parts of the three-dimensional model (1140) visible from said virtual location are configured to be displayed.

10. A cloud server network system (200) as claimed in claim 9, characterised in that,

a viewing user is configured to connect to the cloud server network (320) with a terminal computer (1150) installed with an application (1160) configured for viewing the three-dimensional graphical computer model (1140) from said cloud server network (320),

based on a virtual location of the viewing user within the three-dimensional graphical computer model (1140), only the parts of the three-dimensional model (1140) visible from said virtual location are configured to be displayed.

11. A cloud server network system (200) as claimed in claim 9, characterised in that, additional non-spatial information is configured to be included with the spatial and visibility information of the three-dimensional model (1140) during the drawing stage.

12. A cloud server network system (200) as claimed in claim 11, characterised in that, said additional non-spatial information included with the spatial information of the three-dimensional model (1140) is configured to comprise any of the following:

triangle groups, reflectance and/or refractance attributes of light at a particular location in the three-dimensional model (1140), reflectance of sound at a particular location in the three-dimensional model (1140), smell, temperature, wind, quantity of natural light at a certain time and/or air speed.

13. A cloud server network system (200) as claimed in claim 9, characterised in that, the three-dimensional graphical computer model is configured as a voxel grid with portal graphs while being processed to the end three-dimensional computer model or as the end three-dimensional computer model (1140).

14. A cloud server network system (200) as claimed in claim 9, characterised in that, the user terminal computer (1150) is configured to be a mobile terminal computer, such as a tablet and/or a mobile phone and the viewing application (1160) is a virtual reality software application.

15. A cloud server network system (200) as claimed in claim 9, characterised in that, the information structure of the three-dimensional computer model (1140) in the model drawn and uploaded to the cloud server network (320) is configured to be transferred to and preserved in multiple different user applications (1110 and 1160) using said three-dimensional graphical model (1140).

16. A cloud server network system (200) as claimed in claim 9, characterised in that, the drawing terminal computer (1100) is configured to be a mobile terminal computer, such as a tablet and/or a mobile phone or a desktop computer and the drawing takes place via an application (1110) or via a web browser.

17. A software program product stored in a computer readable medium for compiling and presenting a three-dimensional graphical computer model (1140) is executable in at least one distributed cloud network server system (200) comprising a plurality of servers (321, 322, 323) and at least one user terminal computer (210), characterised in that,

an end three-dimensional graphical computer model (1140) is configured to be composed on said cloud network server system (200) from at least two pre-existing 3D computer models (1130) stored in said cloud network server system (200) by combining said at least two pre-existing 3D computer models (1130), the end three-dimensional model (1140) being stored in a data storage (1120) on the cloud network server system (200),

the end three-dimensional graphical computer model (1140) is configured to be partitioned into smaller cells,

a drawing user connects to the cloud server network (320) with the terminal computer (1100) installed with an application (1110) or a browser configured for viewing and drawing into the end three-dimensional graphical computer model (1140) from said cloud server network (320),

based on a virtual location of the user within the three-dimensional graphical computer model (1140), only the parts of the end three-dimensional model (1140) visible from said virtual location are displayed.

18. A software program as claimed in claim 17, characterised in that,

a viewing user connects to the cloud server network (320) with a terminal computer (1150) installed with an application (1160) configured for viewing the three-dimensional graphical computer model (1140) from said cloud server network (320),

based on a virtual location of the viewing user within the three-dimensional graphical computer model (1140), only the parts of the three-dimensional model (1140) visible from said virtual location are displayed.

19. A software program as claimed in claim 17, characterised in that, additional non-spatial information is included with the spatial and visibility information of the three-dimensional model (1140) during the drawing stage.

20. A software program as claimed in claim 19, characterised in that, said additional non-spatial information included with the spatial information of the three-dimensional model (1140) comprises any of the following:

triangle groups, reflectance and/or refractance attributes of light at a particular location in the three-dimensional model (1140), reflectance of sound at a particular location in the three-dimensional model (1140), smell, temperature, wind, quantity of natural light at a certain time and/or air speed.

21. A software program as claimed in claim 17, characterised in that, the three-dimensional graphical computer model is a voxel grid with portal graphs while being processed to the end three-dimensional computer model or as the end three-dimensional computer model (1140).

22. A software program as claimed in claim 17, characterised in that, the user terminal computer (1150) is a mobile terminal computer, such as a tablet and/or a mobile phone and the viewing application (1160) is a virtual reality software application.

23. A software program as claimed in claim 17, characterised in that, the information structure of the three-dimensional computer model (1140) in the model drawn and uploaded to the cloud server network (320) can be transferred to and preserved in multiple different user applications (1110 and 1160) using said three-dimensional graphical model (1140).

24. A software program as claimed in claim 17, characterised in that, the drawing terminal computer (1100) is a mobile terminal computer, such as a tablet and/or a mobile phone or a desktop computer and the drawing takes place via an application (1110) or via a web browser.