Method, system and device for efficient distribution of real time three dimensional computer modeled image scenes over a network

• US 7,388,585 B2
• Filed: 09/15/2005
• Issued: 06/17/2008
• Est. Priority Date: 09/20/2004
• Status: Active Grant

• Current Assignee: My Virtual Reality Software
• Original Assignee: My Virtual Reality Software
• Inventors: Kristiansen, Stig Ronald
• Primary Examiner(s): Chauhan; Ulka
• Assistant Examiner(s): PRENDERGAST, ROBERTA D
• Application Number:
  US11/227,384
• Publication Number:
  US 20060061584A1
• Time in Patent Office: 1,006 Days
• Field of Search: 345/428, 382/239, 382/240
• US Class Current: 345/428
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities

Abstract

The present invention is related to a method, system and device for distributing in real time data related to three dimensional computer modeled images scenes over a network. To provide a faster and better visual quality of the rendered frames on a client computer, the client computer first receives an initial file comprising a minimum of geometrical data and texture data to provide a minimum of visual quality of the displayed scene. By evaluating an importance factor for textures used in the scenes, the client computer may request the server for further data to improve the visual quality of the scene.

Claims

1. A method of displaying three dimensional computer modeled image scenes streaming from a server to a client computer over a network, wherein the thee dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the method comprising:

transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for initial display;

in the client computer, displaying an image based on the minimum geometric data and the minimum texture data;

in the client computer, initiating a further downloading of data from the server to the client computer according to an importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

2. The method according to claim 1, wherein objects which have an importance factor lower than a maximum threshold are not updated.

3. The method according to claim 1, wherein the importance factor for a current object is based at least in part on at least one of a distance from the observation point to the current object, size of the current object, size of the current object as seen from the observation point, size of at least part of the current object not seen from the observation point, viewing angle to the current object from the observation point, and a parameter related to the current object comprised in the streaming of the three dimensional modeled image scenes.

4. The method according to claim 1, wherein initiating further downloading of data comprises calculating the importance factor for texture data for all objects in one scene at a time from the streaming scenes, downloading the texture data for the object in each respective scene having the highest importance factor, and then displaying each respective scene one at a time.

5. The method according to claim 4, wherein downloading the texture data for the object with the highest importance factor comprises downloading further data for objects in the scene according to the importance factor for each object until there is no more available allocated memory in the client computer.

6. The method according to claim 5, further comprising deleting the further downloaded texture data for the objects in the scenes having an importance factor lower than a maximum threshold when there is no more available allocated memory in the client computer.

7. The method according to claim 5, further comprising calculating an importance factor based at least in part on other downloaded texture data than those being displayed.

8. The method according to claim 7, further comprising using a system parameter or a user selectable parameter to decide which texture data is to be used in the calculation of the importance factor.

9. The method according to claim 1, wherein the transferring of the initial file further comprises:

evaluating if the geometrical data and the texture data are within predefined limits, and if they are, converting the data from floating point format to integer format.

10. A computer system comprising a server and at least one client computer configured to display three dimensional computer modeled image scenes streaming from the server to the client, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the system comprising:

means for transferring an initial file after receipt of a request from the at least one client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an initial displaying;

means, in the at least one client computer, for initiating a further downloading of data from the server to the at least one client computer according to a calculated importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

11. The system according to claim 10, wherein objects which have an importance factor lower than a maximum threshold are not updated.

12. The system according to claim 10, wherein the importance factor for a current object is based at least in part on at least one of a distance from the observation point to the current object, size of the current object, size of the current object as seen from the observation point, size of at least part of the current object not seen from the observation point, viewing angle to the current object from the observation point, and a parameter related to the current object comprised in the streaming of the three dimensional modeled image scenes.

13. The computer system according to claim 10, further comprising means for initiating further downloading of data according to the calculated importance factor for texture data for all objects in one scene at a time from the streaming scenes, wherein the means for initiating further downloading is configured to download the texture data for the object in each respective scene having the highest importance factor, and to then provide displaying of each respective scene one at a time.

14. The computer system according to claim 13, wherein the means for initiating further downloading further comprises means for repeatedly downloading further data for objects in the scene according to the importance factor for each object until there is no more available allocated memory in the at least one client computer.

15. The computer system according to claim 14, further comprising means for deleting the further downloaded texture data for the objects in the scenes having an importance factor lower than a threshold when there is no more available allocated memory in the at least one client computer.

16. The computer system according to claim 14, further comprising means for calculating the importance factor based on downloaded texture data other than those texture data used for displaying.

17. The computer system according to claim 16, further comprising means for using a system parameter or a user selectable parameter to decide which texture data to be used in the calculation of the importance factor.

18. The computer system according to claim 10, further comprising:

means, in the server, for testing if the geometrical data and the texture data are within predefined limits,

means, in the server, for converting the geometrical data and the texture data from floating point format to integer format,

means, in the server, for providing an initial file for a scene comprising the converted data,

means, in the at least one client computer, for converting the geometrical data and the texture data from integer format to floating point format when the initial file has been transferred.

20. The device according to claim 10, wherein objects which have an importance factor lower than a maximum threshold are not updated.

19. An electronic device for a computer system configured to display three dimensional computer modeled image scenes streaming from a server to at least one client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, and to start the displaying by transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an for an initial displaying, the system comprising:

in the client computer, electronic circuitry configured to initiate a further downloading of data from the server to the at least one client computer according to an importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

21. The device according to claim 19, wherein the importance factor for a current object is based at least in part on at least one of a distance from the observation point to the current object, size of the current object, size of the current object as seen from the observation point, size of at least part of the current object not seen from the observation point, viewing angle to the current object from the observation point, and a parameter related to the current object comprised in the streaming of the three dimensional modeled image scenes.

22. The device according to claim 19, wherein the electronic circuitry is configured to initiate downloading of data according to the calculated importance factor for texture data for all objects in one scene at a time from the streaming scenes, and to download the texture data for the object in each respective scene having the highest importance factor, and then displaying each respective scene one at a time.

23. The device according to claim 22, wherein the electronic circuitry is further configured to repeatedly download data for objects in the one scene according to the importance factor for each object until there is no more available allocated memory in the at least one client computer.

24. The device according to claim 23, wherein the electronic circuitry is configured to delete the downloaded texture data for the objects in the scenes having an importance factor lower than a threshold when there is no more available allocated memory in the at least one client computer.

25. The device according to claim 23, wherein the electronic circuitry is further configured to calculate the importance factor based at least in part on other downloaded texture data than those texture data used for displaying.

26. The device according to claim 25, wherein the electronic circuitry is configured to use a system parameter or a user selectable parameter to determine which texture data to be used in the calculation of the importance factor.

27. The device according to claim 19, further comprising:

electronic circuitry in communication with the server, the electronic circuitry configured to;

test if the geometrical data and the texture data are within predefined limits;

convert the geometrical data and the texture data from floating point format to integer format;

provide an initial file for a scene comprising the converted data; and

convert the converted data back from integer format to floating point format before using the data for displaying.

28. A method of displaying three dimensional computer modeled image scenes streaming from a server to a client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the method comprising:

transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for initial display;

in the client computer, displaying an image based on the minimum geometric data and the minimum texture data;

in the client computer, initiating a further downloading of data from the server to the client computer according to an importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

29. A method of displaying three dimensional computer modeled image scenes streaming from a server to a client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the method comprising:

transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for initial display;

in the client computer, displaying an image based on the minimum geometric data and the minimum texture data;

in the client computer, initiating a further downloading of data from the server to the client computer according to an importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

30. A method of displaying three dimensional computer modeled image scenes streaming from a server to a client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the method comprising:

transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for initial display;

in the client computer, displaying an image based on the minimum geometric data and the minimum texture data;

in the client computer, initiating a further downloading of data from the server to the client computer according to an importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

31. A method of displaying three dimensional computer modeled image scenes streaming from a server to a client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the method comprising:

transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for initial display;

in the client computer, displaying an image based on the minimum geometric data and the minimum texture data;

in the client computer, initiating a further downloading of data from the server to the client computer according to an importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

32. A computer system comprising a server and at least one client computer configured to display three dimensional computer modeled image scenes streaming from the server to the client, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the system comprising:

means for transferring an initial file after receipt of a request from the at least one client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an initial displaying;

means, in the at least one client computer, for initiating a further downloading of data from the server to the at least one client computer according to a calculated importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

33. A computer system comprising a server and at least one client computer configured to display three dimensional computer modeled image scenes streaming from the server to the client, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the system comprising:

means for transferring an initial file after receipt of a request from the at least one client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an initial displaying;

means, in the at least one client computer, for initiating a further downloading of data from the server to the at least one client computer according to a calculated importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

34. A computer system comprising a server and at least one client computer configured to display three dimensional computer modeled image scenes streaming from the server to the client, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the system comprising:

means for transferring an initial file after receipt of a request from the at least one client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an initial displaying;

means, in the at least one client computer, for initiating a further downloading of data from the server to the at least one client computer according to a calculated importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

35. A computer system comprising a server and at least one client computer configured to display three dimensional computer modeled image scenes streaming from the server to the client, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, the system comprising:

means for transferring an initial file after receipt of a request from the at least one client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an initial displaying;

means, in the at least one client computer, for initiating a further downloading of data from the server to the at least one client computer according to a calculated importance factor P indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor P above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

36. An electronic device for a computer system configured to display three dimensional computer modeled image scenes streaming from a server to at least one client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, and to start the displaying by transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an for an initial displaying, the system comprising:

in the client computer, electronic circuitry configured to initiate a further downloading of data from the server to the at least one client computer according to an importance factor indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

37. An electronic device for a computer system configured to display three dimensional computer modeled image scenes streaming from a server to at least one client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, and to start the displaying by transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an for an initial displaying, the system comprising:

in the client computer, electronic circuitry configured to initiate a further downloading of data from the server to the at least one client computer according to an importance factor indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

38. An electronic device for a computer system configured to display three dimensional computer modeled image scenes streaming from a server to at least one client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, and to start the displaying by transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an for an initial displaying, the system comprising:

in the client computer, electronic circuitry configured to initiate a further downloading of data from the server to the at least one client computer according to an importance factor indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;

39. An electronic device for a computer system configured to display three dimensional computer modeled image scenes streaming from a server to at least one client computer over a network, wherein the three dimensional computer modeled image scenes comprise geometrical data describing shapes of objects and texture data describing the appearance of the objects in the scenes, and to start the displaying by transferring an initial file after receipt of a request from the client computer to the server computer, the initial file providing minimum geometrical data and minimum texture data for minimum visual quality, the minimum geometric data and minimum texture data to be used for an for an initial displaying, the system comprising:

in the client computer, electronic circuitry configured to initiate a further downloading of data from the server to the at least one client computer according to an importance factor indicating the importance of both visible objects and at present not visible objects that are associated with visible objects in the vicinity of an observation point, such that objects having an importance factor above a minimum threshold are updated with additional data, thereby providing more details for the objects having an importance factor above the minimum threshold, wherein the importance factor is defined as;