Image storage method, image rendering method, image storage apparatus, image processing apparatus, image download method, and computer and storage medium
First Claim
1. An image storage method for storing space data, including real image data of an actually captured image, in association with a predetermined space different from a real image space, comprising:
- the first layer dividing step of hierarchically dividing the real image data into layers in accordance with resolutions;
the second layer dividing step of hierarchically dividing a map that represents correspondence between the real image space and the predetermined space different from the real image space into layers in accordance with resolutions, the resolutions of the divided layers corresponding to resolutions of the layers divided in the first layer dividing step; and
the step of storing obtained hierarchical image data and hierarchical map data in a storage device.
2 Assignments
0 Petitions
Accused Products
Abstract
There is disclosed a progressive display method upon rendering and displaying a virtual object expressed by RS data. When RS data is generated and recorded on the basis of real image data, map data that represents correspondence between a real image space and ray space is divided into a plurality of layers, and the resolutions of the individual layers are hierarchically changed, thus storing hierarchical map data. Upon rendering a virtual image from an arbitrary viewpoint position, rendering is done by looking up the hierarchical map data in turn in ascending order of resolution. Also, there are disclosed a downloading method of selectively downloading RS data and VRML data, and a rendering method of downloaded data. RS data with high resolution is stored in an external memory, and VRML data having a required memory size is stored in an internal memory. In principle, a virtual image is generated by rendering RS data to allow the user to experience the virtual space. When downloading of RS data is not in time, an image rendered based on VRML data is displayed.
50 Citations
68 Claims
-
1. An image storage method for storing space data, including real image data of an actually captured image, in association with a predetermined space different from a real image space, comprising:
-
the first layer dividing step of hierarchically dividing the real image data into layers in accordance with resolutions;
the second layer dividing step of hierarchically dividing a map that represents correspondence between the real image space and the predetermined space different from the real image space into layers in accordance with resolutions, the resolutions of the divided layers corresponding to resolutions of the layers divided in the first layer dividing step; and
the step of storing obtained hierarchical image data and hierarchical map data in a storage device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
7. The method according to claim 1, wherein if the block size is N×
- N, the block segmentation step sets;
N=2P−
1where p is the number of divided layers.
- N, the block segmentation step sets;
-
8. An image rendering method for rendering hierarchical space data stored by a method cited in claim 1 in a virtual space, comprising:
-
the step of reading hierarchical space data of respective layers in ascending order of resolution; and
the rendering step of rendering a virtual image in the order the hierarchical space data is read.
-
-
9. The method according to claim 8, further comprising the detection step of detecting a user'"'"'s viewpoint position, and wherein the rendering step includes the step of rendering a virtual image on the basis of the detected viewpoint position.
-
10. The method according to claim 9, wherein the reading step includes the step of reading hierarchical space data of a space near the detected viewpoint position.
-
11. A storage medium that stores a program for implementing a method cited in claim 1 on a computer.
-
12. A computer that mounts a storage medium cited in claim 11.
-
13. An image storage apparatus for storing space data, including real image data of an actually captured image, in association with a predetermined space different from a real image space, comprising:
-
first layer dividing means for hierarchically dividing the real image data into layers in accordance with resolutions;
second layer dividing means for hierarchically dividing a map that represents correspondence between the real image space and the predetermined space different from the real image space into layers in accordance with resolutions, the resolutions of the divided layers corresponding to resolutions of the layers divided by said first layer dividing means; and
storage means for storing obtained hierarchical image data and hierarchical map data. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21)
-
-
19. The apparatus according to claim 13, wherein if the block size is N×
- N, said block segmentation means sets;
N=2P−
1where p is the number of divided layers.
- N, said block segmentation means sets;
-
20. An image processing apparatus for rendering hierarchical space data stored by an image storage apparatus cited in claim 13 in a virtual space, comprising:
-
means for reading hierarchical space data of respective layers in ascending order of resolution; and
rendering means for rendering a virtual image in the order the hierarchical space data is read.
-
-
21. The apparatus according to claim 20, further comprising detection means for detecting a user'"'"'s viewpoint position, and wherein said rendering means includes the means for rendering a virtual image on the basis of the detected viewpoint position.
-
22. An image downloading method for downloading space data which is formed based on image data of an actually captured image in association with a predetermined space different from a real image space, comprising:
-
the first layer dividing step of hierarchically dividing real image data in the space data into layers in accordance with resolutions;
the second layer dividing step of hierarchically dividing a map that represents correspondence between the real image space and the predetermined space into layers in accordance with resolutions; and
the downloading step of downloading obtained hierarchical image data, and a map that represents correspondence between the real image space and the predetermined space to an external apparatus, wherein the downloading step includes the step of downloading the obtained hierarchical image data and hierarchical map data. - View Dependent Claims (23, 24, 25, 26, 27, 28)
-
-
29. An image processing method comprising:
-
the step of converting input image data into space data that pertains to a predetermined space coordinate system different from an image space coordinate system of the input image data;
the first layer dividing step of hierarchically dividing the space data into layers in accordance with resolutions;
the step of generating a map that represents correspondence between image data in the image space coordinate system and space data in the predetermined space;
the second layer dividing step of hierarchically dividing the map into layers in accordance with resolutions;
the step of storing obtained hierarchical space data and hierarchical map data in a storage device; and
the image generation step of generating an image on the basis of the hierarchical space data and hierarchical map data. - View Dependent Claims (30, 31, 32, 33, 34, 35, 36, 37, 38, 39)
-
-
35. The method according to claim 29, wherein if the block size is N×
- N, the block segmentation step sets;
N=2P−
1where p is the number of divided layers.
- N, the block segmentation step sets;
-
36. The method according to claim 29, further comprising:
-
the step of reading hierarchical space data of respective layers in ascending order of resolution; and
the rendering step of rendering a virtual image in the order the hierarchical space data is read.
-
-
37. The method according to claim 36, further comprising the detection step of detecting a user'"'"'s viewpoint position, and wherein the rendering step includes the step of rendering a virtual image on the basis of the detected viewpoint position.
-
38. The method according to claim 37, wherein the reading step includes the step of reading hierarchical space data of a space near the detected viewpoint position.
-
39. A storage medium that stores a program for implementing a method cited in claim 29 on a computer.
-
40. An image processing apparatus comprising:
-
means for converting input image data into space data that pertains to a predetermined space coordinate system different from an image space coordinate system of the input image data;
first layer dividing means for hierarchically dividing the space data into layers in accordance with resolutions;
means for generating a map that represents correspondence between image data in the image space coordinate system and space data in the predetermined space;
second layer dividing means for hierarchically dividing the map into layers in accordance with resolutions;
means for storing obtained hierarchical space data and hierarchical map data in a storage device; and
image generation means for generating an image on the basis of the hierarchical space data and hierarchical map data. - View Dependent Claims (41, 42, 43, 44, 45, 46, 47, 48, 49, 50)
-
-
46. The apparatus according to claim 40, wherein if the block size is N×
- N, said block segmentation means sets;
N=2P−
1where p is the number of divided layers.
- N, said block segmentation means sets;
-
47. The apparatus according to claim 40, further comprising:
-
means for reading hierarchical space data of respective layers in ascending order of resolution; and
rendering means for rendering a virtual image in the order the hierarchical space data is read.
-
-
48. The apparatus according to claim 47, further comprising detection means for detecting a user'"'"'s viewpoint position, and wherein said rendering means renders a virtual image on the basis of the detected viewpoint position.
-
49. The apparatus according to claim 48, wherein said reading means reads hierarchical space data of a space near the detected viewpoint position.
-
50. A storage medium that stores a program for implementing a method cited in claim 40 on a computer.
-
51. A rendering method for rendering a virtual image viewed from an arbitrary viewpoint position in a virtual space, comprising the steps of:
-
storing first space data in a first format at a first viewpoint position in an internal memory;
storing second space data in a second format at the first viewpoint position in an external memory;
downloading the second space data from the external memory to generate a virtual image at an arbitrary viewpoint position; and
rendering the virtual image at the arbitrary viewpoint position on the basis of the first space data in the internal memory before downloading is completed, and on the basis of the second space data downloaded into the internal memory after downloading is completed. - View Dependent Claims (52, 53, 54, 55, 56, 57, 58, 61, 62, 67)
detecting a user'"'"'s viewpoint position which moves; and
executing a rendering process of an image at a viewpoint position after movement, and a download process of the second space data as parallel processes.
-
-
57. The method according to claim 51, further comprising the steps of:
-
storing space data of a plurality of images captured in advance from a plurality of different viewpoint positions in the external memory as the second space data;
detecting a user'"'"'s viewpoint position which moves;
downloading space data corresponding to a viewpoint position after movement from the external memory; and
rendering a virtual image on the basis of the first space data in the internal memory before downloading is completed, and on the basis of the second space data which has been downloaded into the internal memory and corresponds to the viewpoint position after movement after downloading is completed.
-
-
58. The method according to claim 51, wherein the internal memory has a capacity for storing at least the first and second space data.
-
61. The apparatus according to claim 51, wherein the first space data is VRML space data, and the second space data is ray space data.
-
62. The apparatus according to claim 58, wherein a virtual walk-through environment is provided to a user in the virtual space.
-
67. A program storage medium that stores a computer program for implementing an image rendering method cited in claim 51 via a computer.
-
59. A rendering apparatus for rendering a virtual image at an arbitrary viewpoint position in a virtual space, comprising:
-
an internal memory for storing first space data in a first format at a first viewpoint position;
an external memory for storing second space data in a second format at the first viewpoint position, said external memory being different from said internal memory;
download means for downloading the second space data from said (external memory to generate a virtual image at an arbitrary viewpoint position; and
rendering means for rendering the virtual image at the arbitrary viewpoint position on the basis of the first space data in said internal memory before downloading is completed, and on the basis of the second space data downloaded into said internal memory after downloading is completed. - View Dependent Claims (60, 63, 64, 65, 66)
said apparatus further comprises means for detecting a user'"'"'s viewpoint position which moves, and said download means and rendering means execute a rendering process of an image at a viewpoint position after movement, and a download process of the second space data as parallel processes. -
65. The apparatus according to claim 59, wherein said external memory stores space data of a plurality of images captured in advance from a plurality of different viewpoint positions as the second space data,
said apparatus further comprises means for detecting a user'"'"'s viewpoint position which moves, said download means downloads space data corresponding to a viewpoint position after movement from said external memory, and said rendering means renders a virtual image on the basis of the first space data in said internal memory before downloading is completed, and on the basis of the second space data which has been downloaded into said internal memory and corresponds to the viewpoint position after movement after downloading is completed. -
66. The apparatus according to claim 59, wherein said internal memory has a capacity for storing at least the first and second space data.
-
-
68. A rendering method for rendering a virtual image at an arbitrary viewpoint position in a virtual space, comprising the steps of:
-
rendering first space data in a first data format and second space data in a second data format at an arbitrary viewpoint position as virtual images; and
mapping the virtual image based on the first space data in the virtual space when rendering of the virtual image based on the second space data does not end in time before mapping in the virtual space.
-
Specification