Image display method and image display device

• US 6,967,651 B2
• Filed: 06/06/2002
• Issued: 11/22/2005
• Est. Priority Date: 12/06/1999
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Endoh, Toshio, Fukuda, Mitsuaki
• Primary Examiner(s): Jankus, Almis R.
• Application Number:
  US10/162,594
• Publication Number:
  US 20020180734A1
• Time in Patent Office: 1,265 Days
• Field of Search: 345/419, 345/660, 345/661, 345/662, 345/664, 345/428
• US Class Current: 345/428
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2210/36 Level of detail

Abstract

An image display method is provided, with which a large number of images to be searched can be displayed quickly in a 3D space, with easy to understand image rendering and little strain on the user'"'"'s eyes. The method includes a process of reading in image data of a plurality of images and information about the arrangement of the images in the virtual 3D space; a spatial viewpoint setting/shifting process in which setting and shifting of a spatial viewpoint is specified; a process of converting original shapes of the image data by a parallel expansion projection process including a parallel shifting process regarding the directions of the images and a expansion/shrinking process regarding their size, based on a relation between the information about the arrangement of the images and the spatial viewpoint information, and determining a simplified 2D arrangement as a 2D image; a display image creation process of creating displayed images in accordance with the determined simplified 2D arrangement; and a process of displaying the display images. Through multi-resolution synthesized images and cut images and through the guiding display of a background image, an easy to understand image rendering is achieved, and the eye strain is evaluated and minimized on the basis of the optical flow.

Claims

1. An image display method for arranging and displaying images in a virtual 3D space, comprising:

a process of reading in image data of at least one image;

a process of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting process in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a process of converting original shapes of the read-in image data by a parallel expansion projection process including a parallel shifting process regarding the directions in which the images are seen from the spatial viewpoint and an expansion/shrinking process regarding their size, based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information, and determining a simplified 2D arrangement expressing 2D arrangement of the images seen from the spatial viewpoint as a 2D arrangement of images with shapes that are similar to the original images and a process of rotating an original shape of the read-in image data into an angle of multiples of 90°

, based on the relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation process of creating displayed images by processing the read-in image data in accordance with the determined simplified 2D arrangement of the images; and

a process of displaying the created display images on a display device.

2. An image display method for arranging and displaying images in a virtual 3D space, comprising:

a process of reading in image data of at least one image;

a process of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting process in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

process of converting original shapes of the read-in image data by a parallel expansion projection process including a parallel shifting process regarding the directions in which the images are seen from the spatial viewpoint and an expansion/shrinking process regarding their size based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information, and determining a simplified 2D arrangement expressing a 2D arrangement of the images seen from the spatial viewpoint as a 2D arrangement of images with shapes that are similar to the original images;

a display image creation process of creating displayed images by processing the read-in image data in accordance with the determined simplified 2D arrangement of the images;

a process of displaying the created display images on a display device;

a process of writing image data in accordance with the simplified 2D arrangement of the images in the display image creation; and

a process of writing a substitution image or substitution-related information if an orientation of the images displayed in the simplified 2D arrangement is outside a range of preset values.

3. An image display method for arranging and displaying images in a virtual 3D space, comprising:

an image data reading process of reading in image data of at least one image with at least one resolution, wherein the image data are multi-resolution image data made of a set of image data of at least one resolution;

a process of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting process in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a process of determining a 2D arrangement of position, orientation and size of the images seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation process of selecting the image data of at least one resolution from the read-in multi-resolution image data, in accordance with the determined 2D arrangement, and writing them into display images to create the display images; and

a process of displaying the created display images on a display device.

4. The image display method according to claim 3, wherein the image data reading process can be selected from

a process of reading in all sets of multi-resolution image data of the read-in images prior to the display image creation process;

and

a process of reading in, at the time when image data of one resolution has been selected in the display image creation process, only the selected resolution image data from the multi-resolution image data of the read-in images.

5. The image display method according to claim 4, further comprising an image data deletion process of deleting those image data that have become obsolete from memory, which have been read in with the process of reading in image data,

wherein the image data deletion process can be selected from a process of deleting the image data that have become obsolete when an image data recording capacity has exceeded a preset capacity in memory, and a process of deleting image data of other resolutions when image data of at least one resolution have been selected in the display image creation process.

6. The image display method according to claim 3,

further comprising a multi-resolution image data creation process of creating the multi-resolution image data from the original image data of the images,

wherein the multi-resolution image data creation process includes a process of dividing the original image data into a plurality of partial regions and retrieving only specified partial regions, a process of creating image writing frames adapted to a size of the created images of various resolutions, a process of writing of data into corresponding positions of the image writing frames while adapting to the size of the image writing frames and maintaining the size of the retrieved partial regions as large as possible, and a process of creating the data of one synthesized image by omitting the writing of data into the not-yet-written portions of the image writing frames,

wherein, in the display image creation process, the synthesized image data corresponding to a hierarchy of selected resolutions are written into the display images, creating the display images.

7. The image display method according to claim 6, wherein, in the multi-resolution image data creation process, the process of retrieving only the specified partial regions is a process of specifying those specifying partial regions of the plurality of partial regions into which the original images have been divided for which a differential value of the image data is larger than a preset value.

8. An image display method for arranging and displaying images in a virtual 3D space, comprising:

a process of reading in image data of at least one image;

a process of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting process in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a process of determining a 2D arrangement of position, orientation and size of the images seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation process of writing into the display images cut image data in which a region of a size of an image of the 2D arrangement has been cut from the read-in image data in accordance with the determined 2D arrangement to create the display images; and

a process of displaying the created display images on a display device.

9. The image display method according to claim 8, wherein, in the display image creation process, the region that is cut as the cut image data is taken to be a region of the read-in image data in which a differential value of the image data is larger than a preset value.

10. An image display meted for arranging and displaying images in a virtual 3D space, comprising:

a process of reading in image data of at least one image;

a process of reading in information about an arrangement of the images in the virtual 3D space;

a process of reading in image data of a background image that exists virtually as a background in the virtual 3D space;

a process of reading in information about an arrangement of the background image in the virtual 3D space;

a spatial viewpoint setting/shifting process in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a process of determining a 2D arrangement of position, orientation and size of the images and the background image seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images and the background image in the virtual 3D space and the spatial viewpoint information;

a display image creation process of creating display images by processing the read-in image data and background image data in accordance with the determined 2D arrangement of the images and the background image; and

a process of displaying the created display images on a display device.

11. The image display method according to claim 10, wherein, in the display image creation process, the display images are created by switching the background image data between prepared image data that have been shot of a prepared background object and computer graphics data of the background object, in correspondence to the 2D arrangement of the background image seen from the spatial viewpoint.

12. An image display device for arranging and displaying images in a virtual 3D space, comprising:

an image input portion for reading in image data of at least one image;

an arrangement information input portion for reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting portion that specifies setting and shifting of a spatial viewpoint with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a projection processing portion for converting original shapes of the read-in image data by a parallel expansion projection process including a parallel shifting process regarding the directions in which the images are seen from the spatial viewpoint, an expansion/shrinking process regarding their size, based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information, and determining a simplified 2D arrangement expressing a 2D arrangement of the images seen from the spatial viewpoint as a 2D arrangement of images with shapes that are similar to the original images and a process of rotating an original shape of the read-in image data into an angle multiples of 90°

, based on the relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation processing portion for creating displayed images by processing the read-in image data in accordance with the determined simplified 2D arrangement of the images; and

a display portion for displaying the created display images on a display device.

13. An image display device for arranging and displaying images in a virtual 3D space, comprising:

an image input portion for reading in image data of at least one image with at least one resolution, wherein the image data are multi-resolution image data made of a set of image data of at least one resolution;

an arrangement information input portion for reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting processing portion that specifies setting and shifting of a spatial viewpoint with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a projection processing portion for determining a 2D arrangement of position, orientation and size of the images seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation processing portion for selecting the image data of at least one resolution from the read-in multi-resolution image data, in accordance with the determined 2D arrangement, and writing them into the display images to create the display images; and

a display device for displaying the created display images on a display device.

14. An image display device for arranging and displaying images in a virtual 3D space, comprising:

an image input portion for reading in image data of at least one image;

an arrangement information input portion for reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting processing portion that specifies setting and shifting of a spatial viewpoint with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a projection processing portion for determining a 2D arrangement of position, orientation and size of the images seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation processing portion for writing into the display images cut image data in which a region of a size of an image of the 2D arrangement has been cut from the written image data in accordance with the determined 2D arrangement, to create the display images; and

a display portion for displaying the created display images on a display device.

15. A computer-readable recording medium storing a program for realizing an image display method for arranging and displaying images in a virtual 3D space, the program comprising:

a processing step of reading in image data of at least one image;

a processing step of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting processing step in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a processing step of converting original shapes of the read-in image data by a parallel expansion projection process including a parallel shifting process regarding the directions in which the images are seen from the spatial viewpoint and a expansion/shrinking process regarding their size, based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information, determining a simplified 2D arrangement expressing a 2D arrangement of the images seen from the spatial viewpoint as a 2D arrangement of images with shapes that are similar to the original images and rotating an original shape of the read-in image data into an angle of multiples of 90°

, based on the relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation processing step of creating displayed images by processing the read-in image data in accordance with the determined simplified 2D arrangement of the images; and

a processing step of displaying the created display images on a display device.

16. A computer-readable recording medium storing a program for realizing an image display method for arranging and displaying images in a virtual 3D space, the program comprising:

an image data reading processing step of reading in image data of at least one image with at least one resolution, wherein the image data are multi-resolution image data made of a set of image data of at least one resolution;

a processing step of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting processing step in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a processing step of determining a 2D arrangement of position, orientation and size of the images seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation processing step of selecting the image data of at least one resolution from the read-in multi-resolution image data, in accordance with the determined 2D arrangement, and writing them into the display images to create the display images; and

a processing step of displaying the created display images on a display device.

17. A computer-readable recording medium storing a program for realizing an image display method for arranging and displaying images in a virtual 3D space, the program comprising:

a processing step of reading in image data of at least one image;

a processing step of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting processing step in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a processing step of determining a 2D arrangement of position, orientation and size of the images seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information;

a display image creation processing step of writing into the display images cut image data in which a region of a size of an image of the 2D arrangement has been cut from the written image data in accordance with the determined 2D arrangement to create the display images; and

a processing step of displaying the created display images on a display device.

18. A computer-readable recording medium storing a program for realizing an image display method for arranging and displaying images in a virtual 3D space, the program comprising:

a processing step of reading in image data of at least one image;

a processing step of reading in information about an arrangement of the images in the virtual 3D space;

a processing step of reading in image data of a background image that exists virtually as a background in the virtual 3D space;

a processing step of reading in information about an arrangement of the background image in the virtual 3D space;

a spatial viewpoint setting/shifting processing step in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a processing step of determining a 2D arrangement of position, orientation and size of the images and the background image seen from the spatial viewpoint, by performing a projection process based on a relation between the information about the arrangement of the images and the background image in the virtual 3D space and the spatial viewpoint information;

a display image creation processing step of creating display images by processing the read-in image data end background image data in accordance with the determined 2D arrangement of the images end the background image; and

a processing step of displaying the created display images on a display device.

19. An image display device for arranging and displaying images in a virtual 3D space, comprising:

an image input portion for reading in image data of at least one image;

an arrangement information input portion for reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting portion that specifies setting and shifting of a spatial viewpoint with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a projection processing portion for converting original shapes of the read-in image data by a parallel expansion projection process including a parallel shifting process regarding the directions in which the images are seen from the spatial viewpoint and a expansion/shrinking process regarding their size, based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information, and determining a simplified 2D arrangement expressing a 2D arrangement of the images seen from the spatial viewpoint as a 2D arrangement of images with shapes that are similar to the original images, a process of writing image data in accordance with the simplified 2D arrangement of the images in the display image creation process and a process of writing a substitution image or substitution-related information if an orientation of the images displayed in the simplified 2D arrangement is outside a range of preset values;

a display image creation processing portion for creating displayed images by processing the read-in image data in accordance with the determined simplified 2D arrangement of the images; and

a display portion for displaying the created display images on a display device.

20. A computer-readable recording medium storing a program for realizing an image display method for arranging and displaying images in a virtual 3D space, comprising:

a process of reading in image data of at least one image;

a process of reading in information about an arrangement of the images in the virtual 3D space;

a spatial viewpoint setting/shifting process in which setting and shifting of a spatial viewpoint is specified with parameters including a viewpoint position and a line of sight direction in the virtual 3D space;

a process of converting original shapes of the read-in image data by a parallel expansion projection process including a parallel shifting process regarding the directions in which the images are seen from the spatial viewpoint and an expansion/shrinking process regarding their size, based on a relation between the information about the arrangement of the images in the virtual 3D space and the spatial viewpoint information, and determining a simplified 2D arrangement expressing a 2D arrangement of the images seen from the spatial viewpoint as a 2D arrangement of images with shapes that are similar to the original images;

a display image creation process of creating displayed images by processing the read-in image data in accordance with the determined simplified 2D arrangement of the images;

a process of displaying the created display images on a display device;

a process of writing image data in accordance with the simplified 2D arrangement of the images in the display image creation process; and

a process of writing a substitution image or substitution-related information if an orientation of the images displayed in the simplified 2D arrangement is outside a range of preset values.