Image reproducing method and image generating and reproducing method

• US 6,317,151 B1
• Filed: 01/05/1998
• Issued: 11/13/2001
• Est. Priority Date: 07/10/1997
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Mizukura, Isao, Ohsuga, Mieko
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): WONG, ALLEN C
• Application Number:
  US09/002,926
• Time in Patent Office: 1,408 Days
• Field of Search: 348/36, 348/37, 348/38, 348/39, 348/61, 348/121, 348/159, 348/169, 348/170, 348/171, 348/172, 463/36, 482/8
• US Class Current: 348/36
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6661 for changing the position of the virtual camera
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131

Abstract

At each of n shooting (or capturing) points, a natural scene is captured for a period of time t_S(N), where N=1 to n, using one or more video cameras held at rest, and it is then converted into each of a plurality of stationary-viewpoint images. Furthermore, on the way from the (N−1)th shooting point to the Nth (N=2 to n) shooting point, a natural scene is captured for a period of time t_M(N), where N=2 to n, using the video cameras being moved and it is then converted into each of a plurality of moving-viewpoint images. In order to provide a virtual reality environment for a user, the plurality of stationary-viewpoint images and the plurality of moving-viewpoint images are reproduced in turn according to user'"'"'s movements detected.

Claims

1. An image reproducing system comprising:

image storage means for storing a plurality of stationary-viewpoint images each of which was converted from a natural scene which was captured using an image generating device which was placed at each of a plurality of predetermined shooting points and was held at rest, and for storing a plurality of moving-viewpoint images each of which was converted from a natural scene which was captured using the image generating device as it was moved on the way from each of the plurality of shooting points to a next shooting point, so as to provide a virtual reality environment for a user, the plurality of shooting points starting from a shooting start point being arranged on a path along which the user will move in the virtual reality environment, and each of the plurality of shooting points starting from the shooting start point which corresponds to a start point from which the user will start to move in the virtual reality environment being referred to as the Nth shooting point (N=1 to n, and n is the number of the plural shooting points), an interval between said shooting points being controlled in accordance with a length of said user'"'"'s step,

movement detecting means for detecting a movement of the user in the virtual reality environment; and

reproducing means for reproducing the plurality of stationary-viewpoint images and the plurality of moving-viewpoint images stored in said image storage means in turn according to user'"'"'s movements detected by said movement detecting means;

wherein said moving viewpoint images are captured using a video camera in a video mode so that said stored moving viewpoint images appear as natural motion to a viewer.

2. The image reproducing system according to claim 1, further comprising an environment database for storing environment data on environments at the plurality of shooting points and on the way from each of the plurality of shooting points to the next shooting point, and load providing means for providing the user with a load dependent on said environment data when said reproducing means reproduces one stationary-viewpoint image or one moving-viewpoint image.

3. The image reproducing system according to claim 1, wherein if said movement detecting means detects user'"'"'s movements while said reproducing means is reproducing a stationary-viewpoint image which was created at the Nth shooting point, said reproducing means is switched to reproduction of a moving-viewpoint image which was created on the way to the (N+1)th shooting point next, and wherein unless said movement detecting means detects user'"'"'s movements after the completion of the reproduction of the moving-viewpoint image which was created on the way to the (N+1)th shooting point, said reproducing means is caused to reproduce a stationary-viewpoint image which was created at the (N+1)th shooting point next.

4. The image reproducing system according to claim 3, wherein unless said movement detecting means detects user'"'"'s movements while said reproducing means is reproducing a stationary-viewpoint image which was created at the Nth shooting point for a predetermined period of time, said reproducing means is caused to reproduce the same stationary-viewpoint image repeatedly.

5. The image reproducing system according to claim 3, wherein if said movement detecting means detects user'"'"'s movements after said reproducing means reproduces the moving-viewpoint image which was created on the way to the (N+1)th shooting point for a predetermined period of time, said reproducing means is caused to reproduce a moving-viewpoint image which was created on the way to the (N+2)th shooting point next.

6. A computer-implemented image reproducing method comprising the steps of:

detecting a movement of a user in a virtual reality environment; and

reproducing a plurality of stationary-viewpoint images and a plurality of moving-viewpoint images stored in image storage means in turn according to user'"'"'s movements detected in the above detecting step, each of the plurality of stationary-viewpoint images being converted from a natural scene which was captured using an image generating device which was placed at each of a plurality of predetermined shooting points and was held at rest, each of the plurality of moving-viewpoint images being converted from a natural scene which was captured using the image generating device which was being moved on the way from one of the plurality of shooting points to the next shooting point, the plurality of shooting points starting from a shooting start point being arranged on a path along which the user will move in the virtual reality environment, an interval between said shooting points being controlled in accordance with a length of said user'"'"'s step, and each of the plurality of shooting points starting from the shooting start point which corresponds to a start point from which the user will start to move in the virtual reality environment being referred to as the Nth shooting point (N=1 to n, and n is the number of the plural shooting points);

wherein said moving viewpoint images are captured using a video camera in a video mode so that said stored moving viewpoint images appear as natural motion to a viewer.

7. The computer-implemented image reproducing method according to claim 6, wherein if user'"'"'s movements is detected in said detecting step while a stationary-viewpoint image which was created at the Nth shooting point is being reproduced, switching to reproduction of a moving-viewpoint image which was created on the way to the (N+1)th shooting point is performed, and wherein unless user'"'"'s movements is detected in said detecting step after the completion of the reproduction of the moving-viewpoint image which was created on the way to the (N+1)th shooting point, a stationary-viewpoint image which was created at the (N+1)th shooting point is reproduced next.

8. The computer-implemented image reproducing method according to claim 7, wherein unless user'"'"'s movements is detected in said detecting step while a stationary-viewpoint image which was created at the Nth shooting point is being reproduced for a predetermined period of time, the same stationary-viewpoint image is reproduced repeatedly.

9. The computer-implemented image reproducing method according to claim 7, wherein if user'"'"'s movements is detected in said detecting step after the moving-viewpoint image which was created on the way to the (N+1)th shooting point is reproduced for a predetermined period of time, a moving-viewpoint image which was created on the way to the (N+2)th shooting point is reproduced next.

10. A method of generating and reproducing images, comprising the steps of:

generating each of a plurality of stationary-viewpoint images by capturing and converting a natural scene into each of the plurality of stationary-viewpoint images at each of a plurality of predetermined shooting points using an image generating device which is held at rest at each of the plurality of shooting points, and generating each of a plurality of moving-viewpoint images by capturing and converting a natural scene into each of the plurality of moving-viewpoint images on the way from each of the plurality of shooting points to the next shooting point using the image generating device which is being moved, so as to provide a virtual reality environment for a user, the plurality of shooting points starting from a shooting start point being arranged on a path along which the user will move in the virtual reality environment, an interval between said shooting points being controlled in accordance with a length of said user'"'"'s step, and each of the plurality of shooting points starting from the shooting start point which corresponds to a start point from which the user will start to move in the virtual reality environment being referred to as the Nth shooting point (N=1 to n, and n is the number of the plural shooting points);

storing the plurality of stationary-viewpoint images and the plurality of moving-viewpoint images;

detecting a movement of the user in the virtual reality environment; and

reproducing the plurality of stationary-viewpoint images and the plurality of moving-viewpoint images stored in the above storing step in turn according to users movements detected in the above detecting step;

wherein said moving viewpoint images are captured using a video camera in a video mode so that said stored moving viewpoint images appear as natural motion to a viewer;

wherein said moving viewpoint images are captured using a video camera in a video mode so that said stored moving viewpoint images appear as natural motion to a viewer.

11. The method according to claim 10, further comprising the steps of measuring environment data on environments at each of the plurality of shooting points and on the way from one of the plurality of shooting points to the next shooting point, storing the environment data measured in the above measuring step, and providing the user with a load dependent on the environment data stored in the above storing step while one stationary-viewpoint image or one moving-viewpoint image is reproduced in the reproducing step.

12. The method according to claim 10, wherein if user'"'"'s movements is detected in said detecting step while a stationary-viewpoint image which was created at the Nth shooting point is being reproduced, switching to reproduction of a moving-viewpoint image which was created on the way to the (N+1)th shooting point is performed, and wherein unless user'"'"'s movements is detected in said detecting step after the completion of the reproduction of the moving-viewpoint image which was created on the way to the (N+1)th shooting point, a stationary-viewpoint image which was created at the (N+1)th shooting point is reproduced next.

13. The method according to claim 12, wherein unless user'"'"'s movements is detected in said detecting step while a stationary-viewpoint image which was created at the Nth shooting point is being reproduced for a predetermined period of time, the same stationary-viewpoint image is reproduced repeatedly.

14. The method according to claim 12, wherein if user'"'"'s movements is detected in said detecting step after the moving-viewpoint image which was created on the way to the (N+1)th shooting point is reproduced for a predetermined period of time, a moving-viewpoint image which was created on the way to the (N+2)th shooting point is reproduced next.

15. An image reproducing system comprising:

image storage means for storing a plurality of stationary-viewpoint images each of which was converted from a natural scene which was captured using an image generating device which was placed at each of a plurality of predetermined shooting points and was held at rest, and for storing a plurality of moving-viewpoint images each of which was converted from a natural scene which was captured using the image generating device as it was moved on the way from each of the plurality of shooting points to a next shooting point, so as to provide a virtual reality environment for a user, the plurality of shooting points starting from a shooting start point being arranged on a path along which the user will move in the virtual reality environment, and each of the plurality of shooting points starting from the shooting start point which corresponds to a start point from which the user will start to move in the virtual reality environment being referred to as the Nth shooting point (N=1 to n, and n is the number of the plural shooting points), an interval between said shooting points being controlled in accordance with said user'"'"'s movement,

movement detecting means for detecting a movement of the user in the virtual reality environment; and

reproducing means for reproducing the plurality of stationary-viewpoint images and the plurality of moving-viewpoint images stored in said image storage means in turn according to user'"'"'s movements detected by said movement detecting means;

wherein said moving viewpoint images are captured using a video camera in a video mode so that said stored moving viewpoint images appear as natural motion to a viewer.

16. A method of generating and reproducing images, comprising the steps of:

generating each of a plurality of stationary-viewpoint images by capturing and converting a natural scene into each of the plurality of stationary-viewpoint images at each of a plurality of predetermined shooting points using an image generating device which is held at rest at each of the plurality of shooting points, and generating each of a plurality of moving-viewpoint images by capturing and converting a natural scene into each of the plurality of moving-viewpoint images on the way from each of the plurality of shooting points to the next shooting point using the image generating device which is being moved, so as to provide a virtual reality environment for a user, the plurality of shooting points starting from a shooting start point being arranged on a path along which the user will move in the virtual reality environment, an interval between said shooting points being controlled in accordance with said user'"'"'s movement, and each of the plurality of shooting points starting from the shooting start point which corresponds to a start point from which the user will start to move in the virtual reality environment being referred to as the Nth shooting point (N=1 to n, and n is the number of the plural shooting points);

storing the plurality of stationary-viewpoint images and the plurality of moving-viewpoint images;

detecting a movement of the user in the virtual reality environment; and

reproducing the plurality of stationary-viewpoint images and the plurality of moving-viewpoint images stored in the above storing step in turn according to users movements detected in the above detecting step;

wherein said moving viewpoint images are captured using a video camera in a video mode so that said stored moving viewpoint images appear as natural motion to a viewer.