INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING DEVICE, STORAGE MEDIUM STORING INFORMATION PROCESSING PROGRAM, AND MOVING IMAGE REPRODUCTION CONTROL METHOD

• US 20120307001A1
• Filed: 05/11/2012
• Published: 12/06/2012
• Est. Priority Date: 06/03/2011
• Status: Active Application

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: OSAKO, Satoru, MIYAMOTO, Shigeru
• Application Number:
  US13/469,690
• Publication Number:
  US 20120307001A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/5378 for displaying an additional top view, e.g. radar screens or maps using two or more virtual cameras concurrently A63F13/5252
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/307 for displaying an additional window with a view from the top of the game field, e.g. radar screen
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2215/08 Gnomonic or central projection

Abstract

A panorama moving image including panorama images of a real world is pasted on a model in a virtual space as texture, and an image of the model is captured by a virtual camera. An image-capturing direction of the virtual camera is changed in accordance with an attitude of a portable display device, and the image captured by the virtual camera is displayed on the portable display device.

Claims

1. An information processing system for displaying an image on a portable display device including a sensor for detecting a value in accordance with a movement or an attitude, the information processing system comprising:

a panorama moving image storage unit for storing a moving image captured and recorded in advance, the moving image including frames each including a panorama image of a real world;

a first virtual camera location unit for locating a first virtual camera at a predetermined position in a three-dimensional virtual space;

a first model location unit for locating a model surrounding the predetermined position in the virtual space;

a first virtual camera control unit for changing an attitude of the first virtual camera during reproduction of the moving image in accordance with an attitude of the portable display device based on data which is outputted from the sensor;

a first pasting unit for sequentially reading the panorama image of each frame from the panorama moving image storage unit and for sequentially pasting the panorama image on a surface of the model on the predetermined position side; and

a first display control unit for sequentially capturing an image of the virtual space by the first virtual camera and for sequentially displaying the captured image on the portable display device, in accordance with the pasting of the panorama image of each frame by the first pasting unit.

2. The information processing system according to claim 1, wherein the first pasting unit sequentially pastes the panorama image of each frame such that a fixed position in the panorama image is a fixed position in the model on the surface.

3. The information processing system according to claim 1, wherein the first pasting unit pastes the panorama image of each frame such that a fixed position in the panorama image matches an initial image-capturing direction of the first virtual camera.

4. The information processing system according to claim 1, further comprising an information processing device, separate from the portable display device, capable of transmitting image data to the portable display device by wireless communication;

wherein;

the panorama moving image storage unit, the first model location unit, the first virtual camera location unit, the first virtual camera control unit, the first pasting unit, and the first display control unit are included in the information processing device; and

the first display control unit transmits the captured image to the portable display device by wireless communication, and the portable display device receives the captured image by wireless communication and displays the captured image.

5. The information processing system according to claim 1, further comprising:

a map image storage unit for storing a map image representing an aerial view of a region in which the panorama image has been captured; and

a second display control unit for displaying the map image on a non-portable display device.

6. The information processing system according to claim 5, further comprising an image-capturing position information storage unit for storing information representing an image-capturing position, on the map image, of the panorama image of each frame;

wherein the second display control unit displays information representing the image-capturing position on the map image by use of the information representing the image-capturing position of each frame in accordance with the control on the display performed by the first display control unit based on the panorama image of each frame.

7. The information processing system according to claim 5, further comprising an image-capturing position information storage unit for storing information representing an image-capturing direction, on the map image, of the panorama image of each frame;

wherein the second display control unit displays information representing the image-capturing direction on the map image by use of the information representing the image-capturing direction of each frame in accordance with the control on the display performed by the first display control unit based on the panorama image of each frame.

8. The information processing system according to claim 5, wherein the map image storage unit stores the map image of each frame, and the second display control unit displays the map image of each frame on the portable display device in accordance with the display of the panorama image of each frame performed by the first display control unit on the portable display device.

9. The information processing system according to claim 8, wherein the second display control unit displays the map image in a rotated state in accordance with the attitude of the first virtual camera provided by the first virtual camera control unit.

10. The information processing system according to claim 6, wherein the panorama image of each frame and the information representing the image-capturing position of each frame are saved in one file.

11. The information processing system according to claim 7, wherein the panorama image of each frame and the information representing the image-capturing direction of each frame are saved in one file.

12. The information processing system according to claim 9, wherein the panorama image of each frame and the map image of each frame are saved in one file.

13. The information processing system according to claim 1, further comprising:

a second virtual camera location unit for locating a second virtual camera at the predetermined position; and

a second display control unit for sequentially capturing an image of the virtual space by the second virtual camera and displaying the captured image on a non-portable display device;

wherein an attitude of the second virtual camera is not changed even when the attitude of the first virtual camera is changed by the first virtual camera location unit.

14. The information processing system according to claim 1, wherein the model is a closed space model.

15. The information processing system according to claim 14, wherein:

the panorama image has an image having a dead angle; and

the first pasting unit pastes the panorama image on an area of an inner surface of the closed space model other than an area corresponding to the dead angle.

16. The information processing system according to claim 15, further comprising:

a complementary image storage unit for storing a predetermined complementary image; and

a second pasting unit for pasting the complementary image on the area of the inner surface of the closed space model corresponding to the dead angle.

17. The information processing system according to claim 1, wherein:

the panorama image is a spherical panorama image and has a dead angle in a lower area or an upper area thereof;

the model includes at least an upper part or a lower part of a sphere; and

the first pasting unit pastes the panorama image on an inner surface of the upper part or the lower part.

18. The information processing system according to claim 17, wherein:

the model is a closed space model; and

the information processing system further comprises;

a complementary image storage unit for storing a predetermined complementary image; and

a second pasting unit for pasting the complementary image on an area of an inner surface of the closed space model other than an area on which the panorama image is pasted.

19. The information processing system according to claim 16, wherein:

the complementary image storage unit stores the complementary image of each frame; and

the second pasting unit pastes the complementary image of each frame in accordance with the pasting of the panorama image of each frame performed by the first pasting unit.

20. The information processing system according to claim 19, wherein the panorama image of each frame and the complementary image of each frame are saved in one file.

21. The information processing system according to claim 1, further comprising a type information storage unit for storing type information of the panorama image;

wherein the first model location unit uses different models in accordance with the type information.

22. The information processing system according to claim 21, wherein the panorama image and the type information are saved in one file.

23. The information processing system according to claim 1, wherein:

the portable display device further includes an input unit for receiving an input operation; and

when data outputted from the input unit represents a predetermined operation, the first virtual camera control unit inverts a line-of-sight direction of the first virtual camera during the reproduction of the moving image.

24. An information processing system for displaying an image on a portable display device including a sensor for detecting a value in accordance with a movement or an attitude, the information processing system comprising:

a panorama moving image storage unit for storing a moving image captured and recorded in advance, the moving image including frames each including a spherical or all-around panorama image of a real world;

a first model location unit for locating a model representing a surface of a sphere or a side surface of a cylinder in a three-dimensional virtual space;

a first virtual camera location unit for locating a first virtual camera at a position inside the sphere or the cylinder represented by the model;

a first virtual camera control unit for changing an attitude of the first virtual camera during reproduction of the moving image in accordance with an attitude of the portable display device based on data which is outputted from the sensor;

a first pasting unit for sequentially pasting the panorama image of each frame on an inner surface of the model; and

a first display control unit for sequentially capturing an image of the virtual space by the first virtual camera and for sequentially displaying the captured image on the portable display device, in accordance with the sequential pasting of the panorama image of each frame by the first pasting unit.

25. An information processing device for displaying an image on a portable display device including a sensor for detecting a value in accordance with a movement or an attitude, the information processing device comprising:

a panorama moving image storage unit for storing a moving image captured and recorded in advance, the moving image including frames each including a panorama image of a real world;

a first virtual camera location unit for locating a first virtual camera at a predetermined position in a three-dimensional virtual space;

a first model location unit for locating a model surrounding the predetermined position in the virtual space;

a first virtual camera control unit for changing an attitude of the first virtual camera during reproduction of the moving image in accordance with an attitude of the portable display device based on data which is outputted from the sensor;

a first pasting unit for sequentially reading the panorama image of each frame from the panorama moving image storage unit and for sequentially pasting the panorama image on a surface of the model on the predetermined position side; and

a first display control unit for sequentially capturing an image of the virtual space by the first virtual camera and for sequentially displaying the captured image on the portable display device, in accordance with the sequential pasting of the panorama image of each frame by the first pasting unit.

26. A non-transitory computer-readable storage medium having an information processing program stored thereon, the information processing program being executable by a computer of an information processing device for displaying an image on a portable display device including a sensor for detecting a value in accordance with a movement or an attitude, wherein the information processing program allows the computer to execute:

storing a moving image captured and recorded in advance, the moving image including frames each including a panorama image of a real world;

locating a first virtual camera at a predetermined position in a three-dimensional virtual space;

locating a model surrounding the predetermined position in the virtual space;

changing an attitude of the first virtual camera during reproduction of the moving image in accordance with an attitude of the portable display device based on data which is outputted from the sensor;

sequentially reading the stored panorama image of each frame and sequentially pasting the panorama image on a surface of the model on the predetermined position side; and

sequentially capturing an image of the virtual space by the first virtual camera and sequentially displaying the captured image on the portable display device, in accordance with the sequential pasting of the panorama image of each frame.

27. A moving image reproduction control method for displaying an image on a portable display device including a sensor for detecting a value in accordance with a movement or an attitude, the method comprising:

storing a moving image captured and recorded in advance, the moving image including frames each including a panorama image of a real world;

locating a first virtual camera at a predetermined position in a three-dimensional virtual space;

locating a model surrounding the predetermined position in the virtual space;

changing an attitude of the first virtual camera during reproduction of the moving image in accordance with an attitude of the portable display device based on data which is outputted from the sensor;

sequentially reading the stored panorama image of each frame and sequentially pasting the panorama image on a surface of the model on the predetermined position side; and

sequentially capturing an image of the virtual space by the first virtual camera and sequentially displaying the captured image on the portable display device, in accordance with the sequential pasting of the panorama image of each frame.