GAME SYSTEM, GAME DEVICE, STORAGE MEDIUM STORING GAME PROGRAM, AND IMAGE GENERATION METHOD

• US 20120309523A1
• Filed: 02/08/2012
• Published: 12/06/2012
• Est. Priority Date: 06/02/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: YAMASHITA, Yoshikazu
• Granted Patent: US 8,747,222 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/6669 using a plurality of virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character change rooms
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player

Abstract

An example game system includes a portable display that includes an image-capturing unit, and a display unit. The game device obtains first operation data representing an operation performed on a controller device, second operation data representing an operation performed on the portable display, and data of a image captured by the image-capturing unit. In a virtual space, an action of a first control object is controlled based on first operation data and an action of a second control object is controlled based on second operation data. The game device generates a first game image including a game image that represents the virtual space and corresponds to the first control object, and at least a portion of the captured image. The first game image is displayed on a predetermined display. The game device generates a second game image corresponding to the second control object, and displays the second game image.

Claims

1. A game system comprising a controller device, a portable display device, and a game device, wherein:

the portable display device comprises an image-capturing unit, and a display unit; and

the game device comprises;

a data obtaining unit for obtaining first operation data representing an operation performed on the controller device, second operation data representing an operation performed on the portable display device, and data of a captured image captured by the image-capturing unit;

an action control unit for controlling an action of a first control object based on the first operation data and an action of a second control object based on the second operation data in a virtual space;

a first image generation unit for generating a first game image which includes a game image that represents the virtual space and corresponds to the first control object, and at least a portion of the captured image;

a second image generation unit for generating a second game image that represents the virtual space and corresponds to the second control object;

a first image output unit for outputting the first game image to a predetermined display device separate from the portable display device; and

a second image output unit for outputting the second game image to the portable display device.

2. The game system according to claim 1, wherein:

the game device further comprises a face recognition unit for performing a recognition process for recognizing a face in the captured image; and

the first image generation unit generates the first game image so that the first game image includes a portion of the captured image that includes an image of the face recognized by the face recognition unit.

3. The game system according to claim 2, wherein the first image generation unit performs a predetermined image process on the image of the captured image that includes the face recognized by the face recognition unit, thereby generating the first game image including the processed image.

4. The game system according to claim 2, wherein:

the face recognition unit performs the recognition process in response to the obtainment of the captured image by the data obtaining unit;

the first image generation unit generates a first game image including the image of the recognized face in response to the recognition of the face by the recognition process; and

the first image output unit outputs the first game image to the predetermined display device in response to the generation of the first game image.

5. The game system according to claim 4, wherein:

the data obtaining unit successively obtains data of the captured image; and

the first image generation unit updates the image of the recognized face with a frequency that is less than that with which the data obtaining unit obtains data of the captured image.

6. The game system according to claim 1, wherein:

the game device further comprises;

a first camera setting unit for setting a first virtual camera corresponding to the first control object in the virtual space; and

a second camera setting unit for setting a second virtual camera corresponding to the second control object in the virtual space;

the first image generation unit generates, as a game image corresponding to the first control object, a game image representing the virtual space as viewed from the first virtual camera; and

the second image generation unit generates, as a game image corresponding to the second control object, a game image representing the virtual space as viewed from the second virtual camera.

7. The game system according to claim 6, wherein:

the action control unit performs a control of moving a first object arranged in the virtual space, as the first control object, based on the first operation data; and

the first camera setting unit sets the first virtual camera so that the first object is included in a viewing field range of the first virtual camera.

8. The game system according to claim 6, wherein:

the action control unit performs a control of moving a second object arranged in the virtual space, as the second control object, based on the second operation data; and

the second camera setting unit sets the second virtual camera so that the second object is included in a viewing field range of the second virtual camera.

9. The game system according to claim 1, wherein:

the game device further comprises an image storage unit for storing a captured image obtained at a predetermined point in time in a predetermined storage unit; and

the first image generation unit generates a first game image including at least a portion of the captured image stored in the storage unit.

10. The game system according to claim 1, wherein:

the game device comprises a plurality of controller devices;

the action control unit controls an action of each of the plurality of first control objects based on corresponding first operation data; and

the first image generation unit generates a first game image including a plurality of game images corresponding respectively to the plurality of first control objects and at least a portion of the captured image.

11. The game system according to claim 10, wherein:

the game device further comprises a determination unit for determining whether a predetermined game condition has been satisfied for each of the first control objects; and

the first image generation unit generates a first game image obtained by superimposing at least a portion of the captured image on the game image corresponding to each first control object for which the predetermined game condition has been satisfied.

12. A game system comprising a portable display device and a game device, wherein:

the portable display device comprises an image-capturing unit, and a display unit; and

the game device comprises;

a data obtaining unit for obtaining operation data representing an operation performed on the portable display device, and a captured image captured by the image-capturing unit;

an action control unit for controlling an action of a predetermined control object based on the operation data in a virtual space;

a first image generation unit for generating a first game image which includes a game image that represents the virtual space and corresponds to the control object, and at least a portion of the captured image;

a second image generation unit for generating a second game image that represents the virtual space and corresponds to the control object;

a first image output unit for outputting the first game image to a predetermined display device separate from the portable display device; and

a second image output unit for outputting the second game image to the portable display device.

13. An image generation method to be carried in a game system comprising a controller device, a portable display device, and a game device, wherein:

the portable display device comprises an image-capturing unit, and a display unit, the method comprising;

obtaining first operation data representing an operation performed on the controller device, second operation data representing an operation performed on the portable display device, and data of a captured image captured by the image-capturing unit;

controlling an action of a first control object based on the first operation data and controlling an action of a second control object based on the second operation data in a virtual space;

generating a first game image which includes a game image that represents the virtual space and corresponds to the first control object, and at least a portion of the captured image;

generating a second game image that represents the virtual space and corresponds to the second control object;

displaying the first game image on a predetermined display device separate from the portable display device; and

displaying the second game image on a display unit of the portable display device.

14. The image generation method according to claim 13, further comprising:

performing a recognition process of recognizing a face in the captured image, wherein;

the first game image is generated so that the first game image includes a portion of the captured image that includes an image of the face recognized by the recognition process.

15. The image generation method according to claim 14, wherein a predetermined image process is performed on the image of the captured image that includes the face recognized by the recognition process, thereby generating the first game image including the processed image.

16. The image generation method according to claim 14, wherein:

the recognition process is performed in response to the obtainment of the captured image;

the first game image including the image of the recognized face in response to the recognition of the face by the recognition process; and

the first game image is displayed on the predetermined display device in response to the generation of the first game image.

17. The image generation method according to claim 16, wherein:

data of the captured image is obtained successively; and

the image of the recognized face is updated with a frequency that is less than that with which data of the captured image is obtained.

18. The image generation method according to claim 13, further comprising:

setting a first virtual camera corresponding to the first control object in the virtual space; and

setting a second virtual camera corresponding to the second control object in the virtual space, wherein;

a game image representing the virtual space as viewed from the first virtual camera is generated as a game image corresponding to the first control object; and

a game image representing the virtual space as viewed from the second virtual camera is generated as a game image corresponding to the second control object.

19. The image generation method according to claim 18, wherein:

a first object arranged in the virtual space is controlled, as the first control object, so as to move based on the first operation data; and

the first virtual camera is set so that the first object is included in a viewing field range of the first virtual camera.

20. The image generation method according to claim 18, wherein:

a second object arranged in the virtual space is controlled, as the second control object, so as to move based on the second operation data; and

the second virtual camera is set so that the second object is included in a viewing field range of the second virtual camera.

21. The image generation method according to claim 13, further comprising:

storing a captured image obtained at a predetermined point in time in a predetermined storage unit, wherein;

a first game image is generated including at least a portion of the captured image stored in the storage unit.

22. The image generation method according to claim 13, wherein:

the game device includes a plurality of controller devices;

an action of each of the plurality of first control objects is controlled based on corresponding first operation data in the action control step; and

a first game image is generated including a plurality of game images corresponding respectively to the plurality of first control objects and at least a portion of the captured image.

23. The image generation method according to claim 22, further comprising:

determining whether a predetermined game condition has been satisfied for each of the first control objects, wherein;

a first game image is generated which is obtained by superimposing at least a portion of the captured image on the game image corresponding to each first control object for which the predetermined game condition has been satisfied.

24. An image generation method to be carried in a game system comprising a portable display device and a game device, wherein:

the portable display device comprises an image-capturing unit, and a display unit, the image generation method comprising;

obtaining operation data representing an operation performed on the portable display device and a captured image captured by the image-capturing unit;

controlling an action of a predetermined control object based on the operation data in a virtual space;

generating a first game image which includes a game image that represents the virtual space and corresponds to the control object, and at least a portion of the captured image;

generating a second game image that represents the virtual space and corresponds to the control object;

displaying the first game image on a predetermined display device separate from the portable display device; and

displaying the second game image on a display unit of the portable display device.

25. A game device capable of communicating with a controller device and a portable display device, the game device comprising:

a data obtaining unit for obtaining first operation data representing an operation performed on the controller device, second operation data representing an operation performed on the portable display device, and data of a captured image captured by an image-capturing unit of the portable display device;

an action control unit for controlling an action of a first control object based on the first operation data and controlling an action of a second control object based on the second operation data in a virtual space;

a first image generation unit for generating a first game image which includes a game image that represents the virtual space and corresponds to the first control object, and at least a portion of the captured image;

a second image generation unit for generating a second game image that represents the virtual space and corresponds to the second control object;

a first image output unit for outputting the first game image to a predetermined display device separate from the portable display device; and

a second image output unit for outputting the second game image to the portable display device.

26. A game device capable of communicating with a portable display device, the game device comprising:

a data obtaining unit for obtaining operation data representing an operation performed on the portable display device and a captured image captured by an image-capturing unit of the portable display device;

an action control unit for controlling an action of a predetermined control object based on the operation data in a virtual space;

a first image generation unit for generating a first game image which includes a game image that represents the virtual space and corresponds to the control object, and at least a portion of the captured image;

a second image generation unit for generating a second game image that represents the virtual space and corresponds to the control object;

a first image output unit for outputting the first game image to a predetermined display device separate from the portable display device; and

a second image output unit for outputting the second game image to the portable display device.

27. A non-transitory computer-readable storage medium storing a game program to be executed by a computer of a game device capable of communicating with a controller device and a portable display device, the game program causing the computer to execute:

obtaining first operation data representing an operation performed on the controller device, second operation data representing an operation performed on the portable display device, and data of a captured image captured by an image-capturing unit of the portable display device;

controlling an action of a first control object based on the first operation data and controlling an action of a second control object based on the second operation data in a virtual space;

generating a first game image to be displayed on a predetermined display device separate from the portable display device, the first game image including a game image that represents the virtual space and corresponds to the first control object, and at least a portion of the captured image; and

generating a second game image to be displayed on the portable display device, the second game image representing the virtual space and corresponding to the second control object.

28. A non-transitory computer-readable storage medium storing a game program to be executed by a computer of a game device capable of communicating with a portable display device, the game program causing the computer to execute:

obtaining operation data representing an operation performed on the portable display device, and a captured image captured by an image-capturing unit of the portable display device;

controlling an action of a predetermined control object based on the operation data in a virtual space;

generating a first game image to be displayed on a predetermined display device separate from the portable display device, the first game image including a game image that represents the virtual space and corresponds to the control object, and at least a portion of the captured image; and

generating a second game image to be displayed on the portable display device, the second game image representing the virtual space and corresponding to the control object.