Input system, information processing device, storage medium storing information processing program, and three-dimensional position calculation method

• US 8,845,426 B2
• Filed: 01/19/2012
• Issued: 09/30/2014
• Est. Priority Date: 04/07/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Ohta, Keizo, Musa, Kazuhiro
• Primary Examiner(s): Pierce, Damon
• Assistant Examiner(s): WONG, JEFFREY KEITH
• Application Number:
  US13/354,000
• Publication Number:
  US 20120258796A1
• Time in Patent Office: 985 Days
• Field of Search: 463 30- 32, 345/8, 345/419, 345/633
• US Class Current: 463/30
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2219/2024 Style variation

Abstract

An example input processing system includes a controller device and an information processing device. The controller device transmits data representing an input position on a predetermined input surface and a detection result of an inertia sensor unit to the information processing device. The information processing device calculates an attitude of the controller device based on the detection result of the inertia sensor unit, and sets a predetermined surface in a virtual space so that an attitude of the predetermined surface changes in accordance with the attitude of the controller device. A first image representing an area in the virtual space including a position of the surface is displayed on a predetermined display device. The information processing device calculates, as a position in the virtual space, a position on the surface determined by the input position.

Claims

1. An input system comprising a controller device and an information processing device for calculating a position in a three-dimensional virtual space based on an operation performed on the controller device, wherein:

the controller device comprises;

a position detection unit for detecting an input position on an input surface;

an inertia sensor unit; and

an operation data transmitter for transmitting, to the information processing device, operation data including data representing detection results of the position detection unit and the inertia sensor unit; and

the information processing device comprises a processing system including at least one processor, the processing system being configured to perform operations comprising;

calculating aspects of an attitude of the controller device based on the detection result of the inertia sensor unit;

setting a surface in the virtual space so that an attitude of the virtual space surface changes in accordance with the attitude of the controller device;

generating a first image which represents an area in the virtual space including a position of the virtual space surface;

outputting the first image to a display device; and

calculating, as the position in the virtual space, a position on the virtual space surface determined in accordance with the input position.

2. The input system according to claim 1, wherein the processing system is further configured to generate a predetermined image at a position on the first image corresponding to the calculated position.

3. The input system according to claim 2, wherein the predetermined image comprises an image representing a path extending through a series of positions which are determined in the virtual space by a series of input positions on the input surface.

4. The input system according to claim 1, wherein the processing system is further configured to arrange an object at the calculated position.

5. The input system according to claim 4, wherein the processing system is further configured to move the object based on an operation performed using the controller device.

6. The input system according to claim 5, wherein the processing system is further configured to move the object based on the attitude of the controller device if a condition is satisfied for the operation performed on the controller device, and to not move the object based on the attitude of the controller device if the condition is not satisfied.

7. The input system according to claim 1, wherein:

the controller device further comprises a display unit;

the position detection unit comprises a touch panel provided on a display screen of the display unit;

the processing system of the information processing device is further configured to;

generate a second image which represents an area in the virtual space including a position of the virtual space surface; and

calculate, as the position in the virtual space, a position corresponding to the input position on the display screen; and

the information processing device further comprises a transmitter for transmitting the second image to the controller device.

8. The input system according to claim 7, wherein the processing system is further configured to generate, as the second image, an image representing an area in the virtual space in which the virtual space surface is set.

9. The input system according to claim 8, wherein the processing system is further configured to:

generate the first image using a first virtual camera whose viewing direction is set independently of the attitude of the virtual space surface; and

generate the second image using a second virtual camera whose viewing direction is set to be generally perpendicular to the virtual space surface.

10. The input system according to claim 1, wherein the processing system is further configured to control the attitude of the virtual space surface so that the attitude of the virtual space surface corresponds to the attitude of the controller device.

11. The input system according to claim 1, wherein the processing system is further configured to control the attitude of the virtual space surface so that the attitude of the virtual space surface corresponds to the attitude of the controller device, and control a position of the virtual space surface in accordance with the attitude of the controller device.

12. The input system according to claim 11, wherein the processing system is further configured to control the virtual space surface so that the virtual space surface moves in an arc about a predetermined position in the virtual space in accordance with the attitude of the controller device.

13. The input system according to claim 1, wherein the processing system is further configured to move the virtual space surface based on a direction input operation performed on the controller device.

14. The input system according to claim 1, wherein the processing system is further configured to perform game process using, as an input, the calculated position.

15. An information processing system for calculating a position in a three-dimensional virtual space based on an operation performed on an operation unit, the information processing system comprising:

a position detector for detecting an input position on an input surface provided on the operation unit;

a sensor for detecting a physical quantity for calculating aspects of an attitude of the operation unit; and

a processing system comprising at least one processor, the processing system being configured to perform operations comprising;

obtaining operation data including data representing detection results of the position detector and the sensor unit;

calculating aspects of an attitude of the operation unit based on the physical quantity;

setting a surface in the virtual space based on the calculated attitude;

generating a first image which represents an area in the virtual space including a position of the virtual space surface;

outputting the first image to a display device; and

calculating, as the position in the virtual space, a position on the virtual space surface determined in accordance with the input position.

16. An information processing device for calculating a position in a three-dimensional virtual space based on an operation performed on a controller device, the information processing device comprising:

a processing system comprising at least one processor, the processing system being configured to perform operations comprising;

obtaining operation data including data representing detection results of a touch panel and an inertial sensor of the controller device;

calculating aspects of an attitude of the controller device based on the detection result of the inertial sensor;

setting a surface in the virtual space so that an attitude of the virtual space surface changes in accordance with the attitude of the controller device;

calculating, as the position in the virtual space, a position on the virtual space surface determined in accordance with an input position on the touch panel;

arranging an object at the calculated position;

generating a first image representing an area in the virtual space including a position of the virtual space surface;

outputting the first image to a display device; and

generating, as a second image to be displayed on a display screen on which the touch panel is provided, an image representing an area in the virtual space in which the virtual space surface is set; and

a transmitter for transmitting the second image to the controller device.

17. A computer-readable non-transitory storage medium storing an information processing program for execution by a computer of an information processing device for calculating a position in a three-dimensional virtual space based on an operation performed on a controller device, the information processing program comprising instructions which, when executed, cause the computer to:

obtain operation data including data representing detection results of a touch panel and an inertial sensor of the controller device;

calculate aspects of an attitude of the controller device based on the detection result of the inertial sensor;

set a surface in the virtual space so that an attitude of the virtual space surface changes in accordance with the attitude of the controller device;

calculate, as the position in the virtual space, a position on the virtual space surface determined in accordance with an input position on the touch panel;

arrange an object at the calculated position;

generate, as a first image to be displayed on a display device, an image representing an area in the virtual space including a position of the virtual space surface; and

generate, as a second image to be displayed on a display screen on which the touch panel is provided, an image representing an area in the virtual space in which the virtual space surface is set.

18. A three-dimensional position calculation method for an input system including a controller device and an information processing device for calculating a position in a three-dimensional virtual space based on an operation performed on the controller device, the three-dimensional position calculation method comprising:

calculating aspects of an attitude of the controller device based on a detection result of an inertial sensor of the controller device;

setting a surface in the virtual space so that an attitude of the virtual space surface changes in accordance with the attitude of the controller device;

generating a first image representing an area in the virtual space including a position of the virtual space surface;

outputting the first image to a display device; and

calculating, as the position in the virtual space, a position on the virtual space surface determined in accordance with an input position on an input surface of the controller device.

19. The three-dimensional position calculation method according to claim 18, further comprising generating an image at a position on the first image corresponding to the calculated position on the virtual space surface.

20. The three-dimensional position calculation method according to claim 19, further comprising generating, as the image, an image representing a path extending through a series of positions which are determined in the virtual space by a series of input positions on the input surface.

21. The three-dimensional position calculation method according to claim 18, further comprising arranging an object at the calculated position on the virtual space surface.

22. The three-dimensional position calculation method according to claim 21, further comprising moving the object based on an operation performed on the controller device.

23. The three-dimensional position calculation method according to claim 22, further comprising moving the object based on the attitude of the controller device if a condition is satisfied for the operation performed on the controller device, and not moving the object based on the attitude of the controller device if the condition is not satisfied.

24. The three-dimensional position calculation method according to claim 18, wherein:

the controller device includes a display unit, and a touch panel provided on a display screen of the display unit, the method further comprising;

generating a second image representing an area in the virtual space including a position of the virtual space surface;

transmitting the second image to the controller device; and

calculating, as the position in the virtual space, a position corresponding to the input position on the display screen.

25. The three-dimensional position calculation method according to claim 24, further comprising generating, as the second image, an image representing an area in the virtual space in which the virtual space surface is set.

26. The three-dimensional position calculation method according to claim 25, further comprising:

generating the first image using a first virtual camera whose viewing direction is set independently of an attitude of the virtual space surface; and

generating the second image using a second virtual camera whose viewing direction is set to be generally perpendicular to the virtual space surface.

27. The three-dimensional position calculation method according to claim 18, further comprising controlling an attitude of the virtual space surface so that the attitude of the virtual space surface corresponds to the attitude of the controller device.

28. The three-dimensional position calculation method according to claim 18, further comprising controlling an attitude of the virtual space surface so that the attitude of the virtual space surface corresponds to the attitude of the controller device, and controlling a position of the virtual space surface in accordance with the attitude of the controller device.

29. The three-dimensional position calculation method according to claim 28, further comprising controlling the virtual space surface so that the virtual space surface moves in an arc about a predetermined position in the virtual space in accordance with the attitude of the controller device.

30. The three-dimensional position calculation method according to claim 18, further comprising moving the virtual space surface based on a direction input operation performed on the controller device.

31. The three-dimensional position calculation method according to claim 18, further comprising performing a game process using the calculated position on the virtual space surface as an input.

32. A three-dimensional position calculation method for an information processing system for calculating a position in a three-dimensional virtual space based on an operation performed on an operation unit, the three-dimensional position calculation method comprising:

obtaining operation data including data representing an input position on an input surface provided on the operation unit, and data representing a physical quantity for calculating an attitude of the operation unit;

calculating aspects of the attitude of the operation unit based on the physical quantity;

setting a surface in the virtual space based on the calculated attitude;

generating a first image representing an area in the virtual space including a position of the virtual space surface;

outputting the first image to a display device; and

calculating, as the position in the virtual space, a position on the virtual space surface determined in accordance with the input position.

33. An information processing device for use with a control device including a planar touch screen and an inertial sensor, the information processing device including:

a communication module for receiving, from the control device, touch screen input data corresponding to an input supplied to the touch screen and inertial sensor data corresponding to aspects of an attitude of the control device; and

a processing system including at least one processor, the processing system being configured to set a surface in a virtual space based on the aspects of an attitude of the control device indicated by the inertial sensor data and to determine a position in the virtual space in accordance with an input to the touch screen indicated by the touch screen input data.