Image generation system, image generation method, and information storage medium

• US 8,998,718 B2
• Filed: 03/30/2011
• Issued: 04/07/2015
• Est. Priority Date: 03/31/2010
• Status: Expired due to Fees

• Current Assignee: Bandai Namco Entertainment Inc. (BANDAI NAMCO Holdings Inc.)
• Original Assignee: Namco Bandai Games Inc.
• Inventors: Inagaki, Taketoshi, Yamaguchi, Kentaro, Sakakibara, Tadashi
• Primary Examiner(s): RENWICK, REGINALD A
• Application Number:
  US13/075,497
• Publication Number:
  US 20110244959A1
• Time in Patent Office: 1,469 Days
• Field of Search: 463/37
• US Class Current: 463/37
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An image generation system includes an image information acquisition section that acquires image information from an image sensor, a grasp motion determination section that determines whether or not a player has made a grasp motion with one hand that is a right hand or a left hand based on the image information, a grasp-linked motion determination section that determines whether or not the player has made a grasp-linked motion that follows the grasp motion with the other hand based on the image information when the grasp motion determination section has determined that the player has made the grasp motion with the one hand, and an image generation section that generates an image based on the determination results of the grasp motion and the grasp-linked motion.

Claims

1. An image generation system comprising:

a memory configured to store instructions; and

at least one processor configured to execute the instructions to;

acquire image information from an image sensor;

determine whether or not a player has made a grasp preparation motion based on the image information, the processor processing an image corresponding to a hand of the player that has made the grasp preparation motion when the processor has determined that the player has made the grasp preparation motion;

determine whether or not the player has made a grasp motion that follows the grasp preparation motion with one hand that is a right hand or a left hand based on the image information;

determine whether or not the player has made a grasp-linked motion with the other hand based on the image information when the at least one processor has determined that the player has made the grasp motion with the one hand, the grasp-linked motion being a motion that follows the grasp motion; and

generate an image based on the determination results of the grasp motion and the grasp-linked motion.

2. The image generation system as defined in claim 1,

the at least one processor determining the grasp-linked motion among first to Mth grasp-linked motions (M is an integer equal to or larger than 2) that has been made by the player with the other hand based on the image information from the image sensor.

3. The image generation system as defined in claim 2,

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand based on information about an image area corresponding to the other hand.

4. The image generation system as defined in claim 3,

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand based on the information about the image area within a given period or at a given timing.

5. The image generation system as defined in claim 3,

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand based on a size of the image area.

6. The image generation system as defined in claim 5,

the at least one processor acquiring color image information as the image information, the color image information being image information in which a color value is set to each pixel position;

and

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand based on the size of the image area corresponding to the other hand in the color image information.

7. The image generation system as defined in claim 5,

the at least one processor acquiring depth image information as the image information, the depth information being image information in which a depth value is set to each pixel position;

and

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand based on the size of the image area corresponding to the other hand in the depth information.

8. The image generation system as defined in claim 5,

the at least one processor changing a determination threshold value used when determining the size of the image area based on information about a distance between a display section that displays the generated image and the player.

9. The image generation system as defined in claim 2,

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand based on image information about a recognition member that is fitted to the hand of the player.

10. The image generation system as defined in claim 2,

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand by a shape recognition process on image information about the other hand.

11. The image generation system as defined in claim 2, the processor further configured to execute instructions to:

acquire skeleton information based on the image information from the image sensor, the skeleton information specifying a motion of the player viewed from the image sensor,

the at least one processor determining the grasp-linked motion among the first to Mth grasp-linked motions that has been made by the player with the other hand based on the skeleton information.

12. The image generation system as defined in claim 1, the processor further configured to execute instructions to:

acquire skeleton information based on the image information from the image sensor, the skeleton information specifying a motion of the player viewed from the image sensor,

the at least one processor determining whether or not the player has made the grasp-linked motion with the other hand based on the skeleton information.

13. The image generation system as defined in claim 1, the processor further configured to execute instructions to:

acquire skeleton information based on the image information from the image sensor, the skeleton information specifying a motion of the player viewed from the image sensor, the at least one processor setting an image processing area for determining whether or not the player has made the grasp motion or the grasp-linked motion based on the acquired skeleton information.

14. The image generation system as defined in claim 1, the processor further configured to execute instructions to:

control a character corresponding to the player,

the at least one processor generating an image that indicates a state in which the character makes the grasp motion and the grasp-linked motion when it has been determined that the player has made the grasp motion and the grasp-linked motion.

15. The image generation system as defined in claim 1,

the at least one processor determining whether or not the player has made an attack motion with the other hand as the grasp-linked motion based on the image information from the image sensor.

16. The image generation system as defined in claim 15, the at least one processor further configured to execute instructions to:

set an attack calculation parameter of the attack motion with the other hand based on the image information from the image sensor, and performs an attack calculation process based on the attack calculation parameter.

17. The image generation system as defined in claim 1,

the at least one processor determining whether or not the player has made the grasp motion based on information about an image area corresponding to the hand of the player that has been determined to have made the grasp preparation motion.

18. The image generation system as defined in claim 17,

the at least one processor determining whether or not the player has made the grasp motion based on the information about the image area within a given period or at a given timing.

19. The image generation system as defined in claim 17,

the at least one processor determining whether or not the player has made the grasp motion based on a size of the image area.

20. The image generation system as defined in claim 19,

the at least one processor changing a determination threshold value used when determining the size of the image area based on information about a distance between a display section that displays the generated image and the player.

21. The image generation system as defined in claim 1,

the at least one processor determining whether or not the player has made the grasp motion based on image information about a recognition member that is fitted to the hand of the player.

22. The image generation system as defined in claim 1,

the at least one processor determining whether or not the player has made the grasp motion by a shape recognition process on image information about the hand of the player that has been determined to have made the grasp preparation motion.

23. The image generation system of defined in claim 1, wherein whether the player has made the grasp-linked motion with the other hand is determined by specifying an area corresponding to the other hand based on the skeleton information and subjecting the specified area to image processing.

24. An image generation method comprising:

acquiring image information from an image sensor using at least one processor;

determining whether or not a player has made a grasp preparation motion based on the image information, the processor processing an image corresponding to a hand of the player that has made the grasp preparation motion when the processor has determined that the player has made the grasp preparation motion;

determining whether or not the player has made a grasp motion that follows the grasp preparation motion with one hand that is a right hand or a left hand based on the image information;

determining whether or not the player has made a grasp-linked motion with the other hand based on the image information when it has been determined that the player has made the grasp motion with the one hand, the grasp-linked motion being a motion that follows the grasp motion; and

generating an image based on the determination results of the grasp motion and the grasp-linked motion.

25. The image generation method as defined in claim 24, wherein the determining step includes determining whether the player has made the grasp-linked motion with the other hand by specifying an area corresponding to the other hand based on the skeleton information and subjecting the specified area to image processing.

26. A non-transitory computer-readable information storage medium storing a program that causes a computer to execute image generation steps, comprising:

acquiring image information from an image sensor;

determining whether or not a player has made a grasp preparation motion based on the image information, the processor processing an image corresponding to a hand of the player that has made the grasp preparation motion when the processor has determined that the player has made the grasp preparation motion;

determining whether or not the player has made a grasp motion that follows the grasp preparation motion with one hand that is a right hand or a left hand based on the image information;

determining whether or not the player has made a grasp-linked motion with the other hand based on the image information when it has been determined that the player has made the grasp motion with the one hand, the grasp-linked motion being a motion that follows the grasp motion; and

generating an image based on the determination results of the grasp motion and the grasp-linked motion.

27. The non-transitory computer readable information storage medium as defined in claim 26, wherein the determining step includes determining whether the player has made the grasp-linked motion with the other hand by specifying an area corresponding to the other hand based on the skeleton information and subjecting the specified area to image processing.

28. An image generation system comprising:

a memory configured to store instructions; and

at least one processor configured to execute the instructions to;

acquire image information from an image sensor;

determine whether or not a player has made a grasp motion with one hand that is a right hand or a left hand based on the image information;

determine whether or not the player has made a grasp-linked motion with the other hand based on the image information when the at least one processor has determined that the player has made the grasp motion with the one hand, the grasp-linked motion being a motion that follows the grasp motion and being formed by;

(i) making the other hand flat, or (ii) making the other hand clench into a fist; and

generate an image based on the determination results of the grasp motion and the grasp-linked motion.

29. An image generation method comprising:

acquiring image information from an image sensor using at least one processor;

determining whether or not a player has made a grasp motion with one hand that is a right hand or a left hand based on the image information;

determining whether or not the player has made a grasp-linked motion with the other hand based on the image information when it has been determined that the player has made the grasp motion with the one hand, the grasp-linked motion being a motion that follows the grasp motion and being formed by;

(i) making the other hand flat, or (ii) making the other hand clench into a fist; and

generating an image based on the determination results of the grasp motion and the grasp-linked motion.

30. A non-transitory computer-readable information storage medium storing a program that causes a computer to execute image generation steps, comprising:

acquiring image information from an image sensor;

determining whether or not a player has made a grasp motion with one hand that is a right hand or a left hand based on the image information;

determining whether or not the player has made a grasp-linked motion with the other hand based on the image information when it has been determined that the player has made the grasp motion with the one hand, the grasp-linked motion being a motion that follows the grasp motion and being formed by;

(i) making the other hand flat, or (ii) making the other hand clench into a fist; and

generating an image based on the determination results of the grasp motion and the grasp-linked motion.