IMAGE RECOGNITION APPARATUS, AND OPERATION DETERMINATION METHOD AND PROGRAM THEREFOR

• US 20110141009A1
• Filed: 06/02/2009
• Published: 06/16/2011
• Est. Priority Date: 06/03/2008
• Status: Active Grant

• Current Assignee: Shimane Prefecture government
• Original Assignee: Shimane Prefecture government
• Inventors: Izumi, Kenji
• Granted Patent: US 8,456,416 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/497 Partially or entirely replaying previous game actions
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/634 for replaying partially or entirely the game actions since the beginning of the game
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

An image for an operator is extracted, and an operation determination unit employs a relationship, relative to a marker of an operation input system, for the operator, who is standing behind the marker when viewed by a video camera. When a body part of the operator comes to the front of an operation plane, as viewed by the video camera, the operation determination unit determines that an action for an input operation has been started, and examines the shape and the movement of each individual body part (an open hand, two fingers that are being held up, etc.), and determines whether the shape or the movement is correlated with one of operations that are assumed in advance.

Claims

1. An image recognition apparatus comprising:

three-dimensional imaging means for capturing an image of at least part of an operator, and for generating stereoscopic image data;

a virtual operation plane forming reference, which is visible for an operator and is located in advance within an imaging range of the imaging means, so that the operator is able to identify a virtual operation plane in a predetermined space;

operation determination means for forming the virtual operation plane based on the operation plane forming reference that is read by the three-dimensional imaging means, permitting the three-dimensional imaging means to scan an action of the operator relative to the thus formed virtual operation plane, and for employing a positional relationship between a part of the operator and the virtual operation plane to determine whether or not the action indicates an operation instruction; and

signal output means for outputting a predetermined signal when it is determined that the action indicates an operation instruction.

2. The image recognition apparatus according to claim 1, wherein the virtual operation plane is a plane formed by an upward projection from the operation plane forming reference.

3. The image recognition apparatus according to claim 2, further comprising:

action limiting means for restricting the operator on an action on the operator side along the operation plane forming reference.

4. The image recognition apparatus according to claim 1, wherein the virtual operation plane is a plane formed by a projection upward from a predetermined position, on the side opposite the operator side, along the operation plane forming reference.

5. The image recognition apparatus according to one of claims 1 to 4, wherein when one part of the operator is closer to the three-dimensional imaging means than to the virtual operation plane, the operation determination means determines that the action indicates an operation instruction.

6. The image recognition apparatus according to one of claims 1 to 4, wherein the operation determination means examines the shape or the action of a specific part of the operator that is closer to the three-dimensional imaging means than to the virtual operation plane, and determines that an operation correlated with the shape or the action is currently being performed.

7. The image recognition apparatus according to claim 6, wherein the operation determination means searches storage means wherein operation contents are stored in advance in correlation with the shapes or actions of one part of the operator, and determines that an operation that matches or is correlated with the shape or the action is an operation to be entered.

8. The image recognition apparatus according to one of claims 1 to 7, wherein the operation plane forming reference is indicated on the surface of a floor where the operator is located.

9. The image recognition apparatus according to one of claims 1 to 7, further comprising:

a solid object located between the operator and the three-dimensional imaging means, wherein the operation plane forming reference is a predetermined outer edge of the solid object.

10. The image recognition apparatus according to claim 9, wherein the solid object is a table.

11. The image recognition apparatus according to one of claims 1 to 10, wherein, for the operation plane forming reference, markers are provided on a predetermined screen recorded by the three-dimensional imaging means, so that data can be easily extracted from an image that is read.

12. The image recognition apparatus according to one of claims 1 to 11, further comprising image display means arranged opposite the operator, wherein the operation determination means displays current results for an operation determination on the image display means, so that the operator can obtain the determination results.

13. The image recognition apparatus according to one of claims 1 to 12, further comprising:

operation contents determination means for, when a gesture of the operator has occurred within one of two or more virtual operation layers that are areas defined based on the positional relationship relative to the virtual operation plane, determining the contents of an operation based on an operation type, allocated in advance to the virtual operation layer, and the gesture of the operator performed in the virtual operation layer.

14. An operation determination method, whereby an image recognition apparatus recognizes an image for an operator and determines contents of an operation, the method comprising:

a three-dimensional imaging step of reading a virtual operation plane forming reference, which is visible for an operator and is located in advance within an imaging range of predetermined imaging means, so that the operator is able to identify a virtual operation plane in a predetermined space, and also of capturing an image of at least part of an operator, within the imaging range of the predetermined imaging means, and generating stereoscopic image data

an operation determination step of forming the virtual operation plane based on the operation plane forming reference that is read by the three-dimensional imaging step, and employing a positional relationship between a part of the operator and the virtual operation plane to determine whether or not the action, obtained at the three-dimensional imaging step, indicates an operation instruction; and

a signal output step of outputting a predetermined signal when it is determined that the action indicates an operation instruction.

15. A program that permits an image recognition apparatus to perform an operation determination method, for recognizing an image for an operator and determining contents of an operation, the method comprising:

a three-dimensional imaging step of reading a virtual operation plane forming reference, which is visible for an operator and is located in advance within an imaging range of predetermined imaging means, so that the operator is able to identify a virtual operation plane in a predetermined space, and also of capturing an image of at least part of an operator, within the imaging range of the predetermined imaging means, and generating stereoscopic image data;

an operation determination step of forming the virtual operation plane based on the operation plane forming reference that is read by the three-dimensional imaging step, and employing a positional relationship between a part of the operator and the virtual operation plane to determine whether or not the action, obtained by the three-dimensional imaging means, indicates an operation instruction; and

a signal output step of outputting a predetermined signal when it is determined that the action indicates an operation instruction.

16. An image recognition apparatus comprising:

three-dimensional imaging means for reading an image of at least one part of an operator, and generating a three-dimensional image;

operation determination means for forming a virtual operation plane, which serves as an operation reference for the operator, by employing a pre-designated size, at a pre-designated position in an imaging range of the three-dimensional imaging means, and for permitting the three-dimensional imaging means to scan an action of the operator on the virtual operation plane that is formed, and employing a positional relationship of a part of the operator and the virtual operation plane to determine whether the action indicates an operation instruction;

image display means, visually recognizable by the operator, for employing the positional relationship, obtained by the operation determination means, between one part of the operator, on the opposite side of the three-dimensional imaging means, along the virtual operation plane, and the virtual operation plane, and calculating a distance between the two, and for displaying an indicator that changes in accordance with the distance in order to indicate an operation instruction to be determined; and

signal output means for outputting a predetermined signal when the action indicates an operation instruction.

17. The image recognition apparatus according to claim 16, wherein the indicator becomes smaller when the distance from the part of the operator to the virtual operation plane is reduced.

18. The image recognition apparatus according to claim 16, wherein a hue for the indicator is changed when the distance from the part of the operator to the virtual operation plane is reduced.

19. The image recognition apparatus according to claim 16, wherein transparency for the indicator is lowered when the distance from the part of the operator to the virtual operation plane is reduced.

20. The image recognition apparatus according to claim 16, wherein the contrast between the indicator and an image to be operated displayed on the image display means is increased when the distance from the part of the operator to the virtual operation plane is reduced.

21. The image recognition apparatus according to claim 16, wherein a shape for the indicator is changed when the distance from the part of the operator to the virtual operation plane is reduced.

22. The image recognition apparatus according to one of claims 16 to 21, wherein there are more than one part for the operator, and a plurality of indicators are displayed in correlation with the individual parts of the operator.

23. The image recognition apparatus according to one of claims 16 to 22, further comprising:

operation contents determination means for, when a gesture of the operator has occurred within one of two or more virtual operation layers that are areas defined based on the positional relationship relative to the virtual operation plane, determining the contents of an operation based on an operation type, allocated in advance to the virtual operation layer, and the gesture of the operator performed in the virtual operation layer.