HAND POINTING DEVICE

• US 20010043719A1
• Filed: 03/18/1998
• Published: 11/22/2001
• Est. Priority Date: 03/21/1997
• Status: Active Grant

• Current Assignee: Takenaka Corporation
• Original Assignee: Takenaka Corporation
• Inventors: HARAKAWA, KENICHI, UNNO, KENICHI, IGAWA, NORIO
• Granted Patent: US 6,385,331 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Over an information input space to which an information inputting person comes, a pair of near-infrared light illuminators are arranged in such a manner that the illumination ranges thereof are adjusted so as to illuminate the information inputting person from different directions. A pair of near-infrared-light-sensitive video cameras are also arranged in different positions so as to correspond to the illuminators. The image pickup range of the video cameras is adjusted so that it is out of the range on the floor surface illuminated by the corresponding illuminator, while the information inputting person is within the image pickup range. A controller allows one illuminator at a time to be switched on/off. An image of the information inputting person is picked up by the video camera corresponding to the switched-on illuminator. The information inputting person is extracted based on the images picked up by the video cameras, whereby the position or direction pointed to by the information inputting person is determined.

Claims

1. A hand pointing apparatus comprising:

illuminating means for illuminating a person to be recognized;

a plurality of image pickup means, located in different positions, wherein an image pickup range is adjusted for each image so that said person to be recognized, who is illuminated by said illuminating means, may be within the image pickup range and an illuminated range on a floor surface, which is illuminated by said illuminating means, may be out of the image pickup range; and

determining means for extracting an image part corresponding to said person to be recognized from a plurality of images based on a plurality of images of situations picked up by said plurality of image pickup means, the situations being indicative of said person to be recognized pointing to either a specific position or a specific direction, and for determining either the position or the direction pointed to by said person to be recognized.

2. A hand pointing apparatus comprising:

a plurality of illuminating means for illuminating a person to be recognized from different directions;

a plurality of image pickup means, located in different positions corresponding to each of said plurality of illuminating means, wherein an image pickup range is adjusted so that said person to be recognized, who is illuminated by said corresponding illuminating means, may be within the image pickup range and an illuminated range on a floor surface, which is illuminated by said corresponding illuminating means, may be out of the image pickup range;

controlling means for switching on/off said plurality of illuminating means one by one in sequence, and for controlling so as to pickup an image of said person to be recognized pointing to either a specific position or a specific direction by said image pickup means corresponding to said switched-on illuminating means; and

determining means for extracting an image part corresponding to said person to be recognized from a plurality of images based on a plurality of images picked up by said plurality of image pickup means, and for determining either the position or the direction pointed to by said person to be recognized.

3. A hand pointing apparatus comprising:

a plurality of illuminating means for illuminating a person to be recognized from different directions;

at least one image pickup means for picking up an image of said person to be recognized, who is illuminated by said illuminating means;

discriminating means for switching on/off said plurality of illuminating means one by one in sequence, for comparing a plurality of images of said person to be recognized pointing to either a specific position or a specific direction picked up by the same image pickup means during the switching on of said plurality of illuminating means, and for discriminating between an image part corresponding to said person to be recognized and an image part other than the image part corresponding to said person to be recognized in said plurality of images for at least one image pickup means; and

determining means for extracting the image part corresponding to said person to be recognized from said plurality of images picked up by said image pickup means based on a result of a discrimination by said discriminating means, and for determining either the position or the direction pointed to by said person to be recognized.

4. A hand pointing apparatus comprising:

illuminating means for illuminating a person to be recognized;

a plurality of image pickup means for picking up an image of said person to be recognized, who is illuminated by said illuminating means from different directions;

determining means for extracting an image part corresponding to said person to be recognized from a plurality of images based on a plurality of images of situations picked up by said plurality of image pickup means, the situations being indicative of said person to be recognized pointing to either a specific position or a specific direction, and for determining either the position or the direction pointed to by said person to be recognized; and

preventing means for preventing an object which is not the subject to be recognized from remaining on the floor surface around said person to be recognized.

5. A hand pointing apparatus comprising:

illuminating means for illuminating a person to be recognized who arrives at a predetermined place;

a plurality of image pickup means for picking up an image of said person to be recognized, who is illuminated by said illuminating means from different directions;

storing means for storing information for corresponding three-dimensional coordinates of a plurality of virtual points, positioned near said predetermined place, to the positions of said plurality of virtual points on said plurality of images picked up by said plurality of image pickup means; and

determining means;

for extracting an image part corresponding to said person to be recognized from a plurality of images based on a plurality of images of situations picked up by said plurality of image pickup means, the situations being indicative of said person to be recognized pointing to either a specific position or a specific direction;

for determining the position of a feature point of said person to be recognized in each of said images;

for determining the three-dimensional coordinates of the feature point based on the determined position of the feature point and the information stored in said storing means; and

for determining either the position or the direction pointed to by said person to be recognized based on the determined three-dimensional coordinates of the feature point.

6. A hand pointing apparatus according to

claim 5

, wherein said storing means stores the information for corresponding the three-dimensional coordinates of a multiplicity of virtual points uniformly spaced in a lattice arrangement near said predetermined place, to the positions of said multiplicity of virtual points on said plurality of images picked up by said plurality of image pickup means.

7. A hand pointing apparatus according to

claim 6

, wherein said determining means determines the position of the feature point of said person to be recognized in said images, extracts from said images the virtual points positioned in a region within a predetermined range including said feature point on said images from said images, and determines the three-dimensional coordinates of said feature point based on the three-dimensional coordinates of the common virtual points extracted from said images.

8. A hand pointing apparatus according to

claim 5

further comprising:

generating means for allowing said plurality of image pickup means to pick up images of the situations where markers are positioned in the positions of said virtual points, for generating the information for corresponding the three-dimensional coordinates of said virtual points to the positions of said virtual points on said images, based on the three-dimensional coordinates of said virtual points and the marker positions on said images picked up by said plurality of image pickup means, and for allowing said storing means to store the generated information.

9. A hand pointing apparatus according to

claim 6

further comprising:

generating means for allowing said plurality of image pickup means to pick up images of the situations where markers are positioned in the positions of said virtual points, for generating the information for corresponding the three-dimensional coordinates of said virtual points to the positions of said virtual points on said images, based on the three-dimensional coordinates of said virtual points and the marker positions on said images picked up by said plurality of image pickup means, and for allowing said storing means to store the generated information.

10. A hand pointing apparatus comprising:

illuminating means for illuminating a person to be recognized;

a plurality of image pickup means for picking up images of said person to be recognized, who is illuminated by said illuminating means from different directions;

determining means for extracting an image part corresponding to said person to be recognized from a plurality of images based on a plurality of images of situations picked up by said plurality of image pickup means, the situations being indicative of said person to be recognized pointing to either a specific position or a specific direction, and for determining either the position or the direction pointed to by said person to be recognized;

first detecting means for extracting the image part corresponding to a predetermined part of a body of said person to be recognized from said plurality of images, and for detecting a change in any one of either an area of the extracted image part, an outline of the extracted image part, or a length of an outline of the extracted image part; and

processing means for executing a predetermined processing when said change is detected by said first detecting means.

17. A hand pointing apparatus according to

claim 10

further comprising:

second detecting means for extracting the image part corresponding to the arm of said person to be recognized from said plurality of images, and for detecting whether or not the arm of said person to be recognized is lowered, wherein said processing means continues the current state when said second detecting means detects that the arm of said person to be recognized is lowered.

11. A hand pointing apparatus comprising:

illuminating means for illuminating a person to be recognized;

a plurality of image pickup means for picking up images of said person to be recognized, who is illuminated by said illuminating means from different directions;

determining means for extracting an image part corresponding to said person to be recognized from a plurality of images based on a plurality of images of situations picked up by said plurality of image pickup means, the situations being indicative of said person to be recognized pointing to either a specific position or a specific direction, for determining the three-dimensional coordinates of the feature point whose position is changed when said person to be recognized bends or extends an arm, and the three-dimensional coordinates of a reference point whose position is not changed even if said person to be recognized bends and extends an arm, and for determining either the position or the direction pointed to by said person to be recognized based on the three-dimensional coordinates of the feature point and the three-dimensional coordinates of the reference point; and

processing means for calculating the distance between said reference point and said feature point and for executing a predetermined processing based on the change in the distance between said reference point and said feature point.

12. A hand pointing apparatus according to

claim 11

, wherein said processing means performs a first predetermined processing when the distance between said reference point and said feature point is increased, and performs a second predetermined processing differing from said first predetermined processing when the distance between said reference point and said feature point is reduced.

13. A hand pointing apparatus according to

claim 11

, wherein said processing means detects a rate of change in the distance between said reference point and said feature point, and executes a predetermined processing when the detected rate of change is a threshold value or more.

14. A hand pointing apparatus according to

claim 12

, wherein said processing means detects a rate of change in the distance between said reference point and said feature point, and executes a predetermined processing when the detected rate of change is a threshold value or more.

15. A hand pointing apparatus according to

claim 13

further comprising:

threshold value setting means for requesting said person to be recognized to bend or extend an arm in order to allow said processing means to perform a predetermined processing, and for setting said threshold value based on the rate of the change in the distance between said reference point and said feature point when said person to be recognized bends or extends an arm.

16. A hand pointing apparatus according to

claim 14

further comprising threshold value setting means for requesting said person to be recognized to bend or extend an arm in order to allow said processing means to perform a predetermined processing, and for setting said threshold value based on the rate of the change in the distance between said reference point and said feature point when said person to be recognized bends or extends an arm.

18. A hand pointing apparatus according to

claim 11

further comprising:

second detecting means for extracting the image part corresponding to the arm of said person to be recognized from said plurality of images, and for detecting whether or not the arm of said person to be recognized is lowered, wherein said processing means continues the current state when said second detecting means detects that the arm of said person to be recognized is lowered.