Hand and indicating-point positioning method and hand gesture determining method used in human-computer interaction system

• US 8,970,696 B2
• Filed: 08/23/2011
• Issued: 03/03/2015
• Est. Priority Date: 09/13/2010
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Xiong, Huaixin
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): BRUMFIELD, SHANIKA M
• Application Number:
  US13/215,385
• Publication Number:
  US 20120062736A1
• Time in Patent Office: 1,288 Days
• Field of Search: 345/156, 348/143, 348/333, 382/103, 382/104, 382/156, 382/118, 382/291
• US Class Current: 348/143
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Disclosed are a hand positioning method and a human-computer interaction system. The method comprises a step of continuously capturing a current image so as to obtain a sequence of video images; a step of extracting a foreground image from each of the captured video images, and then carrying out binary processing so as to obtain a binary foreground image; a step of obtaining a vertex set of a minimum convex hull of the binary foreground image, and then creating areas of concern serving as candidate hand areas; and a step of extracting hand imaging features from the respective created areas of concern, and then determining a hand area from the candidate hand areas by carrying out pattern recognition based on the extracted hand imaging features.

Claims

1. A hand-positioning method used in a human-computer interaction system comprises:

continuously capturing a current image of a monitored display area so as to obtain a sequence of video images including a foreground object;

extracting a foreground image from each of the obtained video images, and then carrying out binary processing with regard to the extracted foreground image so as to obtain a binary foreground image;

obtaining a set of vertexes of a minimum convex hull of the binary foreground image, and then creating areas of concern serving as candidate hand areas by letting the respective vertexes of the minimum convex hull of the binary foreground image be central points of the areas of concern; and

extracting hand imaging features from the respective created areas of concern, and then determining a hand area from the candidate hand areas by carrying out pattern recognition based on the extracted hand imaging features, wherein

the binary processing carried out with regard to the extracted foreground image includes;

calculating an edge image of the extracted foreground image;

calculating a gradient image of the extracted foreground image, and then carrying out binary processing with regard to the gradient image so as to obtain a first binary result;

carrying out binary processing with regard to the extracted foreground image by using a predetermined threshold value so as to obtain a second binary result;

combining the edge image, the first binary result, and the second binary result by using an OR logical operation so as to obtain a new binary image; and

filling a closed area in the obtained new binary image.

2. The hand-positioning method according to claim 1, wherein:

each of the areas of concern is a rectangle or a circle just covering a hand of a user.

3. The hand-positioning method according to claim 1, wherein:

the pattern recognition is carried out by using a back propagation (BP) neural network approach, a support vector machine (SVM) approach, or a Boosting approach.

4. The hand-positioning method according to claim 1, wherein:

the hand imaging features include;

area, color, and edges based on pixel information, or shape, texture, and gradient direction histograms based on global information in the areas of concern; and

characters or their combination outside the areas of concern but having one-to-one relationships with the areas of concern, helpful to distinguish a hand from other parts of a body of a user.

5. A human-computer interaction system comprises:

circuitry configured to;

continuously capture a current image of a monitored display area so as to obtain a sequence of video images including a foreground object;

extract a foreground image from each of the obtained video images, and then carry out binary processing with regard to the extracted foreground image so as to obtain a binary foreground image;

obtain a set of vertexes of a minimum convex hull of the binary foreground image, and then create areas of concern serving as candidate hand areas by letting the respective vertexes of the minimum convex hull of the binary foreground image be central points of the areas of concern; and

extract hand imaging features from the respective created areas of concern, and then determine a hand area from the candidate hand areas by carrying out pattern recognition based on the extracted hand imaging features, wherein the circuitry is configured to carry out the binary processing with regard to the extracted foreground image by;

calculating an edge image of the extracted foreground image;

calculating a gradient image of the extracted foreground image, and then carrying out binary processing with regard to the gradient image so as to obtain a first binary result;

carrying out binary processing with regard to the extracted foreground image by using a predetermined threshold value so as to obtain a second binary result;

combining the edge image, the first binary result, and the second binary result by using an OR logical operation so as to obtain a new binary image; and

filling a closed area in the obtained new binary image.

6. The system according to claim 5, wherein:

each of the areas of concern is a rectangle or a circle just covering a hand of a user.

7. The system according to claim 5, wherein:

the pattern recognition is carried out by using a back propagation (BP) neural network approach, a support vector machine (SVM) approach, or a Boosting approach.

8. The system according to claim 5, wherein:

the hand imaging features include;

area, color, and edges based on pixel information, or shape, texture, and gradient direction histograms based on global information in the areas of concern; and

characters or their combination outside the areas of concern but having one-to-one relationships with the areas of concern, helpful to distinguish a hand from other parts of a body of a user.

9. A non-transitory computer-readable medium having machine-executable instructions for execution by a processing system, wherein, the machine-executable instructions are used for carrying out a hand-positioning method used in a human-computer interaction system, and the machine-executable instructions, when executed, cause the processing system to:

continuously capture a current image of a monitored display area so as to obtain a sequence of video images including a foreground object;

extract a foreground image from each of the obtained video images, and then carry out binary processing with regard to the extracted foreground image so as to obtain a binary foreground image;

obtain a set of vertexes of a minimum convex hull of the binary foreground image, and then create areas of concern serving as candidate hand areas by letting the respective vertexes of the minimum convex hull of the binary foreground image be central points of the areas of concern; and

extract hand imaging features from the respective created areas of concern, and then determine a hand area from the candidate hand areas by carrying out pattern recognition based on the extracted hand imaging features, wherein

the binary processing carried out with regard to the extracted foreground image includes;

calculating an edge image of the extracted foreground image;

calculating a gradient image of the extracted foreground image, and then carrying out binary processing with regard to the gradient image so as to obtain a first binary result;

carrying out binary processing with regard to the extracted foreground image by using a predetermined threshold value so as to obtain a second binary result;

combining the edge image, the first binary result, and the second binary result by using an OR logical operation so as to obtain a new binary image; and

filling a closed area in the obtained new binary image.