Method for aligning gesture features of image

• US 7,295,707 B2
• Filed: 07/29/2003
• Issued: 11/13/2007
• Est. Priority Date: 01/29/2003
• Status: Expired due to Fees

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Liu, Cheng-Yi, Chen, I-Yen, Chang, Chin-Chen
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US10/628,511
• Publication Number:
  US 20050089225A1
• Time in Patent Office: 1,568 Days
• Field of Search: 382/181, 382/203, 382/170, 382/190, 382/201, 382/266, 382/279, 382/294, 382/305, 345/441, 345/442
• US Class Current: 382/181
• CPC Class Codes: E05F 15/00 Power-operated mechanisms for wings motor-operated accessories in locks for completing closing or initiating opening of a wing E05B17/00
• CPC Class Codes: E05Y 2400/852 Sensors
• CPC Class Codes: E05Y 2400/86 by hand
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/752 Contour matching
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A method for aligning gesture features of image is disclosed. An input gesture image is captured, and then a closed curve formed by a binary contour image of the gesture image is determined by processing the gesture image. A curvature scale space (CSS) image of the gesture image is drawn based on the closed curve. A convolution operation is performed with respect to the sequence of a coordinate-peak set formed by the CSS image and a predefined function to designate the coordinate with maximal value of integration as a basis point for obtaining a feature parameter of the gesture image. Finally, comparing the feature parameter of the gesture image with each feature parameter of a plurality of reference gesture shapes, thereby determining a gesture shape corresponding to the gesture image.

Claims

1. A method for aligning gesture features of image, comprising the steps of:

capturing an input gesture image;

determining a closed curve formed by a binary contour image of the gesture image by preprocessing the gesture image;

drawing a curvature scale space (CSS) image of the gesture image based on the closed curve;

performing a convolution operation with respect to the sequence of a coordinate-peak set formed by the CSS image and a predefined function F(x) to designate the coordinate with maximal value of integration as a basis point for obtaining feature parameters of the gesture image, wherein each feature parameter of the gesture image is obtained by the steps of

calculating all sets of the peaks in the CSS image to form a coordinate-peak set;

selecting a basis point k_o according to two-dimensional distribution of the coordinate-peak set;

performing a convolution operation with respect to the coordinate-peak set and the predefined function so as to extracting a coordinate u_ko with maximal value of integration as the basis point to align the coordinate-peak set through a circular rotation to form an aligned coordinate-peak set;

calculating the feature parameter based on the aligned coordinate-peak set; and

comparing each feature parameter of the gesture image with each feature parameter of a plurality of reference gesture shapes for determining a gesture shape corresponding to the gesture image.

2. The method as claimed in claim 1, wherein in the step of performing a convolution operation to designate the coordinate,

the coordinate-peak set is expressed as:

${\left\{\left(u_i,{\mathrm{\sigma <br><br>}}_i\right)\right\}}_{i=1,\mathrm{\dots <br><br>}<br><br>,N}^{\mathrm{Orginal}}=\left\{\left(u_1,{\mathrm{\sigma <br><br>}}_1\right),\left(u_2,{\mathrm{\sigma <br><br>}}_2\right),\mathrm{\dots <br><br>}<br><br>,\left(u_j,{\mathrm{\sigma <br><br>}}_j\right),\mathrm{\dots <br><br>}<br><br>,\left(u_N,{\mathrm{\sigma <br><br>}}_N\right)\right\},$

where N is the number of all of the detected peaks in the CSS image;

the basis point k_o is expressed as

$k_o=\arg <br><br><br><br>\underset{k}{\max }<br><br>\left(\underset{i=1}{\overset{k-1}{\sum <br><br>}}<br><br>{\mathrm{\sigma <br><br>}}_i·<br><br>F<br><br>\left(1+u_i-u_k\right)+\underset{i=k}{\overset{N}{\sum <br><br>}}<br><br>{\mathrm{\sigma <br><br>}}_i·<br><br>F<br><br>\left(u_i-u_k\right)\right);<br><br>$

the aligned coordinate-peak set is expressed as

${\left\{\left(u_i,{\mathrm{\sigma <br><br>}}_i\right)\right\}}_{i=1,\mathrm{\dots <br><br>}<br><br>,N}^{\mathrm{aligned}}=\left\{\left(0,{\mathrm{\sigma <br><br>}}_{k_o}\right),\left(u_{k_o+1}-u_{k_o},{\mathrm{\sigma <br><br>}}_{k_o+1}\right),\mathrm{\dots <br><br>}<br><br>,\left(1+u_1-u_{k_o},{\mathrm{\sigma <br><br>}}_1\right),\mathrm{\dots <br><br>}<br><br>,\left(1+u_{k_o-1}-u_{k_o},{\mathrm{\sigma <br><br>}}_{k_o-1}\right)\right\};<br><br>\mathrm{and}$

the feature parameter is expressed by

$F^I={\left\{\left(u_i^I,{\mathrm{\sigma <br><br>}}_i^I\right)\right\}}_{i=1,\mathrm{\dots <br><br>}<br><br>,N}^{\mathrm{aligned}},$

where N is the number of all of the detected peaks in the CSS image, I is the input gesture image.

3. The method as claimed in claim 2, wherein in the step of selecting a basis point, F(x) is defined between the period of [0,1).

4. The method as claimed in claim 3, wherein F(x) is a zero-mean Gauss function.

5. The method as claimed in claim 4, wherein the zero-mean Gauss function is:

$F<br><br>\left(x\right)=\frac{{\mathrm{e<br><br>}}^{-\left[x^2/2<br><br>{\mathrm{\sigma <br><br>}}^2\right]}}{\sqrt{2<br><br>\mathrm{\pi <br><br>}}<br><br>\mathrm{\sigma <br><br>}},0\leqq <br><br>x<<br><br>1,$

where σ

is defined for controlling the changing rate of F(x) in the period of [0,1).

6. The method as claimed in claim 1, wherein in the step of selecting a basis point, F(x) is an absolute increasing or decreasing function.

7. The method as claimed in claim 1, where in the step of comparing each feature parameter of the gesture image with each feature parameter of a plurality of reference gesture shapes, the feature parameter F^S of predetermined reference gesture shape is expressed as

$F^S=\{{\{(u_j^S,{\sigma }_{}^{}}_{}^{}$

j S ) } j = 1 , …

⁢

, M aligned } ,

where M is the number of the peaks of the reference gesture shape, and S is the reference gesture shape.

8. The method as claimed in claim 7, wherein the step of comparing each feature parameter of the gesture image with each feature parameter of a plurality of reference gesture shapes comprises the steps of:

obtaining a sum of distance of the matched peaks and distance of the unmatched peaks between the feature parameter F^I of the gesture image and the feature parameter FS of the reference gesture shapes by performing a distance function;

$\mathrm{dist}<br><br><br><br>\left(F^I,F_S\right)=\underset{\underset{\mathrm{peaks}}{\mathrm{matched}}}{\overset{}{\sum <br><br>}}<br><br>\sqrt{{\left(u_i^I-u_j^S\right)}^2+{\left({\mathrm{\sigma <br><br>}}_i^I-{\mathrm{\sigma <br><br>}}_j^S\right)}^2}+\underset{\underset{\mathrm{peaks}}{\mathrm{unmatched}}}{\overset{}{\sum <br><br>}}<br><br>{\mathrm{\sigma <br><br>}}_i^I+\underset{\underset{\mathrm{peaks}}{\mathrm{unmatched}}}{\overset{}{\sum <br><br>}}<br><br>{\mathrm{\sigma <br><br>}}_j^S;<br><br>$

and

utilizing a nearest neighbor algorithm to determine a reference gesture shape of the gesture image.

9. The method as claimed in claim 8, wherein in the step of utilizing a nearest neighbor algorithm to determine a reference gesture shape of the gesture image, the nearest neighbor algorithm is utilized to recognize gesture for finding the nearest reference gesture shape with respect to the gesture image so as to determine a gesture shape corresponding to the gesture image.

10. The method as claimed in claim 1, wherein in the step of drawing a curvature scale space (CSS) image, the CSS image is obtained by performing the steps of:

determining a curvature

$\mathrm{\kappa <br><br>}<br><br>\left(u\right)=\frac{\stackrel{.}{x}<br><br>\left(u\right)<br><br>\stackrel{\mathrm{¨<br><br>}}{y}<br><br>\left(u\right)-\stackrel{\mathrm{¨<br><br>}}{x}<br><br>\left(u\right)<br><br>\stackrel{.}{y}<br><br>\left(u\right)}{{\left({\stackrel{.}{x}}^2<br><br>\left(u\right)+{\stackrel{.}{y}}^2<br><br>\left(u\right)\right)}^{3/2}}$

 

of a closed curve Γ

={x(u), y(u)}, where u is a normalized arc length parameter,

$\stackrel{.}{x}<br><br>\left(u\right)=\frac{d<br><br>x}{d<br><br>u},\stackrel{\mathrm{¨<br><br>}}{x}<br><br>\left(u\right)=\frac{{d<br><br>}^2<br><br>x}{d<br><br>u^2},\stackrel{.}{y}<br><br>\left(u\right)=\frac{d<br><br>y}{d<br><br>u},\mathrm{and}<br><br><br><br>\stackrel{\mathrm{¨<br><br>}}{y}<br><br>\left(u\right)=\frac{{d<br><br>}^2<br><br>y}{d<br><br>u^2};<br><br>$

performing an operation with respect to the closed curve and a Gauss function to determine a smooth curvature function Γ

_σ

={X(u,σ

), Y(u,σ

)} and its curvature

$\mathrm{\kappa <br><br>}<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=\frac{X_u<br><br>\left(u,\mathrm{\sigma <br><br>}\right)<br><br>Y_{\mathrm{uu}}<br><br>\left(u,\mathrm{\sigma <br><br>}\right)-X_{\mathrm{uu}}<br><br>\left(u,\mathrm{\sigma <br><br>}\right)<br><br>Y_u<br><br>\left(u,\mathrm{\sigma <br><br>}\right)}{{\left({X_u<br><br>\left(u,\mathrm{\sigma <br><br>}\right)}^2+{Y_u<br><br>\left(u,\mathrm{\sigma <br><br>}\right)}^2\right)}^{3/2}}$

 

where

σ

is standard deviation,

$X<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=x<br><br>\left(u\right)*g<br><br>\left(u,\mathrm{\sigma <br><br>}\right)={\int <br><br>}_{-\mathrm{\infty <br><br>}}^{\mathrm{\infty <br><br>}}<br><br>x<br><br>\left(v\right)·<br><br>\frac{1}{\mathrm{\sigma <br><br>}<br><br>\sqrt{2<br><br>\mathrm{\pi <br><br>}}}·<br><br>\exp <br><br>\left(\frac{-{\left(u-v\right)}^2}{2<br><br>{\mathrm{\sigma <br><br>}}^2}\right)<br><br><br><br>d<br><br>v,<br><br>Y<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=y<br><br>\left(u\right)*g<br><br>\left(u,\mathrm{\sigma <br><br>}\right)={\int <br><br>}_{-\mathrm{\infty <br><br>}}^{\mathrm{\infty <br><br>}}<br><br>y<br><br>\left(v\right)·<br><br>\frac{1}{\mathrm{\sigma <br><br>}<br><br>\sqrt{2<br><br>\mathrm{\pi <br><br>}}}·<br><br>\exp <br><br>\left(\frac{-{\left(u-v\right)}^2}{2<br><br>{\mathrm{\sigma <br><br>}}^2}\right)<br><br><br><br>d<br><br>v,<br><br>X_u<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=x<br><br>\left(u\right)*\stackrel{.}{g}<br><br>\left(u,\mathrm{\sigma <br><br>}\right),X_{\mathrm{uu}}<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=x<br><br>\left(u\right)*\stackrel{\mathrm{¨<br><br>}}{g}<br><br>\left(u,\mathrm{\sigma <br><br>}\right),<br><br>Y_u<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=y<br><br>\left(u\right)*\stackrel{.}{g}<br><br>\left(u,\mathrm{\sigma <br><br>}\right),Y_{\mathrm{uu}}<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=y<br><br>\left(u\right)*\stackrel{\mathrm{¨<br><br>}}{g}<br><br>\left(u,\mathrm{\sigma <br><br>}\right),<br><br>\stackrel{.}{<br><br>g}<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=\frac{\partial <br><br>}{\partial <br><br>u}<br><br>g<br><br>\left(u,\mathrm{\sigma <br><br>}\right),\mathrm{and}<br><br><br><br>\stackrel{\mathrm{¨<br><br>}}{g}<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=\frac{{\partial <br><br>}^2<br><br>}{\partial <br><br>u^2}<br><br>g<br><br>\left(u,\mathrm{\sigma <br><br>}\right);<br><br>\mathrm{and}$

utilizing different standard deviations σ

to find a location having zero curvature in Γ

_σ

, and continuously drawing all locations having zero curvature under different standard deviations.

11. The method as claimed in claim 10, wherein the normalized arc length parameter u has a value between 0 and 1.

12. The method as claimed in claim 10, wherein in the step of performing an operation with respect to the closed curve and a Gauss function, the operation performed with respect to the close curve and the Gauss function is a convolution operation.

13. The method as claimed in claim 10, wherein in the step of performing an operation with respect to the closed curve and a Gauss function, the Gauss function is:

$g<br><br>\left(u,\mathrm{\sigma <br><br>}\right)=\frac{1}{\mathrm{\sigma <br><br>}<br><br>\sqrt{2<br><br>\mathrm{\pi <br><br>}}}·<br><br>\exp <br><br>\left(\frac{-u^2}{2<br><br>{\mathrm{\sigma <br><br>}}^2}\right).$

14. The method as claimed in claim 10, wherein in the step of utilizing different standard deviations to find a location having zero curvature, in the u−

σ

coordinate, a position of κ

(u,σ

)=0 is defined as the zero curvature point.