Method and device for segmenting hand gestures
First Claim
1. A method of automatically segmenting a subject'"'"'s hand gestures into words or apprehensible units structured as a plurality of words when recognizing the subject'"'"'s hand gestures, said method comprising:
- storing transition feature data including a feature of a transition gesture which is not observed in the subject'"'"'s body during a gesture representing a word, but is observed when transiting from one gesture to another;
photographing the subject, and storing image data thereof;
extracting an image corresponding to a part of the body in which the transition gesture is observed from the image data;
detecting a motion of the image corresponding to the part of the body in which the transition gesture is observed; and
segmenting the hand gestures by comparing the motion of the image corresponding to the part of the body in which the transition gesture is observed with the transition feature data, and then finding a time position where the transition gesture is observed.
2 Assignments
0 Petitions
Accused Products
Abstract
An object of the present invention is to provide a method of segmenting hand gestures which automatically segments hand gestures to be detected into words or apprehensible units structured by a plurality of words when recognizing the hand gestures without the user'"'"'s presentation where to segment. Transition feature data in which a feature of a transition gesture being not observed during a gesture representing a word but is described when transiting from a gesture to another is previously stored. Thereafter, a motion of image corresponding to the part of body in which the transition gesture is observed is detected (step S106), the detected motion of image is compared with the transition feature data (step S107), and a time position where the transition gesture is observed is determined so as to segment the hand gestures (step S108).
-
Citations
33 Claims
-
1. A method of automatically segmenting a subject'"'"'s hand gestures into words or apprehensible units structured as a plurality of words when recognizing the subject'"'"'s hand gestures, said method comprising:
-
storing transition feature data including a feature of a transition gesture which is not observed in the subject'"'"'s body during a gesture representing a word, but is observed when transiting from one gesture to another;
photographing the subject, and storing image data thereof;
extracting an image corresponding to a part of the body in which the transition gesture is observed from the image data;
detecting a motion of the image corresponding to the part of the body in which the transition gesture is observed; and
segmenting the hand gestures by comparing the motion of the image corresponding to the part of the body in which the transition gesture is observed with the transition feature data, and then finding a time position where the transition gesture is observed. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20)
storing non-transition feature data including a feature of a non-transition gesture which is not observed in the body when transiting from a gesture representing a word to another, but is observed during a gesture representing a word;
extracting an image corresponding to a part of the body in which the non-transition gesture is observed from the image data;
detecting a motion of the image corresponding to the part of the body in which the non-transition gesture is observed; and
finding a time position where the non-transition gesture is observed by comparing the motion of the image corresponding to the part of the body in which the non-transition gesture is observed with the non-transition feature data, wherein said segmenting of the hand gestures does not occur at the time position where the non-transition gesture is observed.
-
-
11. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 10, wherein the non-transition gesture comprises bringing hands closer to each other than a value predetermined for a distance between the hands.
-
12. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 10, wherein the non-transition gesture comprises changing a shape of a mouth.
-
13. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 10, wherein the non-transition gesture comprises a motion of moving a right hand symmetrical to a left hand, or vice-versa.
-
14. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 13, wherein said photographing of the subject and storing image data thereof comprises stereoscopically photographing the subject and storing 3D image data thereof;
-
said extracting of the image corresponding to a part of the body in which the non-transition gesture is observed comprises extracting a 3D image corresponding to the part of the body in which the non-transition gesture is observed from the 3D image data;
said detecting of the motion of the image corresponding to the part of the body in which the non-transition gesture is observed comprises detecting a motion of the 3D image; and
said finding of the time position where the non-transition gesture is observed comprises;
detecting whether changes in a gesture plane for the right hand and a gesture plane for the left hand are in accordance with the motion of the 3D image, and if neither of the gesture planes shows a change, determining that the non-transition gesture is being observed and finding a time position of the non-transition gesture.
-
-
15. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 14, wherein said detecting of whether changes in a gesture plane for the right hand and a gesture plane for the left hand comprises detecting whether changes in a gesture plane for the right hand and a gesture plane for the left hand are in accordance with a change in a normal vector to the gesture planes.
-
16. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 14, further comprising:
-
generating, as to a plurality of 3D gesture codes corresponding to a 3D vector whose direction is varying, a single-motion plane table in which a combination of the 3D gesture codes found in a single plane is included; and
converting the motion of the 3D image into a 3D gesture code string represented by the plurality of 3D gesture codes, wherein said detecting of whether changes in a gesture plane for the right hand and a gesture plane for the left hand comprises detecting whether changes in a gesture plane for the right hand and a gesture plane for the left hand are in accordance with the single-motion plane table.
-
-
17. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 1, further comprising:
-
storing image data of an animation representing the transition gesture;
detecting a status of the transition gesture'"'"'s detection and a status of the hand gesture'"'"'s recognition; and
visually displaying the animation representing the transition gesture to the subject in relation to the status of the transition gesture'"'"'s detection and the status of the hand gesture'"'"'s recognition.
-
-
18. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 17, wherein said visually displaying of the animation comprises changing a speed of the animation in accordance with the status of the hand gesture'"'"'s recognition.
-
19. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 1, wherein said storing transition feature data comprises previously storing transition feature data.
-
20. A method of automatically segmenting a subject'"'"'s hand gestures according to claim 10, wherein said storing non-transition feature data comprises previously storing non-transition feature data.
-
21. A computer program embodied on a computer readable medium for use with a computer for automatically segmenting a subject'"'"'s hand gestures into words or apprehensible units structured by a plurality of words, said computer program comprising:
-
computer readable program code operable to instruct the computer to store transition feature data including a feature of a transition gesture which is not observed in the subjcct'"'"'s body during a gesture representing a word, but is observed when transiting from one gesture to another;
computer readable program code operable to instruct the computer to instruct a camera to photograph the subject and store image data thereof;
computer readable program code operable to instruct the computer to extract an image corresponding to a part of the body in which the transition gesture is observed from the image data;
computer readable program code operable to instruct the computer to detect a motion of the image corresponding to the part of the body in which the transition gesture is observed; and
computer readable program code operable to instruct the computer to segment the hand gestures by comparing the motion of the image corresponding to the part of the body in which the transition gesture is observed with the transition feature data, and then find a time position where the transition gesture is observed. - View Dependent Claims (22, 23, 24, 25, 26)
computer readable program code operable to instruct the computer to store non-transition feature data including a feature of a non-transition gesture which is not observed in the body when transiting from a gesture representing a word to another, but is observed during a gesture representing a word;
computer readable program code operable to instruct the computer to extract an image corresponding to a part of the body in which the non-transition gesture is observed from the image data;
computer readable program code operable to instruct the computer to detect a motion of the image corresponding to the part of the body in which the non-transition gesture is observed; and
computer readable program code operable to instruct the computer to find a time position where the non-transition gesture is observed by comparing the motion of the image corresponding the part of the body in which the non-transition gesture is observed with the non-transition feature data, wherein said computer readable program code operable to instruct the computer to segment the hand gestures is further operable to instruct the computer to not segment the hand gestures at the time position where the non-transition gesture is observed.
-
-
23. A computer program according to claim 21, further comprising:
-
computer readable program code operable to instruct the computer to store image data of an animation representing the transition gesture;
computer readable program code operable to instruct the computer to detect a status of the transition gesture'"'"'s detection and a status of the hand gesture'"'"'s recognition; and
computer readable program code operable to instruct the computer to visually display the animation representing the transition gesture to the subject in relation to the status of the transition gesture'"'"'s detection and the status of the hand gesture'"'"'s recognition.
-
-
24. A computer program according to claim 21, wherein said computer readable program code operable to instruct the computer to store transition feature data comprises computer readable program code operable to instruct the computer to previously store transition feature data.
-
25. A computer program according to claim 22, wherein said computer readable program code operable to instruct the computer to store non-transition feature data comprises computer readable program code operable to instruct the computer to previously store non-transition feature data.
-
26. A computer program according to claim 23, wherein said computer readable program code operable to instruct the computer to store image data of an animation representing the transition gesture comprises computer readable program code operable to instruct the computer to previously store image data of an animation representing the transition gesture.
-
27. A hand gesture segmentation device for automatically segmenting a subject'"'"'s hand gestures into words or apprehensible units structured by a plurality of words when recognizing the subject'"'"'s hand gestures, said device comprising:
-
means for storing transition feature data including a feature of a transition gesture which is not observed in the subject'"'"'s body during a gesture representing a word, but is observed when transiting from one gesture to another;
means for photographing the subject, and storing image data thereof;
means for extracting an image corresponding to a part of the body in which the transition gesture is observed;
means for detecting a motion of the image corresponding to the part of the body in which the transition gesture is observed; and
means for segmenting the hand gestures by comparing the motion of the image corresponding to the part of the body in which the transition gesture is observed with the transition feature data, and then finding a time position where the transition gesture is observed. - View Dependent Claims (28)
means for storing non-transition data including a feature of a non-transition gesture which is not observed in the body when transiting from a gesture representing a word to another, but is observed during a gesture representing a word;
means for extracting an image corresponding to a part if the body in which the non-transition gesture is observed from the image data;
means for detecting a motion of the image corresponding to the part of the body in which the non-transition gesture is observed; and
means for finding a time position where the non-transition gesture is observed by comparing the motion of the image corresponding to the part of the body in which the non-transition gesture is observed with the non-transition feature data, wherein said means for segmenting the hand gestures does not execute segmentation with respect to the hand gestures at the time position where the non-transition gesture is observed.
-
-
29. A motion induction device being incorporated in a hand gesture recognition device for recognizing a subject'"'"'s hand gestures, and in a hand gesture segmentation device for automatically segmenting the hand gesture into words or apprehensible units structured by a plurality of words to visually guide the subject to make a predetermined gesture, said hand gesture segmentation device including a function of detecting a transition gesture which is not observed in the subject'"'"'s body during a gesture representing a word, but is observed when transiting from one gesture to another, and then segmenting the hand gestures, said motion induction device comprising:
-
means for storing image data of an animation representing the transition gesture;
means for detecting a status of the transition gesture'"'"'s detection and a status of the hand gesture'"'"'s recognition by monitoring said hand gesture segmentation device and said hand gesture recognition device; and
means for visually displaying the animation representing the transition gesture to the subject in relation to the status of the transition gesture'"'"'s detection and the status of the hand gesture'"'"'s recognition. - View Dependent Claims (30, 31)
-
-
32. A hand gesture segmentation device for automatically segmenting a subject'"'"'s hand gestures into words or apprehensible units structured by a plurality of words when recognizing the subject'"'"'s hand gestures, said device comprising:
-
means for storing transition feature data including a feature of a transition gesture which is not observed in the subject'"'"'s body during a gesture representing a word, but is observed when transiting from one gesture to another;
means for photographing the subject with a camera placed in a position opposite to the subject, and storing image data thereof;
means for extracting an image corresponding to a part of the body in which the transition gesture is observed from the image data;
means for detecting a motion of the image corresponding to the part of the body in which the transition gesture is observed;
means for segmenting the hand gesture by comparing the motion of the image corresponding to the part of the body in which the transition gesture is observed with the transition feature data, and then finding a time position where the transition gesture is observed;
means for visually displaying the animation representing the transition gesture to the subject in relation to the status of the transition gesture'"'"'s detection and the status of the hand gesture'"'"'s recognition; and
means for concealing said camera from the subject'"'"'s view. - View Dependent Claims (33)
-
Specification