SYSTEM AND METHOD FOR HAND GESTURE RECOGNITION FOR REMOTE CONTROL OF AN INTERNET PROTOCOL TV

• US 20110221974A1
• Filed: 03/09/2011
• Published: 09/15/2011
• Est. Priority Date: 03/11/2010
• Status: Active Grant

• Current Assignee: Deutsche Telekom AG
• Original Assignee: Deutsche Telekom AG
• Inventors: Stegmann, Joachim, Weiss, Christian, Berman, Sigal, Stern, Helman, Lonczewski, Frank, Frolova, Daria
• Granted Patent: US 8,582,037 B2
• Time in Patent Office: Days
• US Class Current: 348/734
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/66 of image moments or centre of gravity
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The invention relates to a gesture recognition system for remote controlling a TV, which comprises: (a) video capturing device for capturing a video sequence of a user which is composed of video frames, and conveying the same into a hand tracker; (b) hand tracker module for receiving said video sequence, inspecting each of said video frames for possible inclusion of a user hand within it, and when a hand is determined to be found in a frame, calculating the hand centroid, and outputting the centroid coordinates into a trajectory buffer; (c) trajectory buffer for receiving from said hand tracker each of said hand centroid coordinates, and maintaining in storage at least several tens of recent hand centroids, that altogether define a trajectory; (d) trajectory segmentation module for continuously inspecting said buffer for possible inclusion of relevant trajectories that are suspected to relate to a hand gesture pattern, and whenever a relevant trajectory is detected, extracting a trajectory segment of the same, and conveying it into a gesture classifier; and (e) gesture classifier for extracting from each received trajectory segment global feature parameters of the same, comparing said extracted global feature parameters with gesture global predefinitions, concluding whether the segment relates to a specific gesture command, and in the affirmative case, transferring the command to the TV.

Claims

1. Gesture recognition system for remote controlling a TV, which comprises:

video capturing device for capturing a video sequence of a user which is composed of video frames, and conveying the same into a hand tracker;

hand tracker module for receiving said video sequence, inspecting each of said video frames for possible inclusion of a user hand within it, and when a hand is determined to be found in a frame, calculating the hand centroid, and outputting the centroid coordinates into a trajectory buffer;

trajectory buffer for receiving from said hand tracker each of said hand centroid coordinates, and maintaining in storage at least several tens of recent hand centroids, that altogether define a trajectory;

trajectory segmentation module for continuously inspecting said buffer for possible inclusion of relevant trajectories that are suspected to relate to a hand gesture pattern, and whenever a relevant trajectory is detected, extracting a trajectory segment of the same, and conveying it into a gesture classifier; and

gesture classifier for extracting from each received trajectory segment global feature parameters of the same, comparing said extracted global feature parameters with gesture global predefinitions, concluding whether the segment relates to a specific gesture command, and in the affirmative case, transferring the command to the TV.

2. Gesture recognition system according to claim 1, wherein the buffer maintains in storage 100-300 of recent hand centroids.

3. Gesture recognition system according to claim 1, wherein said global feature parameters comprises one or more of:

aspect ratio of the segment, variability of the segment, start and end point locations of the segment, intersection of the segment, and switch back traces.

4. Gesture recognition system according to claim 1, wherein the hand tracker bases its operation on hand predefinitions, and statistical calculations relating to the possibility that each pixel in the inspected frame relates to a hand.

5. Gesture recognition system according to claim 1, wherein the statistical calculations consider the similarity of the pixel color to a hand color.

6. Gesture recognition system according to claim 1, wherein the trajectory segment which is extracted by the trajectory segmentation module from the buffer is the whole buffer content or a portion thereof.

7. Gesture recognition system according to claim 1, wherein the trajectory segmentation module bases its detection of relevant trajectory segment on movement predefinitions.

8. Gesture recognition system according to claim 1, wherein the hand tracker employs a switch adaptively turned on and off by the amount of “

motion energy”

in the frame to enable dynamic fusion of motion and color cues.

9. Gesture recognition system according to claim 1, for controlling electrical devices from the group comprising:

TVs, IPTVs, PCs, DVDs, music players, video players, mobile screens, video consoles, wall switches, lamps, heating systems, and security systems.

10. Gesture recognition system according to claim 1, which is training free.

11. Gesture recognition system according to claim 1, which is feedback free.