Tracking bimanual movements

• US 8,259,996 B2
• Filed: 10/31/2007
• Issued: 09/04/2012
• Est. Priority Date: 04/15/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shamaie, Atid
• Primary Examiner(s): LE, BRIAN Q
• Application Number:
  US11/932,766
• Publication Number:
  US 20080219502A1
• Time in Patent Office: 1,770 Days
• Field of Search: 382/103, 382/106, 382/107, 348/169, 715/863
• US Class Current: 382/103
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Hands may be tracked before, during, and after occlusion, and a gesture may be recognized. Movement of two occluded hands may be tracked as a unit during an occlusion period. A type of synchronization characterizing the two occluded hands during the occlusion period may be determined based on the tracked movement of the occluded hands. Based on the determined type of synchronization, it may be determined whether directions of travel for each of the two occluded hands change during the occlusion period. Implementations may determine that a first hand and a second hand are occluded during an occlusion period, the first hand having come from a first direction and the second hand having come from a second direction. The first hand may be distinguished from the second hand after the occlusion period based on a determined type of synchronization characterizing the two hands, and a behavior of the two hands.

Claims

1. A method of identifying an observed bimanual hand gesture, the method comprising:

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures; and

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying, by a computing device that includes hardware, a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture,

wherein tracking movement of a first hand during an observed bimanual gesture comprises extracting a sequence of images of the first hand from a sequence of images of the observed bimanual gesture;

wherein tracking movement of a second hand during an observed bimanual gesture comprises extracting a sequence of images of the second hand from the sequence of images of the observed bimanual gesture;

wherein determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand, using trained Hidden Markov Models that model first-hand movements of the one or more known bimanual gestures to determine the first set of measures of likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures; and

wherein determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand, using trained Hidden Markov Models that model second-hand movements of the one or more known bimanual gestures to determine the second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures.

2. The method of claim 1 wherein:

identifying a sequence of principal components corresponding to the sequence of images of the first hand comprises;

projecting the images from the sequence of images of the first hand into a first-hand eigenspace formed based on principal components determined for a set of training images of first-hand movements of the one or more known bimanual gestures, and

extracting, from the first-hand eigenspace, a first sequence of codewords associated with the projections of the images of the first hand; and

identifying a sequence of principal components corresponding to the sequence of images of the second hand comprises;

projecting the images from the sequence of images of the second hand into a second-hand eigenspace formed based on principal components determined for a set of training images of second-hand movements of the one or more known bimanual gestures, and

extracting, from the second-hand eigenspace, a second sequence of codewords associated with the projections of the images of the second hand.

3. A method of identifying in an observed bimanual hand gesture, the method comprising:

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying, by a computing device that includes hardware, a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture; and

maintaining a Bayesian network for identifying bimanual gestures that includes a first-hand node, a second-hand node, and a bimanual gesture node, wherein identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures comprises;

inputting, to the first-hand node of the Bayesian network, the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

inputting, to the second-hand node of the Bayesian network, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

using the first-hand node of the Bayesian network to determine a first belief of a known bimanual gesture that corresponds to the tracked movement of the first hand based on the input first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

using the second-hand node of the Bayesian network to determine a second belief of a known bimanual gesture that corresponds to the tracked movement of the second hand based on the input second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

inputting the determined first belief and the determined second belief to the bimanual gesture root node; and

using the bimanual gesture root node to identify a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the determined first belief and the determined second belief.

4. A method of identifying an observed bimanual hand gesture, the method comprising:

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying, by a computing device that includes hardware, a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture;

tracking movement of the first and second hands as a unit during an occlusion period of the bimanual gesture; and

determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures,

wherein tracking movement of a first hand during an observed bimanual gesture comprises tracking movement of the first hand during a non-occlusion period of the observed bimanual gesture;

wherein tracking movement of a second hand during the observed bimanual gesture comprises tracking movement of the second hand during the non-occlusion period of the observed bimanual gesture;

wherein determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures comprises determining a first set of measures of likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

wherein determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures comprises determining a second set of measures of likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures; and

wherein identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture comprises identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, and the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures.

5. The method of claim 4 wherein:

tracking movement of the first hand during the non-occlusion period of the observed bimanual gesture comprises extracting a sequence of images of the first hand during the non-occlusion period from a sequence of images of the observed bimanual gesture;

tracking movement of a second hand during the non-occlusion period of the observed bimanual gesture comprises extracting a sequence of images of the second hand during the non-occlusion period from the sequence of images of the observed bimanual gesture;

tracking movement of the first and second hands as a unit during an occlusion period of the bimanual gesture comprises extracting a sequence of images of the first and second hands during the occlusion period from the sequence of images of the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the non-occlusion period, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand during the non-occlusion period, using trained Hidden Markov Models that model first-hand movements of the one or more known bimanual gestures to determine the first set of measures of likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

determining the second set of measures of likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the non-occlusion period, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand during the non-occlusion period, using trained Hidden Markov Models that model second-hand movements of the one or more known bimanual gestures to determine the second set of measures of likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures; and

determining a third set of measures of likelihoods that the tracked movement of the first hand and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first and second hands during the occlusion period, and

based on the identified sequence of principal components corresponding to the sequence of images of the first and second hands during the occlusion period, using trained Hidden Markov Models that model occluded-hand movements of the one or more known bimanual gestures to determine a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures.

6. The method of claim 5 wherein:

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the non-occlusion period comprises;

projecting the images from the sequence of images of the first hand during the non-occlusion period into a first-hand eigenspace formed based on principal components determined for a set of training images of first-hand movements of the one or more known bimanual gestures, and

extracting, from the first-hand eigenspace, a first sequence of codewords associated with the projections of the images of the first hand during the non-occlusion period;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the non-occlusion period comprises;

projecting the images from the sequence of images of the second hand during the non-occlusion period into a second-hand eigenspace formed based on principal components determined for a set of training images of second-hand movements of the one or more known bimanual gestures, and

extracting, from the second-hand eigenspace, a second sequence of codewords associated with the projections of the images of the second hand during the non-occlusion period; and

identifying a sequence of principal components corresponding to the sequence of images of the first and second hands during the occlusion period comprises;

projecting the images from the sequence of images of the first and second hands during the occlusion period into an occluded-hand eigenspace formed based on principal components determined for a set of training images of occluded-hand movements of the one or more known bimanual gestures, and

extracting, from the occluded-hand eigenspace, a third sequence of codewords associated with the projections of the images of the first and second hands during the occlusion period.

7. The method of claim 4 further comprising maintaining a Bayesian network for identifying bimanual gestures that includes a first-hand node, a second-hand node, an occluded-hand node, and a bimanual gesture node, wherein identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures, and the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures comprises:

inputting, to the first-hand node of the Bayesian network, the first set of measures of the likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

inputting, to the second-hand node of the Bayesian network, the second set of measures of the likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

inputting, to the occluded-hand node of the Bayesian network, the third set of measures of the likelihoods that the tracked movement of the first and second hands during the occlusion period corresponds to the one or more known bimanual gestures;

using the first-hand node of the Bayesian network to determine a first belief of a known bimanual gesture that corresponds to the tracked movement of the first hand during the non-occlusion period based on the input first set of measures of the likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

using the second-hand node of the Bayesian network to determine a second belief of a known bimanual gesture that corresponds to the tracked movement of the second hand during the non-occlusion period based on the input second set of measures of the likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

using the occluded-hand node of the Bayesian network to determine a third belief of a known bimanual gesture that corresponds to the tracked movement of the first and second hands during the occlusion period based on the input third set of measures of the likelihoods that the tracked movement of the first and second hands during the occlusion period corresponds to the one or more known bimanual gestures;

inputting the determined first belief, the determined second belief, and the determined third belief to the bimanual gesture root node; and

using the bimanual gesture root node to identify a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the determined first belief, the determined second belief, and the determined third belief.

8. A method of identifying an observed bimanual hand gesture, the method comprising:

tracking movement of a first hand during a first non-occluded segment of an observed bimanual gesture;

tracking movement of a second hand during the first non-occluded segment of the observed bimanual gesture;

determining that the first and second hands are occluded, wherein a determination that the first and second hands are occluded signals a beginning of an occluded segment of the observed bimanual gesture;

tracking movement of the first and second hands as a unit during the occluded segment of the observed bimanual gesture;

determining that the first and second hands are no longer occluded, wherein a determination that the first and second hands are no longer included signals a beginning of a second non-occluded segment of the observed bimanual gesture;

distinguishing, after determining that the first and second hands are no longer occluded, the first hand from the second hand;

after distinguishing the first hand from the second hand, tracking movement of the first hand during the second non-occluded end segment of the observed bimanual gesture;

after distinguishing the first hand from the second hand, tracking movement of the second hand during the second non-occluded segment of the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures;

determining a fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures;

determining a fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures; and

identifying, by a computing device that includes hardware, a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures, the fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures, and the fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures.

9. The method of claim 8 further comprising determining a type of synchronization characterizing the first and second hands during the occluded segment based on the tracked movement of the first and second hands as a unit during the occluded segment, wherein distinguishing the first hand from the second hand comprises distinguishing the first hand from the second hand based on the determined type of synchronization.

10. The method of claim 9 wherein:

tracking movement of a first hand during a first non-occluded segment of an observed bimanual gesture comprises extracting a sequence of images of the first hand during the first non-occluded segment from a sequence of images of the observed bimanual gesture;

tracking movement of a second hand during a second non-occluded segment of an observed bimanual gesture comprises extracting a sequence of images of the second hand during the first non-occluded segment from the sequence of images of the observed bimanual gesture;

tracking movement of the first and second hands as a unit during the occluded segment of the observed bimanual gesture comprises extracting a sequence of images of the first and second hands during the occluded segment from the sequence of images of the observed bimanual gesture;

tracking movement of the first hand during a second non-occluded segment of the observed bimanual gesture comprises extracting a sequence of images of the first hand during the second non-occluded segment from a sequence of images of the observed bimanual gesture;

tracking movement of the second hand during the second non-occluded segment of the observed bimanual gesture comprises extracting a sequence of images of the second hand during the second non-occluded segment from the sequence of images of the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the first non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand during the first non-occluded segment, using trained Hidden Markov Models that model first-hand movements of first non-occluded segments of the one or more known bimanual gestures to determine the first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the first non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand during the first non-occluded segment, using trained Hidden Markov Models that model second-hand movements of first non-occluded segments of the one or more known bimanual gestures to determine the second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

determining a third set of measures of likelihoods that the tracked movement of the first and second hands during the occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first and second hands during the occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the first and second hands during the occluded segment, using trained Hidden Markov Models that model occluded-hand movements of occluded segments of the one or more known bimanual gestures to determine the third set of measures of likelihoods that the tracked movement of the first and second hands during the occluded segment corresponds to the one or more known bimanual gestures;

determining a fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the second non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand during the second non-occluded segment, using trained Hidden Markov Models that model first-hand movements of second non-occluded segments of the one or more known bimanual gestures to determine the fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures; and

determining a fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the second non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand during the second non-occluded segment, using trained Hidden Markov Models that model second-hand movements of second non-occluded segments of the one or more known bimanual gestures to determine the fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures.

11. The method of claim 10 wherein:

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the first non-occluded segment comprises;

projecting the images from the sequence of images of the first hand during the first non-occluded segment into a first first-hand eigenspace formed based on principal components determined for a set of training images of first-hand movements of first non-occluded segments of the one or more known bimanual gestures, and

extracting, from the first first-hand eigenspace, a first sequence of codewords associated with the projections of the images of the first hand during the first non-occluded segment;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the first non-occluded segment comprises;

projecting the images from the sequence of images of the second hand during the first non-occluded segment into a first second-hand eigenspace formed based on principal components determined for a set of training images of second-hand movements of first non-occluded segments of the one or more known bimanual gestures, and

extracting, from the first second-hand eigenspace, a second sequence of codewords associated with the projections of the images of the second hand during the first non-occluded segment;

identifying a sequence of principal components corresponding to the sequence of images of the first and second hands during the occluded segment comprises;

projecting the images from the sequence of images of the first and second hands during the occluded segment into an occluded-hand eigenspace formed based on principal components determined for a set of training images of first and second hand movements of occluded segments of the one or more known bimanual gestures, and

extracting, from the occluded-hand eigenspace, a third sequence of codewords associated with the projections of the images of the first and second hands during the occluded segment;

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the second non-occluded segment comprises;

projecting the images from the sequence of images of the first hand during the second non-occluded segment into a second first-hand eigenspace formed based on principal components determined for a set of training images of first-hand movements of second non-occluded segments of the one or more known bimanual gestures, and

extracting, from the second first-hand eigenspace, a fourth sequence of codewords associated with the projections of the images of the first hand during the second non-occluded segment; and

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the second non-occluded segment comprises;

projecting the images from the sequence of images of the second hand during the second non-occluded segment into a second second-hand eigenspace formed based on principal components determined for a set of training images of second-hand movements of second non-occluded segments of the one or more known bimanual gestures, and

extracting, from the second second-hand eigenspace, a fifth sequence of codewords associated with the projections of the images of the second hand during the second non-occluded segment.

12. A device comprising:

at least one processor; and

at least one computer-readable storage medium having stored thereon instructions which, when executed, causes the device to perform operations comprising;

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures; and

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture,

wherein tracking movement of a first hand during an observed bimanual gesture comprises extracting a sequence of images of the first hand from a sequence of images of the observed bimanual gesture;

wherein tracking movement of a second hand during an observed bimanual gesture comprises extracting a sequence of images of the second hand from the sequence of images of the observed bimanual gesture;

wherein determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand, using trained Hidden Markov Models that model first-hand movements of the one or more known bimanual gestures to determine the first set of measures of likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures; and

wherein determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand, using trained Hidden Markov Models that model second-hand movements of the one or more known bimanual gestures to determine the second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures.

13. The device of claim 12 wherein:

identifying a sequence of principal components corresponding to the sequence of images of the first hand comprises;

projecting the images from the sequence of images of the first hand into a first-hand eigenspace formed based on principal components determined for a set of training images of first-hand movements of the one or more known bimanual gestures, and

extracting, from the first-hand eigenspace, a first sequence of codewords associated with the projections of the images of the first hand; and

identifying a sequence of principal components corresponding to the sequence of images of the second hand comprises;

projecting the images from the sequence of images of the second hand into a second-hand eigenspace formed based on principal components determined for a set of training images of second-hand movements of the one or more known bimanual gestures, and

extracting, from the second-hand eigenspace, a second sequence of codewords associated with the projections of the images of the second hand.

14. A device comprising:

at least one processor; and

at least one computer-readable storage medium having stored thereon instructions which, when executed, causes the device to perform operations comprising;

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture; and

maintaining a Bayesian network for identifying bimanual gestures that includes a first-hand node, a second-hand node, and a bimanual gesture node, wherein identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures comprises;

inputting, to the first-hand node of the Bayesian network, the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

inputting, to the second-hand node of the Bayesian network, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

using the first-hand node of the Bayesian network to determine a first belief of a known bimanual gesture that corresponds to the tracked movement of the first hand based on the input first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

using the second-hand node of the Bayesian network to determine a second belief of a known bimanual gesture that corresponds to the tracked movement of the second hand based on the input second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

inputting the determined first belief and the determined second belief to the bimanual gesture root node; and

using the bimanual gesture root node to identify a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the determined first belief and the determined second belief.

15. A device comprising:

at least one processor; and

at least one computer-readable storage medium having stored thereon instructions which, when executed, causes the device to perform operations comprising;

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture;

tracking movement of the first and second hands as a unit during an occlusion period of the bimanual gesture; and

determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures,

wherein tracking movement of a first hand during an observed bimanual gesture comprises tracking movement of the first hand during a non-occlusion period of the observed bimanual gesture;

wherein tracking movement of a second hand during the observed bimanual gesture comprises tracking movement of the second hand during the non-occlusion period of the observed bimanual gesture;

wherein determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures comprises determining a first set of measures of likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

wherein determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures comprises determining a second set of measures of likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures; and

wherein identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture comprises identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, and the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures.

16. A device comprising:

at least one processor; and

at least one computer-readable storage medium having stored thereon instructions which, when executed, causes the device to perform operations comprising;

tracking movement of a first hand during a first non-occluded segment of an observed bimanual gesture;

tracking movement of a second hand during the first non-occluded segment of the observed bimanual gesture;

determining that the first and second hands are occluded, wherein a determination that the first and second hands are occluded signals a beginning of an occluded segment of the observed bimanual gesture;

tracking movement of the first and second hands as a unit during the occluded segment of the observed bimanual gesture;

determining that the first and second hands are no longer occluded, wherein a determination that the first and second hands are no longer included signals a beginning of a second non-occluded segment of the observed bimanual gesture;

distinguishing, after determining that the first and second hands are no longer occluded, the first hand from the second hand;

after distinguishing the first hand from the second hand, tracking movement of the first hand during the second non-occluded end segment of the observed bimanual gesture;

after distinguishing the first hand from the second hand, tracking movement of the second hand during the second non-occluded segment of the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures;

determining a fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures;

determining a fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures; and

identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures, the fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures, and the fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures.

17. The device of claim 16 wherein the operations further comprise determining a type of synchronization characterizing the first and second hands during the occluded segment based on the tracked movement of the first and second hands as a unit during the occluded segment, wherein distinguishing the first hand from the second hand comprises distinguishing the first hand from the second hand based on the determined type of synchronization.

18. The device of claim 17 wherein:

tracking movement of a first hand during a first non-occluded segment of an observed bimanual gesture comprises extracting a sequence of images of the first hand during the first non-occluded segment from a sequence of images of the observed bimanual gesture;

tracking movement of a second hand during a second non-occluded segment of an observed bimanual gesture comprises extracting a sequence of images of the second hand during the first non-occluded segment from the sequence of images of the observed bimanual gesture;

tracking movement of the first and second hands as a unit during the occluded segment of the observed bimanual gesture comprises extracting a sequence of images of the first and second hands during the occluded segment from the sequence of images of the observed bimanual gesture;

tracking movement of the first hand during a second non-occluded segment of the observed bimanual gesture comprises extracting a sequence of images of the first hand during the second non-occluded segment from a sequence of images of the observed bimanual gesture;

tracking movement of the second hand during the second non-occluded segment of the observed bimanual gesture comprises extracting a sequence of images of the second hand during the second non-occluded segment from the sequence of images of the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the first non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand during the first non-occluded segment, using trained Hidden Markov Models that model first-hand movements of first non-occluded segments of the one or more known bimanual gestures to determine the first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the first non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand during the first non-occluded segment, using trained Hidden Markov Models that model second-hand movements of first non-occluded segments of the one or more known bimanual gestures to determine the second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

determining a third set of measures of likelihoods that the tracked movement of the first and second hands during the occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first and second hands during the occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the first and second hands during the occluded segment, using trained Hidden Markov Models that model occluded-hand movements of occluded segments of the one or more known bimanual gestures to determine the third set of measures of likelihoods that the tracked movement of the first and second hands during the occluded segment corresponds to the one or more known bimanual gestures;

determining a fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the second non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand during the second non-occluded segment, using trained Hidden Markov Models that model first-hand movements of second non-occluded segments of the one or more known bimanual gestures to determine the fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures; and

determining a fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the second non-occluded segment, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand during the second non-occluded segment, using trained Hidden Markov Models that model second-hand movements of second non-occluded segments of the one or more known bimanual gestures to determine the fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures.

19. The device of claim 18 wherein:

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the first non-occluded segment comprises;

projecting the images from the sequence of images of the first hand during the first non-occluded segment into a first first-hand eigenspace formed based on principal components determined for a set of training images of first-hand movements of first non-occluded segments of the one or more known bimanual gestures, and

extracting, from the first first-hand eigenspace, a first sequence of codewords associated with the projections of the images of the first hand during the first non-occluded segment;

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the first non-occluded segment comprises;

projecting the images from the sequence of images of the second hand during the first non-occluded segment into a first second-hand eigenspace formed based on principal components determined for a set of training images of second-hand movements of first non-occluded segments of the one or more known bimanual gestures, and

extracting, from the first second-hand eigenspace, a second sequence of codewords associated with the projections of the images of the second hand during the first non-occluded segment;

identifying a sequence of principal components corresponding to the sequence of images of the first and second hands during the occluded segment comprises;

projecting the images from the sequence of images of the first and second hands during the occluded segment into an occluded-hand eigenspace formed based on principal components determined for a set of training images of first and second hand movements of occluded segments of the one or more known bimanual gestures, and

extracting, from the occluded-hand eigenspace, a third sequence of codewords associated with the projections of the images of the first and second hands during the occluded segment;

identifying a sequence of principal components corresponding to the sequence of images of the first hand during the second non-occluded segment comprises;

projecting the images from the sequence of images of the first hand during the second non-occluded segment into a second first-hand eigenspace formed based on principal components determined for a set of training images of first-hand movements of second non-occluded segments of the one or more known bimanual gestures, and

extracting, from the second first-hand eigenspace, a fourth sequence of codewords associated with the projections of the images of the first hand during the second non-occluded segment; and

identifying a sequence of principal components corresponding to the sequence of images of the second hand during the second non-occluded segment comprises;

projecting the images from the sequence of images of the second hand during the second non-occluded segment into a second second-hand eigenspace formed based on principal components determined for a set of training images of second-hand movements of second non-occluded segments of the one or more known bimanual gestures, and

extracting, from the second second-hand eigenspace, a fifth sequence of codewords associated with the projections of the images of the second hand during the second non-occluded segment.

20. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed, cause a computing device to perform operations comprising:

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture;

tracking movement of the first and second hands as a unit during an occlusion period of the bimanual gesture; and

determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures,

wherein tracking movement of a first hand during an observed bimanual gesture comprises tracking movement of the first hand during a non-occlusion period of the observed bimanual gesture;

wherein tracking movement of a second hand during the observed bimanual gesture comprises tracking movement of the second hand during the non-occlusion period of the observed bimanual gesture;

wherein determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures comprises determining a first set of measures of likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

wherein determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures comprises determining a second set of measures of likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures; and

wherein identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture comprises identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, and the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures.

21. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed, cause a computing device to perform operations comprising:

tracking movement of a first hand during a first non-occluded segment of an observed bimanual gesture;

tracking movement of a second hand during the first non-occluded segment of the observed bimanual gesture;

determining that the first and second hands are occluded, wherein a determination that the first and second hands are occluded signals a beginning of an occluded segment of the observed bimanual gesture;

tracking movement of the first and second hands as a unit during the occluded segment of the observed bimanual gesture;

determining that the first and second hands are no longer occluded, wherein a determination that the first and second hands are no longer included signals a beginning of a second non-occluded segment of the observed bimanual gesture;

distinguishing, after determining that the first and second hands are no longer occluded, the first hand from the second hand;

after distinguishing the first hand from the second hand, tracking movement of the first hand during the second non-occluded end segment of the observed bimanual gesture;

after distinguishing the first hand from the second hand, tracking movement of the second hand during the second non-occluded segment of the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures;

determining a fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures;

determining a fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures; and

identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures, the fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures, and the fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures.

22. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed, cause a computing device to perform operations comprising:

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures; and

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture,

wherein tracking movement of a first hand during an observed bimanual gesture comprises extracting a sequence of images of the first hand from a sequence of images of the observed bimanual gesture;

wherein tracking movement of a second hand during an observed bimanual gesture comprises extracting a sequence of images of the second hand from the sequence of images of the observed bimanual gesture;

wherein the instructions that, when executed, cause the computing device to determine a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures include instructions that, when executed, cause the computing device to perform operations comprising;

identifying a sequence of principal components corresponding to the sequence of images of the first hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand, using trained Hidden Markov Models that model first-hand movements of the one or more known bimanual gestures to determine the first set of measures of likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures; and

wherein determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand, using trained Hidden Markov Models that model second-hand movements of the one or more known bimanual gestures to determine the second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures.

23. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed, cause a computing device to perform operations comprising:

tracking movement of a first hand during an observed bimanual gesture;

tracking movement of a second hand during the observed bimanual gesture;

determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture; and

maintaining a Bayesian network for identifying bimanual gestures that includes a first-hand node, a second-hand node, and a bimanual gesture node, wherein the instructions that, when executed, cause the computing device to identify a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures further include instructions that, when executed, cause the computing device to perform operations comprising;

inputting, to the first-hand node of the Bayesian network, the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

inputting, to the second-hand node of the Bayesian network, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

using the first-hand node of the Bayesian network to determine a first belief of a known bimanual gesture that corresponds to the tracked movement of the first hand based on the input first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

using the second-hand node of the Bayesian network to determine a second belief of a known bimanual gesture that corresponds to the tracked movement of the second hand based on the input second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

inputting the determined first belief and the determined second belief to the bimanual gesture root node; and

using the bimanual gesture root node to identify a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the determined first belief and the determined second belief.

24. An apparatus comprising:

means for tracking movement of a first hand during an observed bimanual gesture;

means for tracking movement of a second hand during the observed bimanual gesture;

means for determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

means for determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures; and

means for identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture, based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

wherein the means for tracking movement of a first hand during an observed bimanual gesture comprise means for extracting a sequence of images of the first hand from a sequence of images of the observed bimanual gesture;

wherein the means for tracking movement of a second hand during an observed bimanual gesture comprise means for extracting a sequence of images of the second hand from the sequence of images of the observed bimanual gesture;

wherein the means for determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures comprise means for;

identifying a sequence of principal components corresponding to the sequence of images of the first hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the first hand, using trained Hidden Markov Models that model first-hand movements of the one or more known bimanual gestures to determine the first set of measures of likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures; and

wherein determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to one or more known bimanual gestures comprises;

identifying a sequence of principal components corresponding to the sequence of images of the second hand, and

based on the identified sequence of principal components corresponding to the sequence of images of the second hand, using trained Hidden Markov Models that model second-hand movements of the one or more known bimanual gestures to determine the second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures.

25. An apparatus comprising:

means for tracking movement of a first hand during an observed bimanual gesture;

means for tracking movement of a second hand during the observed bimanual gesture;

means for determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

means for determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

means for identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture, based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures; and

means for maintaining a Bayesian network for identifying bimanual gestures that includes a first-hand node, a second-hand node, and a bimanual gesture node, wherein the means for identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures comprise means for;

inputting, to the first-hand node of the Bayesian network, the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

inputting, to the second-hand node of the Bayesian network, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

using the first-hand node of the Bayesian network to determine a first belief of a known bimanual gesture that corresponds to the tracked movement of the first hand based on the input first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures;

using the second-hand node of the Bayesian network to determine a second belief of a known bimanual gesture that corresponds to the tracked movement of the second hand based on the input second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

inputting the determined first belief and the determined second belief to the bimanual gesture root node; and

using the bimanual gesture root node to identify a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the determined first belief and the determined second belief.

26. An apparatus comprising:

means for tracking movement of a first hand during an observed bimanual gesture;

means for tracking movement of a second hand during the observed bimanual gesture;

means for determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures;

means for determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

means for identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture, based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures and the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures;

means for tracking movement of the first and second hands as a unit during an occlusion period of the bimanual gesture; and

means for determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures;

wherein the means for tracking movement of a first hand during an observed bimanual gesture comprise means for tracking movement of the first hand during a non- occlusion period of the observed bimanual gesture;

wherein the means for tracking movement of a second hand during the observed bimanual gesture comprise means for tracking movement of the second hand during the non- occlusion period of the observed bimanual gesture;

wherein the means for determining a first set of measures of likelihoods that the tracked movement of the first hand corresponds to one or more known bimanual gestures comprise means for determining a first set of measures of likelihoods that the tracked movement of the first hand during the non-occlusion period corresponds to the one or more known bimanual gestures;

wherein the means for determining a second set of measures of likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures comprise means for determining a second set of measures of likelihoods that the tracked movement of the second hand during the non-occlusion period corresponds to the one or more known bimanual gestures; and

wherein the means for identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture comprise means for identifying a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand corresponds to the one or more known bimanual gestures, and the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occlusion period corresponds to the one or more known bimanual gestures.

27. An apparatus comprising:

means for tracking movement of a first hand during a first non-occluded segment of an observed bimanual gesture;

means for tracking movement of a second hand during the first non-occluded segment of the observed bimanual gesture;

means for determining that the first and second hands are occluded, wherein a determination that the first and second hands are occluded signals a beginning of an occluded segment of the observed bimanual gesture;

means for tracking movement of the first and second hands as a unit during the occluded segment of the observed bimanual gesture;

means for determining that the first and second hands are no longer occluded, wherein a determination that the first and second hands are no longer included signals a beginning of a second non-occluded segment of the observed bimanual gesture;

means for distinguishing, after determining that the first and second hands are no longer occluded, the first hand from the second hand;

means for, after distinguishing the first hand from the second hand, tracking movement of the first hand during the second non-occluded end segment of the observed bimanual gesture;

means for, after distinguishing the first hand from the second hand, tracking movement of the second hand during the second non-occluded segment of the observed bimanual gesture;

means for determining a first set of measures of likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to one or more known bimanual gestures;

means for determining a second set of measures of likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures;

means for determining a third set of measures of likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures;

means for determining a fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures;

means for determining a fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures; and

means for identifying, by a computing device that includes hardware, a particular one of the one or more known bimanual gestures as corresponding to the observed bimanual gesture based on the first set of measures of the likelihoods that the tracked movement of the first hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the second set of measures of the likelihoods that the tracked movement of the second hand during the first non-occluded segment corresponds to the one or more known bimanual gestures, the third set of measures of the likelihoods that the tracked movement of the first and second hands as a unit during the occluded segment corresponds to the one or more known bimanual gestures, the fourth set of measures of likelihoods that the tracked movement of the first hand during the second non-occluded segment corresponds to the one or more known bimanual gestures, and the fifth set of measures of likelihoods that the tracked movement of the second hand during the second non-occluded segment corresponds to the one or more known bimanual gestures.