VISUAL LANGUAGE FOR HUMAN COMPUTER INTERFACES

• US 20140253429A1
• Filed: 03/14/2013
• Published: 09/11/2014
• Est. Priority Date: 03/08/2013
• Status: Active Grant

• Current Assignee: FastVDO LLC
• Original Assignee: FastVDO LLC
• Inventors: Dai, Wei, Topiwala, Pankaj, Krishnan, Madhu Peringassery
• Granted Patent: US 9,524,028 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

Embodiments of the invention recognize human visual gestures, as captured by image and video sensors, to develop a visual language for a variety of human computer interfaces. One embodiment provides a method for recognizing a hand gesture positioned by a user hand. The method includes steps of capturing a digital color image of a user hand against a background, applying a general parametric model to the digital color image of the user hand to generate a specific parametric template of the user hand, receiving a second digital image of the user hand positioned to represent a hand gesture, detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand, and recognizing the hand gesture based at least in part on the detected hand contour. Other embodiments include recognizing hand gestures, facial gestures or body gestures captured in a video.

Claims

1. A computer-implemented method for recognizing hand gestures, the method comprising:

capturing a digital color image of a user hand against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user hand to generate a specific parametric template of the user hand;

receiving a second digital image of the user hand, wherein the user hand in the second digital image is positioned to represent a hand gesture;

detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand; and

recognizing the hand gesture based at least in part on the detected hand contour.

2. The method of claim 1, wherein applying the general parametric model comprises:

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selection of the plurality of pixels being defined by the general parametric model; and

generating the specific parametric template using the selected plurality of the pixels in the digital image of the user hand.

3. The method of claim 2, wherein selecting the plurality of pixels from the pixels in the digital image comprises:

applying the general parametric model to the digital image of the user hand, the general parametric model having a predefined hand outline superimposed in the digital image of the user hand; and

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selected plurality of pixels being in the predefined hand outline.

4. The method of claim 2, wherein selecting the plurality of pixels from the pixels in the digital image further comprises:

applying the general parametric model to the digital image of the user hand, the general parametric model having a predefined region of interest superimposed in the digital image of the user hand;

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selected plurality of pixels being in the predefined region of interest.

5. The method of claim 2, wherein generating the specific parametric template using the selected plurality of the pixels comprises:

applying a color model to the selected pixels of the digital image of the user hand;

generating a skin map of the user hand; and

generating the specific parametric template based on the generated skin map of the user hand.

6. The method of claim 1, further comprising:

applying two independent general parametric models to the digital image of the user hand; and

generating two specific parametric templates using a plurality of the pixels in the digital image of the user hand;

combining the two specific parametric templates to obtain an improved parametric template of user hand.

7. The method of claim 6, further comprising:

applying two independent color space models to the digital image of the user hand;

generating two skin maps of the user hand using a plurality of the pixels in the digital image of the user hand; and

combining the two skin maps to generate an improved skin map of the user hand.

8. The method of claim 7, wherein a first color space model of the two independent color space models is selected from a hue-saturation-value color space and the first color model is represented by hue, saturation and value parameters.

9. The method of claim 7, wherein a second color space model of the two independent color space models is selected from a luma-chroma color space and the second color model is represented by luma and chroma parameters.

10. The method of claim 7, wherein applying two independent color space models to the digital image of the user hand comprises:

for a color space of a color space model chosen from the color spaces of the two independent color space models;

calculating one or more first-order statistics of the color space model in the chosen color space;

calculating one or more first-order statistics of the color space model in a red-green-blue color space;

determining distribution ranges of each parameter of the chosen color space model in the chosen color space; and

determining distribution ranges of each parameter of the chosen color space model in the red-green-blue color space.

11. The method of claim 7, wherein generating the improved skin map of the user hand comprises:

performing a logical inclusive OR operation to the two generated skin maps, the logical inclusive OR operation selecting a pixel of the digital image of the user hand responsive to the pixel being in one of the two generated skin maps; and

generating the improved skin map of the user hand based on the logical inclusive OR operation.

12. The method of claim 1, wherein generating the specific parametric template using the selected plurality of the pixels further comprises:

segmenting pixels representing the user hand from the pixels representing the background of the digital image of the user hand using the specific parameter template of the user hand.

13. The method of claim 1, wherein detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand comprises:

detecting pixels of the hand gesture that represent noise;

regrouping a plurality of isolated regions of the hand gesture image into one or more segments based on the specific parametric template of the user hand; and

selecting the largest segment of the hand gesture image to represent the hand contour of the hand gesture.

14. The method of claim 1, wherein recognizing the hand gesture based at least in part on the detected hand contour comprises:

approximating the detected hand contour by one or more polygonal segments;

finding a convex hull of the approximated polygonal hand contour;

extracting on or more features of the detected hand contour based on an analysis of convexity of the approximated polygonal hand contour; and

recognizing the hand gesture based on the extracted features.

15. The method of claim 14, wherein extracting one or more features of the detected hand contour comprises:

detecting one or more convexity defects of the convex hull, a convexity defect comprising a start point, a depth point and an end point, each of the point being representing by a pixel of the second digital image of the user hand;

identifying center of the digital image of the user hand;

filtering one or more points that do not represent finger tips or finger joints of the user hand; and

refining the convexity defects based on the filtering result to generate a refined polygon approximating the detected hand contour.

16. The method of claim 15, further comprising:

comparing the refined polygon approximating the detected hand contour with a set of known hand gestures; and

determining the hand gesture based on the comparison.

17. The method of claim 1, wherein recognizing the hand gesture based at least in part on the detected hand contour further comprises:

comparing the detected hand contour with a set of known hand gestures; and

determining the hand gesture based on the comparison.

18. The method of claim 17, wherein the set of known hand gestures comprises the American Sign Language set of hand gestures.

19. A computer-implemented method for recognizing hand gestures, the method comprising:

receiving an input video of a user hand against a background, the input video comprising a plurality of video frames of the user hand captured using a digital image capturing device, and a video frame of the plurality video frame representing a digital color image of the user hand at a time instance;

selecting a video frame of the plurality of the video frames of the input video as a reference frame;

applying a general parametric model to the selected video frame of the user hand to generate a specific parameter template of the user hand;

subsequently receiving one or more video frames of input video of the user hand, wherein the user hand in a subsequently received video frame is positioned to represent a hand gesture;

for each subsequently received video frame, detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand and a motion vector associated with the received video frame; and

recognizing the hand gesture based at least in part on the detected hand contours and motion vectors associated with the subsequently received video frames.

20. The method of claim 19, wherein the hand gesture is recognized as one of a known set of hand gestures.

21. A computer-implemented method for recognizing facial gestures, the method comprising:

capturing a digital color image of a user face against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user face to generate a specific parametric template of the user face;

continuously receiving additional digital images of the user face, wherein the user face in the additional digital images is configured to represent a facial gesture, with or without motion;

detecting a series of facial features and associated motion vectors of the facial gesture utilizing the specific parametric template of the user face; and

recognizing the facial gesture based at least in part on the detected facial gestures and associated motion vectors.

22. A computer-implemented method for recognizing body gestures, the method comprising:

capturing a digital color image of a user body against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user body to generate a specific parametric template of the user body;

continuously receiving additional digital images of the user body, wherein the user body in the additional digital images is configured to represent a body gesture, with or without motion;

detecting a series of body features and associated motion vectors of the body gesture utilizing the specific parametric template of the user body; and

recognizing the body gesture based at least in part on the detected body gestures and associated motion vectors.

23. A computer-implemented method for recognizing hand gestures, the method comprising:

capturing a color digital image of a user hand against a background, the digital image captured using a digital image capturing device;

selecting a plurality of pixels in the digital image that represent the user hand;

registering a user hand by applying a general parametric model to the selected pixels to generate a specific parametric model (parametric template) of the user hand;

receiving a second digital image of the user hand, wherein the user hand in the second digital image is positioned to represent a hand gesture;

detecting a hand contour of the hand gesture utilizing the parametric template of the user hand; and

recognizing the hand gesture utilizing the detected hand contour.

24. A non-transitory computer-readable storage medium storing executable computer program instructions for recognizing hand gestures, the computer programs instructions comprising code for:

capturing a digital color image of a user hand against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user hand to generate a specific parametric template of the user hand;

receiving a second digital image of the user hand, wherein the user hand in the second digital image is positioned to represent a hand gesture;

detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand; and

recognizing the hand gesture based at least in part on the detected hand contour.

25. The computer-readable storage medium of claim 24, wherein the computer program instructions for applying the general parametric model comprise computer program instructions for:

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selection of the plurality of pixels being defined by the general parametric model; and

generating the specific parametric template using the selected plurality of the pixels in the digital image of the user hand.

26. The computer-readable storage medium of claim 24, wherein the computer program instructions for selecting the plurality of pixels from the pixels in the digital image comprise computer program instructions for:

applying the general parametric model to the digital image of the user hand, the general parametric model having a predefined hand outline superimposed in the digital image of the user hand; and

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selected plurality of pixels being in the predefined hand outline.

27. The computer-readable storage medium of claim 25, wherein the computer program instructions for selecting the plurality of pixels from the pixels in the digital image further comprise computer program instructions for:

applying the general parametric model to the digital image of the user hand, the general parametric model having a predefined region of interest superimposed in the digital image of the user hand;

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selected plurality of pixels being in the predefined region of interest.

28. The computer-readable storage medium of claim 25, wherein the computer program instructions for generating the specific parametric template using the selected plurality of the pixels comprise computer program instructions for:

applying a color model to the selected pixels of the digital image of the user hand;

generating a skin map of the user hand; and

generating the specific parametric template based on the generated skin map of the user hand.

29. The computer-readable storage medium of claim 24, further comprising:

applying two independent general parametric models to the digital image of the user hand; and

generating two specific parametric templates using a plurality of the pixels in the digital image of the user hand;

combining the two specific parametric templates to obtain an improved parametric template of user hand.

30. The computer-readable storage medium of claim 29, further comprising computer program instructions for:

applying two independent color space models to the digital image of the user hand;

generating two skin maps of the user hand using a plurality of the pixels in the digital image of the user hand; and

combining the two skin maps to generate an improved skin map of the user hand.

31. The computer-readable storage medium of claim 30, wherein a first color space model of the two independent color space models is selected from a hue-saturation-value color space and the first color model is represented by hue, saturation and value parameters.

32. The method of claim 30, wherein a second color space model of the two independent color space models is selected from a luma-chroma color space and the second color model is represented by luma and chroma parameters.

33. The computer-readable storage medium of claim 30, wherein the computer program instructions for applying two independent color space models to the digital image of the user hand comprise computer program instructions for:

for a color space of a color space model chosen from the color spaces of the two independent color space models;

calculating one or more first-order statistics of the color space model in the chosen color space;

calculating one or more first-order statistics of the color space model in a red-green-blue color space;

determining distribution ranges of each parameter of the chosen color space model in the chosen color space; and

determining distribution ranges of each parameter of the chosen color space model in the red-green-blue color space.

34. The computer-readable storage medium of claim 30, wherein the computer program instructions for generating the improved skin map of the user hand comprise computer program instructions for:

performing a logical inclusive OR operation to the two generated skin maps, the logical inclusive OR operation selecting a pixel of the digital image of the user hand responsive to the pixel being in one of the two generated skin maps; and

generating the improved skin map of the user hand based on the logical inclusive OR operation.

35. The computer-readable storage medium of claim 24, wherein the computer program instructions for generating the specific parametric template using the selected plurality of the pixels further comprise computer program instructions for:

segmenting pixels representing the user hand from the pixels representing the background of the digital image of the user hand using the specific parameter template of the user hand.

36. The computer-readable storage medium of claim 24, wherein the computer program instructions for detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand comprise computer program instructions for:

detecting pixels of the hand gesture that represent noise;

regrouping a plurality of isolated regions of the hand gesture image into one or more segments based on the specific parametric template of the user hand; and

selecting the largest segment of the hand gesture image to represent the hand contour of the hand gesture.

37. The computer-readable storage medium of claim 24, wherein the computer program instructions for recognizing the hand gesture based at least in part on the detected hand contour comprise computer program instructions for:

approximating the detected hand contour by one or more polygonal segments;

finding a convex hull of the approximated polygonal hand contour;

extracting on or more features of the detected hand contour based on an analysis of convexity of the approximated polygonal hand contour; and

recognizing the hand gesture based on the extracted features.

38. The computer-readable storage medium of claim 37, wherein the computer program instructions for extracting one or more features of the detected hand contour comprise computer program instructions for:

detecting one or more convexity defects of the convex hull, a convexity defect comprising a start point, a depth point and an end point, each of the point being representing by a pixel of the second digital image of the user hand;

identifying center of the digital image of the user hand;

filtering one or more points that do not represent finger tips or finger joints of the user hand; and

refining the convexity defects based on the filtering result to generate a refined polygon approximating the detected hand contour.

39. The computer-readable storage medium of claim 38, further comprising computer program instructions for:

comparing the refined polygon approximating the detected hand contour with a set of known hand gestures; and

determining the hand gesture based on the comparison.

40. The computer-readable storage medium of claim 24, wherein recognizing the hand gesture based at least in part on the detected hand contour further comprises:

comparing the detected hand contour with a set of known hand gestures; and

determining the hand gesture based on the comparison.

41. The computer-readable storage medium of claim 40, wherein the set of known hand gestures comprises American Sign Language set of hand gestures.

42. A non-transitory computer-readable storage medium storing executable computer program instructions for recognizing hand gestures, the computer program instructions comprising code for:

receiving an input video of a user hand against a background, the input video comprising a plurality of video frames of the user hand captured using a digital image capturing device, and a video frame of the plurality video frame representing a digital color image of the user hand at a time instance;

selecting a video frame of the plurality of the video frames of the input video as a reference frame;

applying a general parametric model to the selected video frame of the user hand to generate a specific parameter template of the user hand;

subsequently receiving one or more video frames of input video of the user hand, wherein the user hand in a subsequently received video frame is positioned to represent a hand gesture;

for each subsequently received video frame, detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand and a motion vector associated with the received video frame; and

recognizing the hand gesture based at least in part on the detected hand contours and motion vectors associated with the subsequently received video frames.

43. The computer-readable storage medium of claim 42, wherein the hand gesture is recognized as one of a known set of hand gestures.

44. A non-transitory computer-readable storage medium storing executable computer program instructions for recognizing facial gestures, the computer program instructions comprising code for:

capturing a digital color image of a user face against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user face to generate a specific parametric template of the user face;

continuously receiving additional digital images of the user face, wherein the user face in the additional digital images is configured to represent a facial gesture, with or without motion;

detecting a series of facial features and associated motion vectors of the facial gesture utilizing the specific parametric template of the user face; and

recognizing the facial gesture based at least in part on the detected facial gestures and associated motion vectors.

45. A non-transitory computer-readable storage medium storing executable computer program instructions for recognizing body gestures, the computer program instructions comprising code for:

capturing a digital color image of a user body against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user body to generate a specific parametric template of the user body;

continuously receiving additional digital images of the user body, wherein the user body in the additional digital images is configured to represent a body gesture, with or without motion;

detecting a series of body features and associated motion vectors of the body gesture utilizing the specific parametric template of the user body; and

recognizing the body gesture based at least in part on the detected body gestures and associated motion vectors.

46. A non-transitory computer-readable storage medium storing executable computer program instructions for recognizing hand gestures, the computer program instructions comprising code for:

capturing a color digital image of a user hand against a background, the digital image captured using a digital image capturing device;

selecting a plurality of pixels in the digital image that represent the user hand;

registering a user hand by applying a general parametric model to the selected pixels to generate a specific parametric model (parametric template) of the user hand;

receiving a second digital image of the user hand, wherein the user hand in the second digital image is positioned to represent a hand gesture;

detecting a hand contour of the hand gesture utilizing the parametric template of the user hand; and

recognizing the hand gesture utilizing the detected hand contour.

47. An apparatus for recognizing hand gestures, the apparatus comprising:

a computer processor for performing steps comprising;

capturing a digital color image of a user hand against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user hand to generate a specific parametric template of the user hand;

receiving a second digital image of the user hand, wherein the user hand in the second digital image is positioned to represent a hand gesture;

detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand; and

recognizing the hand gesture based at least in part on the detected hand contour.

48. The apparatus of claim 47, wherein applying the general parametric model comprises:

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selection of the plurality of pixels being defined by the general parametric model; and

generating the specific parametric template using the selected plurality of the pixels in the digital image of the user hand.

49. The apparatus of claim 48, wherein selecting the plurality of pixels from the pixels in the digital image comprises:

applying the general parametric model to the digital image of the user hand, the general parametric model having a predefined hand outline superimposed in the digital image of the user hand; and

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selected plurality of pixels being in the predefined hand outline.

50. The apparatus of claim 48, wherein selecting the plurality of pixels from the pixels in the digital image further comprises:

applying the general parametric model to the digital image of the user hand, the general parametric model having a predefined region of interest superimposed in the digital image of the user hand;

selecting a plurality of pixels from the pixels in the digital image that represents the user hand, the selected plurality of pixels being in the predefined region of interest.

51. The apparatus of claim 48, wherein generating the specific parametric template using the selected plurality of the pixels comprises:

applying a color model to the selected pixels of the digital image of the user hand;

generating a skin map of the user hand; and

generating the specific parametric template based on the generated skin map of the user hand.

52. The apparatus of claim 47, further comprising:

applying two independent general parametric models to the digital image of the user hand; and

generating two specific parametric templates using a plurality of the pixels in the digital image of the user hand;

combining the two specific parametric templates to obtain an improved parametric template of user hand.

53. The apparatus of claim 52, further comprising:

applying two independent color space models to the digital image of the user hand;

generating two skin maps of the user hand using a plurality of the pixels in the digital image of the user hand; and

combining the two skin maps to generate an improved skin map of the user hand.

54. The apparatus of claim 53, wherein a first color space model of the two independent color space models is selected from a hue-saturation-value color space and the first color model is represented by hue, saturation and value parameters.

55. The apparatus of claim 53, wherein a second color space model of the two independent color space models is selected from a luma-chroma color space and the second color model is represented by luma and chroma parameters.

56. The apparatus of claim 53, wherein applying two independent color space models to the digital image of the user hand comprises:

for a color space of a color space model chosen from the color spaces of the two independent color space models;

calculating one or more first-order statistics of the color space model in the chosen color space;

calculating one or more first-order statistics of the color space model in a red-green-blue color space;

determining distribution ranges of each parameter of the chosen color space model in the chosen color space; and

determining distribution ranges of each parameter of the chosen color space model in the red-green-blue color space.

57. The apparatus of claim 53, wherein generating the improved skin map of the user hand comprises:

performing a logical inclusive OR operation to the two generated skin maps, the logical inclusive OR operation selecting a pixel of the digital image of the user hand responsive to the pixel being in one of the two generated skin maps; and

generating the improved skin map of the user hand based on the logical inclusive OR operation.

58. The apparatus of claim 47, wherein generating the specific parametric template using the selected plurality of the pixels further comprises:

segmenting pixels representing the user hand from the pixels representing the background of the digital image of the user hand using the specific parameter template of the user hand.

59. The apparatus of claim 47, wherein detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand comprises:

detecting pixels of the hand gesture that represent noise;

regrouping a plurality of isolated regions of the hand gesture image into one or more segments based on the specific parametric template of the user hand; and

selecting the largest segment of the hand gesture image to represent the hand contour of the hand gesture.

60. The apparatus of claim 47, wherein recognizing the hand gesture based at least in part on the detected hand contour comprises:

approximating the detected hand contour by one or more polygonal segments;

finding a convex hull of the approximated polygonal hand contour;

extracting on or more features of the detected hand contour based on an analysis of convexity of the approximated polygonal hand contour; and

recognizing the hand gesture based on the extracted features.

61. The apparatus of claim 60, wherein extracting one or more features of the detected hand contour comprises:

detecting one or more convexity defects of the convex hull, a convexity defect comprising a start point, a depth point and an end point, each of the point being representing by a pixel of the second digital image of the user hand;

identifying center of the digital image of the user hand;

filtering one or more points that do not represent finger tips or finger joints of the user hand; and

refining the convexity defects based on the filtering result to generate a refined polygon approximating the detected hand contour.

62. The apparatus of claim 61, further comprising:

comparing the refined polygon approximating the detected hand contour with a set of known hand gestures; and

determining the hand gesture based on the comparison.

63. The apparatus of claim 47, wherein recognizing the hand gesture based at least in part on the detected hand contour further comprises:

comparing the detected hand contour with a set of known hand gestures; and

determining the hand gesture based on the comparison.

64. The apparatus of claim 63, wherein the set of known hand gestures comprises American Sign Language set of hand gestures.

65. An apparatus for recognizing hand gestures, the apparatus comprising:

a computer processor for performing steps comprising;

receiving an input video of a user hand against a background, the input video comprising a plurality of video frames of the user hand captured using a digital image capturing device, and a video frame of the plurality video frame representing a digital color image of the user hand at a time instance;

selecting a video frame of the plurality of the video frames of the input video as a reference frame;

applying a general parametric model to the selected video frame of the user hand to generate a specific parameter template of the user hand;

subsequently receiving one or more video frames of input video of the user hand, wherein the user hand in a subsequently received video frame is positioned to represent a hand gesture;

for each subsequently received video frame, detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand and a motion vector associated with the received video frame; and

recognizing the hand gesture based at least in part on the detected hand contours and motion vectors associated with the subsequently received video frames.

66. The apparatus of claim 65, wherein the hand gesture is recognized as one of a known set of hand gestures.

67. An apparatus for recognizing facial gestures, the apparatus comprising:

a computer processor for performing steps comprising;

capturing a digital color image of a user face against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user face to generate a specific parametric template of the user face;

continuously receiving additional digital images of the user face, wherein the user face in the additional digital images is configured to represent a facial gesture, with or without motion;

detecting a series of facial features and associated motion vectors of the facial gesture utilizing the specific parametric template of the user face; and

recognizing the facial gesture based at least in part on the detected facial gestures and associated motion vectors.

68. An apparatus for recognizing body gestures, the apparatus comprising:

a computer processor for performing steps comprising;

capturing a digital color image of a user body against a background, the digital image captured using a digital image capturing device;

applying a general parametric model to the digital color image of the user body to generate a specific parametric template of the user body;

continuously receiving additional digital images of the user body, wherein the user body in the additional digital images is configured to represent a body gesture, with or without motion;

detecting a series of body features and associated motion vectors of the body gesture utilizing the specific parametric template of the user body; and

recognizing the body gesture based at least in part on the detected body gestures and associated motion vectors.

69. An apparatus for recognizing hand gestures, the apparatus comprising:

a computer processor for performing steps comprising;

capturing a color digital image of a user hand against a background, the digital image captured using a digital image capturing device;

selecting a plurality of pixels in the digital image that represent the user hand;

registering a user hand by applying a general parametric model to the selected pixels to generate a specific parametric model (parametric template) of the user hand;

receiving a second digital image of the user hand, wherein the user hand in the second digital image is positioned to represent a hand gesture;

detecting a hand contour of the hand gesture utilizing the parametric template of the user hand; and

recognizing the hand gesture utilizing the detected hand contour.

70. An apparatus for creating visual codes, the apparatus comprising:

a computer processor for performing steps comprising;

receiving one or more user input commands and a plurality of human visual gestures;

encoding the plurality of human visual gestures into one or more visual codes based on the user input commands; and

storing the encoded visual codes in a computer storage medium.

71. The apparatus of claim 70, wherein the user input commands are received using at least one of following:

tactile input by the user;

live feed from a camera;

voice commands;

hand configurations of the user;

eye movements of the user;

handwritings of the user;

finger or hand movements of the user;

facial movements of the user; and

body movements of the user.

72. The apparatus of claim 70, wherein the user input commands are created using an authoring tool having one or more shorthand instruction codes included in a digital composition codebook.

73. The apparatus of claim 70, wherein the plurality of the human visual gestures comprise at least one of:

a plurality of hand gestures;

a plurality of facial gestures; and

a plurality of body gestures.

74. The apparatus of claim 70, wherein the visual codes are for video editing and the user input commands comprise at least one of:

capturing a sensory signal, wherein the sensory signal includes at least a video signal or an audio signal, and the capturing records the sensory signal;

expressing completion of action;

transitioning from a first visual scene to a second visual scene;

blending a digital representation of the user face into a visual scene;

blending a digital representation of another user face into a visual scene that contains a digital representation of the user face;

adding a sound track selected by the user to a visual scene; and

composing a composite sound track based on a plurality of sound tracks selected by the user.

75. The apparatus of claim 74, wherein a visual code of the visual code for video editing comprises one or more corresponding human visual gestures.

76. The apparatus of claim 70, wherein encoding the plurality of human visual gestures into one or more visual codes based on the user input commands comprises:

generating one or more shortcuts of a visual code, wherein the one or more shortcuts associated with a visual code form a hierarchy; and

generating a digital composition codebook comprising the visual codes and shorthand instruction codes associated with the visual codes, wherein the visual codes and shorthand instruction codes associated with the visual codes are structured based on the hierarchy among the visual codes and the shorthand instruction codes.