Hand-gesture recognition method

• US 9,696,808 B2
• Filed: 12/17/2008
• Issued: 07/04/2017
• Est. Priority Date: 07/13/2006
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Nishihara, H. Keith, Kaehler, Adrian, Jangaard, Lars, Hsu, Shi-Ping
• Primary Examiner(s): Vu, Kieu
• Assistant Examiner(s): BASOM, BLAINE T
• Application Number:
  US12/337,225
• Publication Number:
  US 20090103780A1
• Time in Patent Office: 3,121 Days
• Field of Search: 715702, 715863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

One embodiment of the invention includes a method of providing device inputs. The method includes illuminating hand gestures performed via a bare hand of a user in a foreground of a background surface with at least one infrared (IR) light source. The method also includes generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface and generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface. The method also includes determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images. The method also includes determining a provided input gesture based on the plurality of three-dimensional features of the bare hand and comparing the provided input gesture with a plurality of predefined gesture inputs in a gesture library. The method further includes providing at least one device input corresponding to interaction with displayed visual content based on the provided input gesture corresponding to one of the plurality of predefined gesture inputs.

Claims

1. A method or providing device inputs, the method comprising:

illuminating hand gestures performed via a bare hand of a user in a three-dimensional foreground of a background surface with at least one infrared (IR) light source;

generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;

generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand the background surface;

determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images;

determining a first provided input gesture based on the plurality of three-dimensional features of the bare hand, the provided input gesture comprising the bare hand being positioned in a flat, open hand gesture and being rotated at an associated wrist approximately 180 degrees;

determining a second provided input gesture comprising the flat, open hand gesture being maintained subsequent to rotation of the associated wrist while at least one additional gesture input is performed from one of the bare hand and another bare hand of the user;

comparing the provided first and second input gestures with a plurality of predefined gesture inputs in a gesture library;

providing at least one first device input corresponding to interaction with displayed visual content based on the provided first input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one first device input responsive to the provided first input gesture comprising one of a page-up input and a page-down input associated with a document; and

providing at least one second device input corresponding to interaction with the displayed visual content based on the provided second input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one second device input responsive to the provided second input gesture comprising one of;

scrolling a page of the document one of upwards and downwards in response to the at least one additional gesture input from the bare hand;

repeating at least one of the page-up and the page-down inputs associated with the document in response to the at least one additional gesture input; and

auto-turning a page of the document at one of an acceleration and a deceleration that is based on a duration at which the at least one additional gesture input is maintained.

2. The method of claim 1, wherein determining the provided second input gesture further comprises determining a second input gesture comprising the bare hand being positioned in the flat, open hand gesture and being rotated at the wrist approximately 180 degrees while concurrently moving forward, and wherein providing the at least one device input comprises providing one of a plurality of page-up inputs and a plurality of page down inputs associated with the document responsive to the second input gesture.

3. A method of providing device inputs, the method comprising:

illuminating hand gestures performed via a bare hand of a user in a three-dimensional foreground of a background surface with at least one infrared (IR) light source;

generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;

generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;

determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images;

determining a provided input gesture based on the plurality of three-dimensional features of the bare hand, the provided input gesture comprising the bare hand switching from a first sub-gesture comprising an extended thumb, and extended index finger, and an extended middle finger to a second sub-gesture comprising a retracted thumb, an extended index finger, and an extended middle finger and is moved one of up, down, left, and right while maintaining the second sub-gesture;

comparing the provided input gesture with a plurality of predefined gesture inputs in a gesture library; and

providing at least one device input corresponding to interaction with displayed visual content based on the provided input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one device input responsive to the provided input gesture comprising tilting a view one of up and down in response to the movement of the maintained second sub-gesture up and down, respectively, and panning the view one of left and right in response to the movement of the maintained second sub-gesture left and right, respectively.

4. A method of providing device inputs, the method comprising:

illuminating hand gestures performed via a bare hand of a user in a three-dimensional foreground of a background surface with at least one infrared (IR) light source;

generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;

generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;

determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images;

determining a provided input gesture based on the plurality of three-dimensional features of the bare hand, the provided input gesture comprising the bare hand of the user pointing at the portion of the displayed visual content with an extended index finger and an extended thumb that is subsequently retracted followed by a rotation of an associated wrist;

comparing the provided input gesture with a plurality of predefined gesture inputs in a gesture library; and

providing at least one device input corresponding to interaction with displayed visual content based on the provided input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one device input responsive to the provided input gesture comprising tilting a view associated with the displayed visual content in response to the retraction of the thumb, the tilting being based on the rotation of the wrist.