Interface apparatus and interface method

• US 7,844,921 B2
• Filed: 06/01/2007
• Issued: 11/30/2010
• Est. Priority Date: 08/25/2006
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Yuasa, Mayumi, Okada, Ryuzo, Kishikawa, Nobuhisa, Stenger, Bjorn, Taniguchi, Yasuhiro, Ike, Tsukasa, Yokoi, Kentaro
• Primary Examiner(s): Bashore; William L
• Assistant Examiner(s): Hur; Ece
• Application Number:
  US11/757,159
• Publication Number:
  US 20080052643A1
• Time in Patent Office: 1,278 Days
• Field of Search: 715/863, 345/358
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An apparatus for a user to interface with a control object apparatus by a posture or a motion of the user'"'"'s physical part. An image input unit inputs an image including the user'"'"'s physical part. A gesture recognition unit recognizes the posture or the motion of the user'"'"'s physical part from the image. A control unit controls the control object apparatus based on an indication corresponding to the posture or the motion. A gesture information display unit displays an exemplary image of the posture or the motion recognized for the user'"'"'s reference to indicate the control object apparatus.

Claims

1. An apparatus for interfacing with a control object apparatus, comprising:

an image input unit configured to input an image including a user'"'"'s physical part;

a gesture recognition unit configured to recognize a posture or a motion of the user'"'"'s physical part from the image;

a control unit configured to control the control object apparatus based on an indication corresponding to the posture or the motion; and

a gesture information display unit configured to display an exemplary image of the posture or the motion recognized by the gesture recognition unit, wherein the gesture recognition unit includes a gesture evaluation unit configured to calculate a probability that the posture or the motion is included in each partial region of the image;

a candidate region selection unit configured to select at least one partial region having a highest probability as a physical part candidate region; and

a gesture decision unit configured to decide that the posture or the motion is included in the physical part candidate region when the probability is above a threshold, which indication the posture or motion included in the physical part candidate region represents, and the gesture information display unit displays the posture or the motion recognized by the gesture recognition unit, the probability that the posture or motion is included in each partial region of the image, and a graph of a probability that the posture or the motion represents each indication.

2. The apparatus according to claim 1, wherein the gesture information display unit displays an image of the physical part candidate region.

3. The apparatus according to claim 2, wherein the gesture information display unit discriminately displays the image of the physical part candidate region based on the indication represented by the posture or the region.

4. The apparatus according to claim 1, wherein the gesture information display unit discriminately displays the probability that the posture or the motion represents each indication.

5. The apparatus according to claim 1, wherein the gesture information display unit additionally displays a mark of the threshold on the graph.

6. The apparatus according to claim 1, wherein the gesture information display unit discriminately displays an exemplary image of the indication represented by the posture or the motion included in the physical part candidate region.

7. The apparatus according to claim 6, wherein the gesture information display unit displays the exemplary image and the graph, each corresponding to the same indication.

8. The apparatus according to claim 7, wherein the gesture information display unit displays a plurality of menus, a plurality of exemplary images, and a graph of each exemplary image, and discriminately displays at least one of the plurality of menus based on the indication.

9. A method for interfacing with a control object apparatus, comprising:

inputting an image including a user'"'"'s physical part;

recognizing a posture or a motion of the user'"'"'s physical part from the image;

controlling the control object apparatus based on an indication corresponding to the posture or the motion; and

displaying an exemplary image of the posture or the motion recognized for a user'"'"'s reference to indicate the control object apparatus,

wherein the recognizing includes calculating a probability that the posture or the motion is included in each partial region of the image;

selecting at least one partial region having a highest probability as a physical part candidate region;

deciding that the posture or the motion is included in the physical part candidate region when the probability is above a threshold, deciding which indication the posture or the motion included in the physical part candidate region represents, and the displaying includes displaying the posture or the motion recognized, and the probability that the posture or motion is included in each partial region of the image, and a graph of a probability that the posture or the motion represents each indication.

10. A computer readable medium storing computer-readable instructions thereon for a control object apparatus, the computer-readable instructions when executed by the control object apparatus cause the control object apparatus to perform a method comprising:

inputting an image including a user'"'"'s physical part;

recognizing a posture or a motion of the user'"'"'s physical part from the image;

controlling the control object apparatus based on an indication corresponding to the posture or the motion; and

displaying an exemplary image of the posture or the motion recognized for the user'"'"'s reference to indicate the control object apparatus,

wherein the recognizing includes

calculating a probability that the posture or the motion is included in each partial region of the image;

selecting at least one partial region having a highest probability as a physical part candidate region; and

deciding that the posture or the motion is included in the physical part candidate region when the probability is above a threshold,

deciding which indication the posture or motion included in the physical part candidate region represents, and

the displaying includes displaying

the posture or the motion recognized, and

the probability that the posture or motion is included in each partial region of the image, and

a graph of a probability that the posture or motion represents each indication.