Manipulation input device which detects human hand manipulations from captured motion images

• US 8,378,970 B2
• Filed: 10/20/2008
• Issued: 02/19/2013
• Est. Priority Date: 10/22/2007
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Nishida, Yoshihiro, Otsuka, Isao, Okada, Reiko, Matsutani, Kiyoshi, Tsuji, Masayuki
• Primary Examiner(s): Negron, Daniell L
• Application Number:
  US12/289,063
• Publication Number:
  US 20090102788A1
• Time in Patent Office: 1,583 Days
• Field of Search: None
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

When a vehicle navigation system is manipulated by taking pictures of a user hand motion and gesture with a camera, as the number of apparatuses and operational objects increases, the associated hand shapes and hand motions increase, thus causing a complex manipulation for a user. Furthermore, in detecting a hand with the camera, when the image of a face having color tone information similar to that of a hand appears in an image taken with a camera, or outside light rays such as sun rays or illumination rays vary, detection accuracy is reduced. To overcome such problems, a manipulation input device is provided that includes a limited hand manipulation determination unit and a menu representation unit, whereby a simple manipulation can be achieved and manipulation can accurately be determined. In addition, detection accuracy can be improved by a unit that selects a single result from results determined by a plurality of determination units, based on images taken with a plurality of cameras.

Claims

1. A manipulation input device, comprising:

a plurality of different kinds of picture-taking apparatuses which include a visible light camera;

a plurality of hand region detectors that detect a region of a human hand from a motion image taken with each of the plurality of picture-taking apparatus;

a plurality of hand manipulation determination units that determine a hand manipulation from a shape and a motion of the detected hand region;

a hand manipulation determination and selection unit that selects a single result from results determined by the plurality of hand manipulation determination units; and

a selection menu representation unit that notifies a user of a menu selected based on the manipulation selected by the hand manipulation determination and selection unit,

wherein at least one of the hand region determination units comprises;

a skin tone region extraction unit that extracts a region of a skin tone from a motion image taken with the visible light camera;

a difference region extraction unit that extracts region data differing between current motion image data and background image data in an image taken with the visible light camera;

a binary correction unit that digitizes image data of the region extracted by the skin tone region extraction unit and the difference region extraction unit, to then correct the digitized image data;

a distance calculator that calculates minimum distances from inner points of the binary image region obtained by the binary correction unit to an edge thereof, to detect as the center of a hand a distance maximum point where the calculated values become a maximum one;

a center weighted correction unit that adds to the distance value of the distance maximum point a correction value such that the closer the position of the distance maximum point is to the center of an imaging screen, the larger the correction value, and the closer the position thereof is to the end portion of the imaging screen, the smaller the correction value;

a candidate hand region detector that detects a hand region from a binary image region that includes a point where the distance value of the distance maximum point to which the correction value is added is a maximum;

an edge length and area calculator that calculates an edge length, and an area, of the candidate hand region; and

a hand region determination unit that determines whether or not the candidate hand region is a hand from the calculated edge length and area.

2. The manipulation input device of claim 1, wherein the different kinds of picture-taking apparatuses include an infrared camera.