Action recognizing apparatus and recording medium in that action recognizing program is recorded

• US 6,335,977 B1
• Filed: 05/28/1998
• Issued: 01/01/2002
• Est. Priority Date: 05/28/1997
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Kage, Hiroshi
• Primary Examiner(s): Au, Amelia M.
• Assistant Examiner(s): MILLER, MARTIN E
• Application Number:
  US09/084,950
• Time in Patent Office: 1,314 Days
• Field of Search: 382/153, 382/154, 382/103, 382/107, 345/474, 345/473
• US Class Current: 382/107
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/8029 Fighting without shooting
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Motion of a human under various conditions is reproduced by a computer graphics and the like in a real time manner without using auxiliary means such as a light reflection marker. An action recognizing apparatus is arranged by an image input unit 1 for imaging an image containing motion of an imaging object at a plurality of imaging angles by a plurality of image sensors to thereby acquire a plurality of two-dimensional images; a two-dimensional motion detecting unit 2 for calculating a direction of motion and strength thereof of each of pixels within each of the two-dimensional images imaged, a region specifying unit 3 for specifying such a region having strength of the calculated motion larger than, or equal to a predetermined strength value; a three-dimensional motion detecting unit 4 for positioning the specific region specified with respect to the plurality of two-dimensional images, whereby a three-dimensional position of the specific region and three-dimensional motion of the specific region are calculated; a portion predicting unit 5 for predicting that the calculated three-dimensional motion corresponds to any of the portions of the imaging object; and an action reproducing unit 6 for moving a model in a simulation manner based the motion predicted by the portion predicting unit 5 with respect to each of the portions, whereby motion of the imaging object is reproduced on the model.

Claims

1. An action recognizing apparatus comprising:

an image input unit for forming images of an imaged object at a plurality of imaging angles with a plurality of image sensors, and acquiring a plurality of two-dimensional images indicating motion of the imaged object, each of the two-dimensional images including a plurality of pixels;

a two-dimensional motion detecting unit for calculating direction and magnitude of motion of the imaged object from the pixels of the two-dimensional images;

a region specifying unit for specifying at least one specific region within the two-dimensional images where the motion of the imaged object has a magnitude at least equal to a threshold motion magnitude, wherein said region specifying unit

subdivides the two-dimensional images into a plurality of scanning ranges which may overlap each other,

scans each of the scanning ranges to select as the specific regions the scanning ranges having maximum averaged motion magnitude, and

selects the scanning ranges having an averaged motion magnitude larger than the threshold motion magnitude as the specific regions;

a three-dimensional motion detecting unit for establishing position of a specific region specified by said region specifying unit relative to the two-dimensional images, and calculating a three-dimensional position of the specific region and a three-dimensional motion of the specific region;

a portion predicting unit for predicting whether the three-dimensional motion of the specific region calculated by said three-dimensional motion detecting unit corresponds to any of a plurality of respective portions of the imaged object; and

an action reproducing unit for moving a model in a simulation based on the three-dimensional motion predicted by said portion predicting unit with respect to each of the plurality of portions so that the motion of the imaged object is reproduced by the model.

2. The action recognizing apparatus as claimed in claim 1 wherein said action reproducing unit reproduces the motion of the imaged object as motion by a computer graphic model.

3. The action recognizing apparatus as claimed in claim 1 wherein the imaged object is a human and the portions correspond to body parts of the human including head, hands, and feet.

4. A recording medium in which an action recognizing program is recorded, the action recognizing program causing a computer to:

calculate a direction and magnitude of motion of an imaged object from two-dimensional images of the imaged object formed at a plurality of imaging angles, each of the two-dimensional images including a plurality of pixels;

specify a specific region of the plurality of two-dimensional images having a calculated motion magnitude at least equal to a threshold motion magnitude by

subdividing the two-dimensional images into a plurality of scanning ranges which may overlap each other,

scanning each of the scanning ranges to select as the specific regions the scanning ranges having maximum averaged motion magnitude, and

selecting the scanning ranges having an averaged motion magnitude larger than the threshold motion magnitude as the specific regions;

position the specific region with respect to the plurality of two-dimensional images and calculate a three-dimensional position of the specific region and three-dimensional motion in the specific region;

predict the three-dimensional motion of the specific region corresponding to any of a plurality of portions of the imaged object; and

move a model in a simulation based on motion with respect to each of the portions of the imaged object to reproduce motion of the imaged object by the model.