Dynamic image processing method and device and medium

• US 6,993,157 B1
• Filed: 05/18/2000
• Issued: 01/31/2006
• Est. Priority Date: 05/18/1999
• Status: Expired due to Fees

• Current Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Oue, Yasuhiro, Sugimoto, Kazuhide
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Bhatnagar, Anand
• Application Number:
  US09/719,374
• Time in Patent Office: 2,084 Days
• Field of Search: 382/103, 382/104, 382/107, 382/100, 382/169, 382/170, 382/171, 382/173, 382/181, 382/203, 348169-172, 348/208.14, 715/863, 345156-184, 713/150, 713/186
• US Class Current: 382/103
• CPC Class Codes: B60R 25/2045 by hand gestures
• CPC Class Codes: B60R 25/252 Fingerprint recognition
• CPC Class Codes: B60R 25/305 using a camera
• CPC Class Codes: E05F 15/00 Power-operated mechanisms for wings motor-operated accessories in locks for completing closing or initiating opening of a wing E05B17/00
• CPC Class Codes: E05Y 2400/86 by hand
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G07C 9/00563 using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

A motion image processing method and device for authenticating a user using a specific device, using motion information of an object. Time series monochrome images, obtained by photographing an object, using a camera, are input. An object is detected from an initial frame of the input time series images, using a basic shape characteristic, and a plurality of reference points to be tracked are automatically determined in the object. Then, corresponding points of the respective reference points are detected in an image frame other than the initial frame among the input time series images. Subsequently, motion information of a finger is calculated, based on the result of tracking the respective reference points and an assumption of limited motion in a 3D plane. Based on the calculated motion parameter, a solid object is subjected to coordinate conversion, and displayed if necessary. As a result of the tracking, a reference point in each frame is updated. Tracking, motion parameter calculation, and displaying are repeated with respect to subsequent frames.

Claims

1. A motion image processing method comprising:

extracting a shape characteristic of an object contained in time series images by processing an initial frame of the time series images, wherein extracting the shape characteristic includes extracting a contour based on edge information of an input image of the object, detecting characteristic points consisting of a bending point, which is a point with a quick change of an edge gradient direction based on edge information, a curving point, which is a point on a contour with inversion of a curvature sign and a moderate change of an edge gradient direction, and a transition point, which is a point on a contour with a change of a curvature from zero to another value, or vice versa, and a moderate change of the edge gradient direction, dividing the contour into a straight line segment and a concave or convex curve segment based on the characteristic points, and combining segments obtained in the same frame, whereby a specific portion is detected in the image obtained by projecting the object;

detecting, using the shape characteristic, reference points on a same plane in a 3D space for tracking motion, in an image in the initial frame, and a point corresponding to each of the reference points in a frame different in time from the initial frame to be tracked;

tracking the reference points over time to calculate motion information of the object in the 3D space, wherein motion information of the plane in the 3D space is calculated, based on at least four reference points for tracking and corresponding points obtained in a plurality of time series frames, and also on an assumption of limited motion for a plane in a 3D space such that pairs of reference points and corresponding points are located on the same plane in the 3D space; and

modifying a display object including a model or a pointer provided beforehand, based on the motion information.

2. A motion image processing method according to claim 1, wherein at least one of the reference points for tracking is determined on a contour of the specific portion detected in the image obtained by projecting the object.

3. A motion image processing method according to claim 1, wherein a shape characteristic of the contour of the object is expressed using a plurality of segments defined by characteristic points.

4. A motion image processing method according to claim 1, wherein at least one of the reference points for tracking a corresponding point is determined on the segment.

5. A motion image processing device comprising:

means for extracting shape characteristics of an object contained in time series images by processing an initial frame of the time series images, wherein the means for extracting shape characteristics extracts a contour based on edge information of an input image of the object, detects characteristic points consisting of a bending point, which is a point with a quick change of an edge gradient direction based on edge information, a curving point, which is a point on a contour with inversion of a curvature sign and a moderate change of the edge gradient direction, and a transition point, which is a point on a contour with a change of a curvature from zero to another value, or vice versa, and a moderate change of an edge gradient direction, divides the contour into a straight line segment and a concave or convex curve segment based on a characteristic point, and combines segments obtained in the same frame, whereby a specific portion is detected in the image obtained by projecting the object;

means for detecting a reference point in the frame, using the shape characteristics;

means for tracking the reference point over time to calculate motion information of the object in a 3D space; and

means for modifying a display object including a model or a pointer, provided beforehand, based on the motion information, wherein motion information of a plane in the 3D space is calculated, based on a plurality of reference points for tracking and corresponding points obtained in a plurality of time series frames, and also on an assumption of limited motion for a plane in a 3D space such that pairs of the reference points and corresponding points are located on the plane in the 3D space.

6. A motion image processing device according to claim 5, wherein at least one of the reference points for tracking is determined on a contour of the specific portion detected in the image obtained by projecting the object.

7. A motion image processing device according to claim 5, wherein a shape characteristic of the contour of the object is expressed using a plurality of segments defined by characteristic points.

8. A motion image processing device according to claim 5, wherein at least one of the reference points for tracking a corresponding point is determined on the segment.

9. A motion image processing device according to claim 5, wherein at least four points on a same plane in a 3D space are selected, as reference points for tracking motion, in an image in an initial frame, and a corresponding point corresponding to each of the reference points is detected in a frame different in time from the initial frame to be tracked.

10. A computer-readable medium storing a program which when executed by a computer performs:

extracting a shape characteristic of an object contained in time series images by processing an initial frame of the time series images, wherein extracting the shape characteristic includes extracting a contour based on edge information of an input image of the object, detecting characteristic points consisting of a bending point, which is a point with a quick change of an edge gradient direction based on edge information, a curving point, which is a point on a contour with inversion of a curvature sign and a moderate change of the edge gradient direction, and a transition point, which is a point on a contour with a change of a curvature from zero to other value, or vice versa, and a moderate change of an edge gradient direction, dividing the contour into a straight line segment and a concave or convex curve segment based on the characteristic points, and combining segments obtained in same frame, whereby a specific portion is detected in the projected image of the object;

detecting, using the shape characteristic, reference points on a same plane in a 3D space for tracking motion, in an image in the initial frame, and a point corresponding to each of the reference points is detected in a frame different in time from the initial frame to be tracked;

tracking the reference point over time to calculate the motion information of the object in the 3D space, wherein motion information of the plane in the 3D space is calculated, based on at least four reference points for tracking and corresponding points obtained in a plurality of time series frames, and also on an assumption of limited motion for a plane in a 3D space such that pairs of reference points and corresponding points are located on the same plane in the 3D space; and

modifying a display object including a model or a pointer provided beforehand, based on the motion information.

11. A medium according to claim 10, wherein at least one of the reference points for tracking is determined on a contour of the specific portion detected in the image obtained by projecting the object.

12. A medium according to claim 10, wherein a shape characteristic of the contour of the object is expressed using a plurality of segments defined by characteristic points.

13. A medium according to claim 10, wherein at least one of the reference points for tracking a corresponding points is determined on the segment.