Motion-based tracking with pan-tilt-zoom camera

• US 20020167537A1
• Filed: 05/11/2001
• Published: 11/14/2002
• Est. Priority Date: 05/11/2001
• Status: Abandoned Application

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Trajkovic, Miroslav
• Application Number:
  US09/854,119
• Publication Number:
  US 20020167537A1
• Time in Patent Office: Days
• US Class Current: 345/649
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

A motion-estimation scheme is provided that employs a combination of motion estimation and compensation techniques. Low resolution images are computed from two consecutive image frames, and feature points are determined and matched between the two low resolution images. Statistical methods are used to estimate the motion in terms of a translation and rotation of the image plane. Corresponding feature points in the original images are matched, based on the estimated motion of the low-resolution images. Statistical techniques are then applied to determine a homography matrix that describes the motion between the corresponding feature points in the original images, and this matrix is used to align the original images. Differences between the aligned images are identified, to indicate the movement of one or more objects in the image.

Claims

1. A method of aligning a first image to a second image, comprising;

determining a first alignment approximation, based on distances between one or more points in the first image and the second image,

aligning the second image to the first image, based on the first alignment approximation, to form an initially aligned second image,

determining a second alignment approximation, based on distances between one or more points in the first image and the initially aligned second image, and

aligning the second image to the first image, based on a combination of the first and second alignment approximations.

2. The method of claim 1, wherein

aligning the second image to the first image based on the combination of the first and second alignment approximations is effected by:

aligning the initially aligned second image, which is based on the first alignment approximation, to the first image, based on the second alignment approximation.

3. The method of claim 1, wherein

determining the first alignment approximation is based on a low-resolution representation of the first and second images, and

determining the second alignment approximation is based on a higher-resolution representation of the first and second images.

4. The method of claim 1, wherein

determining at least one of the first alignment and second alignment approximations includes applying the RANSAC algorithm.

5. The method of claim 1, wherein

determining the first alignment approximation includes an approximation of at least one of a rotation component and a translation component in an image space of the first and second images.

6. The method of claim 5, wherein

determining the second alignment approximation includes an approximation of components of a 3×

3 homographic matrix.

7. The method of claim 1, wherein

determining the second alignment approximation includes an approximation of components of a 3×

3 homographic matrix.

8. The method of claim 1, wherein

determining at least one of the first and second alignment approximations includes

identifying corners in the first and second images based on a determination of Minimum Intensity Changes at the corners.

9. A method of tracking an object based on a first image and a second image, comprising:

aligning the first and second images to form a set of aligned images, and

detecting motion by comparing the set of aligned images,

wherein

aligning the first and second images includes;

determining a first alignment approximation, based on distances between one or more points in the first image and the second image,

aligning the second image to the first image, based on the first alignment approximation, to form an initially aligned second image,

determining a second alignment approximation, based on distances between one or more points in the first image and the initially aligned second image, and

aligning the second image to the first image, based on a combination of the first and second alignment approximations.

10. The method of claim 9, wherein

determining the first alignment approximation is based on a low-resolution representation of the first and second images, and

determining the second alignment approximation is based on a higher-resolution representation of the first and second images.

11. The method of claim 9, further including

identifying the object in the set of aligned images based on color matching.

12. The method of claim 9, further including

determining a location of the object in each image of the set of aligned images, and

determining a movement of the object by comparing the location of the object in each image.

14. The motion detecting system of claim 13, wherein

the processor is configured to:

determine the first alignment approximation by processing a low-resolution representation of at least one of the first and second images, and

determine the second alignment approximation by processing a higher-resolution representation of the first and second images.

15. The motion detecting system of claim 13, further including

one or more cameras for producing the first and second images.

16. The motion detecting system of claim 13, further including

a memory for storing a representation of a target image, and

wherein

the processor is further configured to identify a target within the set of aligned images, based on the representation of the target image.

17. The motion detecting system of claim 16, wherein

the representation of the target image is a characterization based on color content of the target image.

18. The motion detecting system of claim 13, further including

determining a location of an object in each image of the set of aligned images, and

determining a movement of the object by comparing the location of the object in each image.

19. The motion detecting system of claim 13, wherein

determining the first alignment approximation includes an approximation of at least one of a rotation component and a translation component.

20. The motion detecting system of claim 19, wherein

determining the second alignment approximation includes an approximation of components of a 3×

3 homographic matrix.

13. A motion detecting system comprising:

a processor that is configured to;

align a first image and a second image, to form a set of aligned images, by;

determining a first alignment approximation, based on distances between one or more points in the first image and the second image,

aligning the second image to the first image, based on the first alignment approximation, to form an initially aligned second image,

determining a second alignment approximation, based on distances between one or more points in the first image and the initially aligned second image, and

aligning the second image to the first image, based on a combination of the first and second alignment approximations; and

compare the set of aligned images to identify motion of objects within the first and second images.