Object tracking method and apparatus using stereo images

• US 20050216274A1
• Filed: 02/16/2005
• Published: 09/29/2005
• Est. Priority Date: 02/18/2004
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Hyunwoo
• Application Number:
  US11/058,203
• Publication Number:
  US 20050216274A1
• Time in Patent Office: Days
• US Class Current: 704/276
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

An object tracking method and apparatus, the method includes: segmenting a segment of a zone, in which an object is located, from a current frame among consecutively input images and obtaining predetermined measurement information of the segment; determining a plurality of searching zones centered around the segment and predicting parameters of the segment in the current frame based on measurement information of a preceding frame in the searching zones; selecting predetermined searching zones as partial searching candidate zones from the predicted parameters; measuring a visual cue of the segment; and estimating parameters of the segment of the current frame in the partial searching candidate zones based on the visual cue and the predicted parameters and determining parameters having the largest parameter values from the estimated parameters as parameters of the segment.

Claims

1. An object tracking method comprising:

segmenting a segment of a zone, in which an object is located, from a current frame among consecutively input images and obtaining predetermined measurement information of the segment;

determining a plurality of searching zones centered around the segment and predicting parameters of the segment in the current frame based on measurement information of a preceding frame in the plurality of the searching zones;

selecting predetermined searching zones as partial searching candidate zones from the predicted parameters of the segment in the current frame;

measuring a visual cue of the segment; and

estimating parameters of the segment of the current frame in the partial searching candidate zones based on the visual cue and the predicted parameters and determining parameters having largest estimated parameter values as parameters of the segment.

2. The method of claim 1, wherein the predetermined searching zones are zones that an ellipse having a size determined from geometry of an input inputting the images is divided into a plurality of directions centered around the segment.

3. The method of claim 1, wherein the predetermined measurement information of the segment is obtained by averaging depth information of the input images, which is measured along a straight line in any one direction in the segment.

4. The method of claim 3, wherein, when depth information of a kth frame is D_k and parameters of the segment are represented as x_k, the prediction is represented as a prior probability p(x_k|D_k-1) by the following equation:

p(x_k|D_k-1)≈

∫

p(x_k|x_k-1)p(x_k-1|D_k-1,{tilde over (D)}_k)dx_k-1

where {tilde over (D)}_k indicates the depth information partially obtained in the current frame with respect to the object.

5. The method of claim 4, wherein p(x_k-1|D_k-1, {tilde over (D)}_k) is calculated by the following equation:

p(x_k-1|D_k-1,{tilde over (D)}_k)=(y_k^depth)^Ty₀^depth

where y₀^depth indicates the depth information according to a one dimensional depth map of a reference segment and y_k^depth indicates the depth information according to depth maps of circumference of x_k-1 in the kth frame.

6. The method of claim 5, wherein, if the object is determined as a new object in the current frame, N positions are sampled in the searching zones centered around an initial position x₀ of the segment, and the prior probability of x₀ is determined as 1/N.

7. The method of claim 6, further comprising:

initializing information related to the new object by storing image information including an ID of the new object and depth and color information of the segment in a database.

8. The method of claim 7, wherein the determining of the new object comprises:

obtaining the image information; and

comparing the image information with values stored in the database, and determining the object as a new object if the image information is not substantially same with the values stored in the database.

9. The method of claim 7, wherein the initializing of the information related to the object comprises:

storing information including the ID of the new object, the image information, a central position and scale of the segment.

10. The method of claim 1, wherein the partial searching candidate zones are zones whose predicted parameter values are larger than a predetermined value, or a predetermined number of zones selected in order of a largest predicted parameter to lowest predicted parameter.

11. The method of claim 1, wherein the visual cue comprises at least one of probabilities of a color histogram, average depth information, motion information, and shape information measured with respect to the segment, or combinations thereof.

12. The method of claim 11, wherein the estimated parameters are normalized by a probability to be measured including at least one of a color histogram, average depth information, motion information, shape information in the current frame with respect to depth information measured with respect to the object in the preceding frame, or combinations thereof.

13. The method of claim 1, further comprising:

masking the segment;

searching another object by searching zones except the masked segment in the image; and

repeating from segmenting a segment of a zone through searching another object if another object exists.

14. The method of claim 13, further comprising:

searching an object, which does not appear in the current image, in the database, which stores information of the objects, and deleting the searched object from the database, if all objects in the image are masked.

30. A computer readable medium embedded with processing instructions for performing the method of claim 1 using a computer.

15. An object tracking apparatus comprising:

an image inputting unit consecutively inputting images including a zone having an object;

an image segmenting unit detecting and segmenting a segment of the zone from a current frame among the input images and obtaining predetermined measurement information of the segment;

a predicting unit determining a plurality of searching zones centered around the segment and predicting parameters of the segment in the current frame based on the measurement information of a preceding frame in the plurality of the searching zones;

a visual cue measuring unit measuring a visual cue including at least one of probabilities of average depth information, color information, motion information, shape information of the segment, or combinations thereof; and

a tracking unit estimating parameters of the segment for the current frame in the searching zones based on the visual cue and the predicted parameters and determining parameters having largest parameters among the estimated parameters as parameters of the segment for use in tracking the object in a future frame.

16. The apparatus of claim 15, wherein the predicting unit selects zones, that an ellipse having a size determined from geometry of the image inputting unit is divided into a plurality of directions centering around the segment, as the searching zones.

17. The apparatus of claim 16, wherein the image segmenting unit obtains an average depth information measured along a straight line of any one direction in the segment, as the measurement information of the segment.

18. The apparatus of claim 17, wherein the predicting unit, when depth information of a kth frame is D_k and parameters of the segment are represented as x_k, predicts the parameters according to a prior probability p(x_k|D_k-1) using the following equation:

p(x_k|D_k-1)≈

∫

p(x_k|x_k-1)p(x_k-1|D_k-1,{tilde over (D)}_k)dx_k-1

where {tilde over (D)}_k indicates depth information partially obtained in the current frame with respect to the object.

19. The apparatus of claim 18, wherein the predicting unit obtains p(x_k-1, D_k-1, {tilde over (D)}_k) using the following equation:

p(x_k-1|D_k-1,{tilde over (D)}_k)=(y_k^depth)^Ty₀^depth

where y₀^depth indicates depth information according to a one dimensional depth map of a reference segment and y_k^depth indicates depth information according to depth maps of circumference of x_k-1 in the kth frame.

20. The apparatus of claim 15, further comprising:

a database; and

an initializing unit storing depth and color information of the segment with an ID of the new object in the database and initializing parameters of the segment, if the object is a new object.

21. The apparatus of claim 20, wherein the tracking unit further comprises:

a mask, masking the segment to classify the segment from other zones of the image, when the parameters of the segment are determined.

22. An object tracking method comprising:

detecting an object from an input image;

determining a position of the detected object;

calculating possible prior positions of the detected object;

measuring a visual cue of the detected object in order to estimate a post position of the detected object; and

calculating the post position of the detected object from the visual cue.

23. The method of claim 22, wherein the object is detected using a depth variation pattern and/or a depth variation range using the input image.

24. The method of claim 22, wherein the detected object is determined to be a new detected object based on reference information stored in a database storing previously detected objects.

25. The method of claim 24, wherein if the detected object is determined to be a new object, storing an ID and related information of the new object in the database.

26. The method of claim 25, wherein if the detected object is not the new object, the detected object is not again included in the database.

27. The method of claim 22, wherein the position of the detected object is determined by a search ellipse centered around the object.

28. A computer readable medium embedded with processing instructions for performing the method of claim 22 using a computer.

29. The method of claim 22, further comprising determining whether the detected object is a new object through review of a database of previously detected objects.

31. An object tracking apparatus to track objects in images having corresponding frames, the apparatus comprising:

an image segmenting unit detecting an object in a corresponding zone of an image and segmenting a segment of the zone from a current frame and obtaining predetermined measurement information of the segment;

a predicting unit determining at least one search zone centered around the segment and predicting parameters of the segment in the current frame based on the measurement information of a preceding frame in the at least one searching zone;

a visual cue measuring unit measuring a visual cue using the segment; and

a tracking unit estimating parameters of the segment for the current frame in the search zone based on the visual cue and the predicted parameters and determining parameters having largest parameters among the estimated parameters as parameters of the segment for use in tracking the object in a future frame.