METHOD FOR TRACKING AN OBJECT IN AN IMAGE SEQUENCE

• US 20170294028A1
• Filed: 04/04/2017
• Published: 10/12/2017
• Est. Priority Date: 04/12/2016
• Status: Active Application

• Current Assignee: City University of Hong Kong, Hong Kong Baptist University
• Original Assignee: City University of Hong Kong, Hong Kong Baptist University
• Inventors: Chen, Logn, Zhao, Zhongying, Yan, Hong
• Granted Patent: US 10,255,692 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10076 4D tomography; Time-sequential 3D tomography
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/35 using statistical methods

Abstract

A method for tracking an object in an image sequence with at least a first image and a second image includes the steps of: identifying a plurality of objects in the first image and a plurality of objects in the second image; determining matching probability of an object in the second image with an object in the first image, based on relative positional relationships between the plurality of objects in the first image and relative positional relationships between the plurality of objects in the second image; and determining, based on the determined matching probability, a matching result relating to whether the object in the second image matches with the object in the first image. A match indicates that the object in the second image corresponds with or originates from the corresponding object in the first image.

Claims

1. A method for tracking an object in an image sequence with at least a first image and a second image, the method comprising the steps of:

identifying a plurality of objects in the first image and a plurality of objects in the second image;

determining matching probability of an object in the second image with an object in the first image, based on relative positional relationships between the plurality of objects in the first image and relative positional relationships between the plurality of objects in the second image; and

determining, based on the determined matching probability, a matching result relating to whether the object in the second image matches with the object in the first image, wherein a match indicates that the object in the second image corresponds with or originates from the corresponding object in the first image.

2. The method in accordance with claim 1, wherein the identification step comprises the step of:

determining positions of a plurality of objects in the first image and a plurality of objects in the second image by segmentation.

3. The method in accordance with claim 1, wherein the relative positional relationships between the plurality of objects in the first image comprises relative positional relationship between each of the respective objects in the first image with each of the other respective objects in the first image;

and the relative positional relationships between the plurality of objects in the second image comprises relative positional relationship between each of the respective objects in the second image with each of the other respective objects in the second image.

4. The method in accordance with claim 1, further comprising the step of:

repeating the matching probability determination step and the matching determination step for each of the plurality of objects in the first image and each of the plurality of objects in the second image.

5. The method in accordance with claim 1, wherein the identification step comprises the step of:

defining a plurality of objects in the first image at a first time point t as a set Q_t={q₁, . . . , q_i, . . . , q_x, . . . , q_n} and a plurality of objects in the second image at a second time point t+1 as a set S_t+1={s₁, . . . , s_j . . . , s_y . . . , s_m}, where n≧

2 and m≧

2.

6. The method in accordance with claim 5, wherein the matching probability determination step comprises the steps of:

calculating a compatibility coefficient γ

to determine compatibility of an object q_i in the first image matching an object s_j in the second image and an object q_x in the first image matching an object s_y in the second image, wherein the larger the compatibility coefficient, the higher the probability that object q_i matches object s_j and object q_x matches object s_y.

7. The method in accordance with claim 6, wherein the step of calculating the compatibility coefficient γ

comprises the steps of;

calculating a distance compatibility coefficient α

between an object pair in the first image and an object pair in the second image;

calculating an angle compatibility coefficient β

between a vector defined by the object pair in the first image and a vector defined by the object pair in the second image;

calculating the compatibility coefficient γ

by multiplying the distance compatibility coefficient α

with the angle compatibility coefficient β

.

10. The method in accordance with claim 6, further comprising the step of:

repeating the compatibility coefficient γ

calculation for each the plurality of objects in the first image and each the plurality of objects in the second image.

11. The method in accordance with claim 10, wherein the matching probability determination step further comprises the steps of:

determining an initial matching probability of object s_j in the second image matching object q_i in the first image based on the determined compatibility coefficients γ

;

repeating the initial matching probability determination step of each of the objects in the second image and each of the objects in the first image so as to form a matching matrix between the first image and the second image using the initial matching probability determined for the plurality of objects in the second image and the plurality of objects in the first image;

iteratively updating the matching matrix based on one or more predetermined equations to determine the matching probability.

12. The method in accordance with claim 11, wherein the initial matching probability of object s_j in the second image matching object q_i in the first image is determined based on

p_j→

i=Σ

_x=1ⁿΣ

_y=1^mp<sub>y→

x</sub>⁰γ

_ijxy^t,t+1.

13. The method in accordance with claim 12, further comprising the step of normalizing the initial matching probability based on

14. The method in accordance with claim 13, wherein the matching matrix between the first image at the first time point t and the second image at the second time point t+1 has the form of:

15. The method in accordance with claim 13, wherein the matching matrix is iteratively updated based on the following equations:

16. The method in accordance with claim 13, wherein the matching matrix is iteratively updated based on the following equations:

17. The method in accordance with claim 11, further comprising the step of:

stopping the iteration when all the values in an updated matching matrix becomes close to 0 or close to 1, where a value close to 0 indicates that object s_j in the second image and object q_i in the first image are not matched and a value close to 1 indicates that object s_j in the second image and object q_i in the first image are matched.

18. The method in accordance with claim 4, further comprising the step of:

generating a diagram showing a correspondence between objects in the second image with objects the first image based on the matching result.

19. The method in accordance with claim 18, wherein the diagram is a tree diagram.

20. The method in accordance with claim 1, wherein images in the image sequence are microscopic images.

21. The method in accordance with claim 1, wherein the objects are biological cells.

22. The method in accordance with claim 21, wherein images in the image sequence are taken during development of the biological cells, wherein the biological cells are movable and dividable from one image to another.

8. The method in accordance with claim 8, wherein the distance compatibility coefficient α

is calculated based on

9. The method in accordance with claim 8, wherein the angle compatibility coefficient β

is calculated based on

23. A system for tracking an object in an image sequence with at least a first image and a second image, comprising:

one or more processors arranged to

identify a plurality of objects in the first image and a plurality of objects in the second image;

determine matching probability of an object in the second image with an object in the first image, based on relative positional relationships between the plurality of objects in the first image and relative positional relationships between the plurality of objects in the second image; and

determine, based on the determined matching probability, a matching result relating to whether the object in the second image matches with the object in the first image, wherein a match indicates that the object in the second image corresponds with or originates from the corresponding object in the first image.

24. The system in accordance with claim 23, wherein the one or more processors are further arranged to repeat the matching probability determination step and the matching determination step for each of the plurality of objects in the first image and each of the plurality of objects in the second image

25. The system in accordance with claim 23, further comprising:

an imaging device arranged to obtain the first image and the second image.

26. The system in accordance with claim 23, further comprising:

a display arranged to display the matching result.

27. A non-transient computer readable medium for storing computer instructions that, when executed by one or more processors, causes the one or more processors to perform a system for tracking an object in an image sequence with at least a first image and a second image, the method comprising the steps of:

identifying a plurality of objects in the first image and a plurality of objects in the second image;

determining matching probability of an object in the second image with an object in the first image, based on relative positional relationships between the plurality of objects in the first image and relative positional relationships between the plurality of objects in the second image; and

determining, based on the determined matching probability, a matching result relating to whether the object in the second image matches with the object in the first image, wherein a match indicates that the object in the second image corresponds with or originates from the corresponding object in the first image.