Method for tracking an object in an image sequence

• US 10,255,692 B2
• Filed: 04/04/2017
• Issued: 04/09/2019
• Est. Priority Date: 04/12/2016
• Status: Active Grant

• Current Assignee: City University of Hong Kong, Hong Kong Baptist University
• Original Assignee: City University of Hong Kong
• Inventors: Chen, Logn, Zhao, Zhongying, Yan, Hong
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US15/478,354
• Publication Number:
  US 20170294028A1
• Time in Patent Office: 735 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10076 4D tomography; Time-sequential 3D tomography
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/35 using statistical methods

Abstract

A method for tracking a biological cell in an image sequence with at least a first image and a second image includes the steps of: identifying a plurality of biological cells in the first image and a plurality of biological cells in the second image; determining matching probability of a biological cell in the second image with a biological cell in the first image, based on relative positional relationships between the plurality of biological cells in the first image and relative positional relationships between the plurality of biological cells in the second image; and determining, based on the determined matching probability, a matching result relating to whether the biological cell in the second image matches with the biological cell in the first image. A match indicates that the biological cell in the second image corresponds with or originates from the corresponding biological cell in the first image.

Claims

1. A method for tracking a biological cell in an image sequence, the image sequence including at least, a first image and a second image taken during development of biological cells in which biological cells are movable and dividable, the method comprising the steps of:

identifying, using one or more processors, a plurality of biological cells in the first image and a plurality of biological cells in the second image;

determining, using the one or more processors, a matching probability of a biological cell in the second image with a biological cell in the first image, based on relative positional relationships between the plurality of biological cells in the first image and relative positional relationships between the plurality of biological cells in the second image; and

determining, using the one or more processors and based on the determined matching probability, a matching result indicating whether the biological cell in the second image matches with the biological cell in the first image, wherein a match indicates that the biological cell in the second image corresponds with or originates from the corresponding biological cell in the first image; and

displaying, at an output device operable connected with the one or more processors, the matching result.

2. The method in accordance with claim 1, wherein the identification step comprises the step of:

determining positions of a plurality of biological cells in the first image and a plurality of biological cells in the second image by segmentation.

3. The method in accordance with claim 1, wherein the relative positional relationships between the plurality of biological cells in the first image comprises relative positional relationship between each of the respective biological cells in the first image with each of the other respective biological cells in the first image;

and the relative positional relationships between the plurality of biological cells in the second image comprises relative positional relationship between each of the respective biological cells in the second image with each of the other respective biological cells in the second image.

4. The method in accordance with claim 1, further comprising the step of:

repeating the matching probability determination step and the matching determination step for each of the plurality of biological cells in the first image and each of the plurality of biological cells in the second image.

5. The method in accordance with claim 4, further comprising the step of:

generating a diagram showing a correspondence between biological cells in the second image with biological cells the first image based on the matching result.

6. The method in accordance with claim 5, wherein the diagram is a tree diagram.

7. The method in accordance with claim 1, wherein the identification step comprises the step of:

defining a plurality of biological cells in the first image at a first time point t as a set Q_t={q₁, . . . , q_i, . . . , q_x, . . . , q_n} and a plurality of biological cells in the second image at a second time point t+1 as a set S_t+1={s₁, . . . , s_j, . . . , s_y, . . . , s_m}, where n≥

2 and m≥

2.

8. The method in accordance with claim 7, wherein the matching probability determination step comprises the steps of:

calculating a compatibility coefficient γ

to determine compatibility of a biological cell q_i in the first image matching a biological cell s_j in the second image and a biological cell q_x in the first image matching a biological cell s_y in the second image, wherein the larger the compatibility coefficient, the higher the probability that biological cell q_i matches biological cell s_j and biological cell q_x matches biological cell s_y.

9. The method in accordance with claim 8, wherein the step of calculating the compatibility coefficient γ

comprises the steps of;

calculating a distance compatibility coefficient α

between a biological cell pair in the first image and a biological cell pair in the second image;

calculating an angle compatibility coefficient β

between a vector defined by the biological cell pair in the first image and a vector defined by the biological cell pair in the second image; and

calculating the compatibility coefficient γ

by multiplying the distance compatibility coefficient α

with the angle compatibility coefficient β

.

10. The method in accordance with claim 9, wherein the distance compatibility coefficient α

is calculated based on

11. The method in accordance with claim 10, wherein the angle compatibility coefficient β

is calculated based on

12. The method in accordance with claim 8, further comprising the step of:

repeating the compatibility coefficient γ

calculation for each the plurality of biological cells in the first image and each the plurality of biological cells in the second image.

13. The method in accordance with claim 12, wherein the matching probability determination step further comprises the steps of:

determining an initial matching probability of biological cell s_j in the second image matching biological cell q_i in the first image based on the determined compatibility coefficients γ

;

repeating the initial matching probability determination step of each of the biological cells in the second image and each of the biological cells in the first image so as to form a matching matrix between the first image and the second image using the initial matching probability determined for the plurality of biological cells in the second image and the plurality of biological cells in the first image; and

iteratively updating the matching matrix based on one or more predetermined equations to determine the matching probability.

14. The method in accordance with claim 13, wherein the initial matching probability of biological cell s_j in the second image matching biological cell q_i in the first image is determined based on

p_j→

i=Σ

_x=1ⁿΣ

_y=1^mp<sub>y→

x</sub>⁰γ

_ijxy^t,t+1.

15. The method in accordance with claim 14, further comprising the step of normalizing the initial matching probability based on

16. The method in accordance with claim 15, wherein the matching matrix between the first image at the first time point t and the second image at the second time point t+1 has the form of:

17. The method in accordance with claim 15, wherein the matching matrix is iteratively updated based on the following equations:

18. The method in accordance with claim 15, wherein the matching matrix is iteratively updated based on the following equations:

19. The method in accordance with claim 13, further comprising the step of:

stopping the iteration when all the values in an updated matching matrix becomes close to 0 or close to 1, where a value close to 0 indicates that biological cell s_j in the second image and biological cell q_i in the first image are not matched and a value close to 1 indicates that biological cell s_j in the second image and biological cell q_i in the first image are matched.

20. The method in accordance with claim 1, wherein images in the image sequence are microscopic images.

21. A system for tracking an biological cell in an image sequence, the image sequence including at least, a first image and a second image, taken during development of biological cells in which biological cells are moveable and dividable, the system comprising:

one or more processors arranged to identify a plurality of biological cells in the first image and a plurality of biological cells in the second image;

determine matching probability of an biological cell in the second image with an biological cell in the first image, based on relative positional relationships between the plurality of biological cells in the first image and relative positional relationships between the plurality of biological cells in the second image; and

determine, based on the determined matching probability, a matching result indicating whether the biological cell in the second image matches with the biological cell in the first image, wherein a match indicates that the biological cell in the second image corresponds with or originates from the corresponding biological cell in the first image; and

a display, operably connected with the one or more processors, arranged to display the matching result.

22. The system in accordance with claim 21, wherein the one or more processors are further arranged to repeat the matching probability determination step and the matching determination step for each of the plurality of biological cells in the first image and each of the plurality of biological cells in the second image.

23. The system in accordance with claim 21, further comprising:

an imaging device, operably connected with the one or more processors, arranged to obtain the first image and the second image.