Online domain adaptation for multi-object tracking

• US 9,984,315 B2
• Filed: 05/05/2015
• Issued: 05/29/2018
• Est. Priority Date: 05/05/2015
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Gaidon, Adrien, Vig, Eleonora
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Kwan, Matthew
• Application Number:
  US14/704,350
• Publication Number:
  US 20160328613A1
• Time in Patent Office: 1,120 Days
• Field of Search: 348153
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

Methods and systems for online domain adaptation for multi-object tracking. Video of an area of interest can be captured with an image-capturing unit. The video (e.g., video images) can be analyzed with a pre-trained object detector utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure to jointly adapt online all trackers associated with the pre-trained object detector and a pre-trained category-level model from the trackers in order to efficiently track a plurality of objects in the video captured by the image-capturing unit.

Claims

1. A method for online domain adaptation for multi-object tracking, said method comprising:

pre-training an object detector and a category-level model, wherein said pre-trained object detector is trained offline for at least one category of interest using a general-purpose labeled dataset and wherein said pre-trained object detector is associated with a plurality of trackers;

capturing video of an area of interest with a video camera; and

analyzing said video with said pre-trained object detector utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure, wherein said convex multi-task learning and said associated self-tuning stochastic optimization procedure jointly adapt online all trackers among said plurality of trackers associated with said pre-trained object detector and said pre-trained category-level model from said trackers to efficiently track a plurality of objects in said video captured by said video camera and wherein said associated self-tuning stochastic optimization procedure includes a use of learning rates and regularization parameters in which an update of at least one tracker of among said trackers includes a contribution of all other trackers among said trackers including both current and past trackers thereof, and wherein said learning rates are automatically set per-frame and per-target with respect to said video.

2. The method of claim 1 wherein said self-tuning stochastic optimization procedure comprises self-tuning of hyper-parameters.

3. The method of claim 2 further comprising selecting said hyper-parameters including at least two critical hyper-parameters comprising a learning rate among said learning rates and a regularization factor among said regularization parameters.

4. The method of claim 2 wherein said convex multi-task learning comprises logistic regression and hard negative mining.

5. The method of claim 1 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

6. The method of claim 5 wherein said convex multi-task learning comprises logistic regression and hard negative mining.

7. The method of claim 6 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

8. A system for online domain adaptation for multi-object tracking, said system comprising:

an image capturing unit that captures video of an area of interest; and

a pre-trained object detector that is pre-trained offline for at least one category of interest using a general-purpose labeled dataset and wherein said pre-trained object detector is associated with a plurality of trackers, wherein said pre-trained object detector analyzes said video utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure, wherein said convex multi-task learning and said associated self-tuning stochastic optimization procedure jointly adapt online all trackers among said plurality of trackers associated with said pre-trained object detector and a pre-trained category-level model from said trackers to efficiently track a plurality of objects in said video captured by said image capturing unit and wherein said associated self-tuning stochastic optimization procedure includes a use of learning rates and regularization parameters in which an update of at least one tracker of among said trackers includes a contribution of all other trackers among said trackers including both current and past trackers thereof, and wherein said learning rates are automatically set per-frame and per-target with respect to said video.

9. The system of claim 8 wherein said self-tuning stochastic optimization procedure comprises self-tuning of hyper-parameters.

10. The system of claim 9 wherein said convex multi-task learning comprises logistic regression and hard negative mining.

11. The system of claim 9 wherein said hyper-parameters include at least two critical hyper-parameters comprising a learning rate among said learning rates and a regularization factor from among said regularization parameters.

12. The system of claim 11 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

13. The system of claim 8 wherein said convex muti-task learning comprises logistic regression and hard negative mining.

14. The system of claim 13 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

15. A system for online domain adaptation for multi-object tracking, said system comprising:

at least one processor; and

a non-transitory computer-usable medium embodying computer program code, said computer-usable medium capable of communicating with said at least one processor, said computer program code comprising instructions executable by said at least one processor and configured for;

pre-training an object detector and a category-level model, wherein said pre-trained object detector is trained offline for at least one category of interest using a general-purpose labeled dataset and wherein said pre-trained object detector is associated with a plurality of trackers;

capturing video of an area of interest with a video camera; and

analyzing said video with said pre-trained object detector utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure, wherein said convex multi-task learning and said associated self-tuning stochastic optimization procedure jointly adapt online all trackers among said plurality of trackers associated with said pre-trained object detector and said pre-trained category-level model from said trackers to efficiently track a plurality of objects in said video captured by said video camera and wherein said associated self-tuning stochastic optimization procedure includes a use of learning rates and regularization parameters in which an update of at least one tracker of among said trackers includes a contribution of all other trackers among said trackers including both current and past trackers thereof, and wherein said learning rates are automatically set per-frame and per-target with respect to said video.

16. The system of claim 15 wherein said self-tuning stochastic optimization procedure comprises self-tuning of hyper-parameters.

17. The system of claim 16 wherein said instructions further comprise selecting said hyper-parameters including at least two critical hyper-parameters comprising a learning rate among said learning rates and a regularization factor from said regularization parameters.

18. The system of claim 17 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back.

19. The system of claim 16 wherein said pre-trained object detector comprises a category-level detector that moves from said at least one category of interest to instances and back, and wherein said convex multi-task learning comprises logistic regression and hard negative mining.

20. The system of claim 15 wherein said convex multi-task learning comprises logistic regression and hard negative mining.