System and method to improve object tracking using multiple tracking systems

• US 9,791,541 B2
• Filed: 12/19/2014
• Issued: 10/17/2017
• Est. Priority Date: 12/19/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Kim, Kyungnam, Jeong, Changsoo, Owechko, Yuri
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US14/578,046
• Publication Number:
  US 20160180197A1
• Time in Patent Office: 1,033 Days
• Field of Search: None
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

A method includes receiving first tracking data and a first confidence value at a tracker selection system from a first tracking system. The method includes receiving second tracking data and a second confidence value at the tracker selection system from a second tracking system. The tracking systems may be configured to track an object in a sequence of images, and the first and second tracking data may indicate locations of regions where the corresponding tracking system has tracked the object in an image of the sequence of images. The confidence values may indicate likelihoods that a corresponding tracking system is tracking the object. The method further includes providing output data to the first tracking system and to the second tracking system. The output data may include data selected from the first tracking data and the second tracking data based on a result of a comparison of the confidence values.

Claims

1. A method comprising:

receiving first tracking data and a first confidence value at a tracker selection system from a first tracking system, wherein the first tracking system is configured to track an object in a sequence of images using a first tracking algorithm, wherein the first tracking algorithm is an adaptive tracking algorithm configured to modify an object model by updating data representing the object, wherein the first tracking data indicates a first location of a first region where the first tracking system has tracked the object in an image of the sequence of images, and wherein the first confidence value indicates a likelihood that the first tracking system is tracking the object;

receiving second tracking data and a second confidence value at the tracker selection system from a second tracking system, wherein the second tracking system is configured to track the object in the sequence of images using a second tracking algorithm different from the first tracking algorithm, wherein the second tracking algorithm is a non-adaptive tracking algorithm configured to not modify the object model when updating data representing the object, wherein the second tracking data indicates a second location of a second region where the second tracking system has tracked the object in the image, and wherein the second confidence value indicates a likelihood that the second tracking system is tracking the object; and

providing output data to the first tracking system and to the second tracking system, wherein the output data includes data selected from the first tracking data and the second tracking data based on a result of a comparison of the first confidence value and the second confidence value.

2. The method of claim 1, wherein the first tracking system and the second tracking system are configured to track the object in a subsequent image of the sequence of images based on the output data.

3. The method of claim 1, wherein the output data includes the first tracking data when the first confidence value is greater than or equal to the second confidence value.

4. The method of claim 1, wherein the output data includes the second tracking data when the second confidence value exceeds the first confidence value, the method further comprising:

receiving, at the first tracking system, location data from a third tracking system configured to;

identify a location in a first image of the sequence of images based on a detection of a user selection of a region of the first image,

receive a second image of the sequence of images,

determine a group of pixels in a second image of the sequence of images that match the identified location, and

update the location data based on the determined group of pixels in the second image; and

receiving, at the second tracking system, the location data.

5. The method of claim 1, wherein the first tracking system is a feature-based tracking system configured to track the object based on feature data extracted from the sequence of images, and wherein the second tracking system is an instance-based tracking system configured to track the object based on one or more instances of the object in the sequence of images.

6. The method of claim 5, further comprising:

determining one or more characteristics of the sequence of images; and

determining whether the one or more characteristics include at least one characteristic of a first set of characteristics, wherein the output data includes the first tracking data when the one or more characteristics include at least one characteristic of the first set of characteristics.

7. The method of claim 6, further comprising:

determining whether the one or more characteristics include at least one characteristic of a second set of characteristics; and

disabling the first tracking system when the one or more characteristics include at least one characteristic of the second set of characteristics.

8. The method of claim 6, wherein the one or more characteristics include an image type, a number of pixels per image, a change in scale between images, a change in rotation between images, or a combination thereof.

9. The method of claim 1, wherein the first tracking system and the second tracking system are configured to receive location data from a third tracking system during a first time period, and wherein the first tracking system and the second tracking system do not process images during the first time period.

10. The method of claim 9, wherein the first time period includes an initialization time period of the first tracking system and the second tracking system, wherein the first tracking system and the second tracking system are configured to process one out of every n-number of images where n is a positive integer greater than or equal to 2, and wherein the third tracking s stem is configured to process each image of the sequence of images.

11. The method of claim 9, wherein the third tracking system is configured to track the object by:

determining a first group of pixels in a first image of the sequence of images that corresponds to the object;

comparing the first group of pixels to a second group of pixels in a second image of the sequence of images; and

determining the location data based on a location of the second group of pixels when a result of the comparison exceeds a threshold value, and wherein the location data include coordinates of a center of the second group of pixels and a size of the second group of pixels.

12. The method of claim 9, wherein the first tracking system and the second tracking system are configured to determine the first tracking data and the second tracking data based on the location data.

13. The method of claim 9, further comprising providing the output data to the third tracking system, wherein the third tracking system is further configured to generate second location data based on the output data.

14. A tracker selection system comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving first tracking data and a first confidence value from a first tracking system, wherein the first tracking system is configured to track an object in a sequence of images using a first tracking algorithm, wherein the first tracking algorithm is an adaptive tracking algorithm configured to modify an object model by updating data representing the object, wherein the first tracking data indicates a first location of a first region where the first tracking system has tracked the object in an image of the sequence of images, and wherein the first confidence value indicates a likelihood that the first tracking system is tracking the object;

receiving second tracking data and a second confidence value from a second tracking system, wherein the second tracking system is configured to track the object in the sequence of images using a second tracking algorithm different from the first tracking algorithm, wherein the second tracking algorithm is a non-adaptive tracking algorithm configured to not modify the object model when updating data representing the object, wherein the second tracking data indicates a second location of a second region where the second tracking system has tracked the object in the image, and

wherein the second confidence value indicates a likelihood that the second tracking system is tracking the object; and

providing output data to the first tracking system and to the second tracking system, wherein the output data includes data selected from the first tracking data and the second tracking data based on a result of a comparison of the first confidence value and the second confidence value.

15. The tracker selection system of claim 14, wherein the operations further comprise providing the output data to a sub-window tracking system configured to determine sub-window data based on the output data and to provide the sub-window data to the first tracking system and to the second tracking system.

16. The tracker selection system of claim 15, wherein the sub-window data indicates a location of a sub-window in a subsequent image of the sequence of images, and wherein the first tracking system and the second tracking system are configured to, based on receipt of the sub-window data, search the sub-window of the subsequent image to track the object when processing the subsequent image.

17. The tracker selection system of claim 14, wherein the sequence of images is received from an image capture device, and wherein the processor, the memory, the first tracking system, the second tracking system, and the image capture device are integrated in an autonomous vehicle.

18. The tracker selection system of claim 17, wherein the operations further comprise providing the output data to a server, to another system of the autonomous vehicle, or a combination thereof.

19. A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to:

receive first tracking data and a first confidence value at a tracker selection system from a first tracking system, wherein the first tracking system is configured to track an object in a sequence of images using a first tracking algorithm, wherein the first tracking algorithm is an adaptive tracking algorithm configured to modify an object model by updating data representing the object, wherein the first tracking data indicates a first location of a first region where the first tracking system has tracked the object in an image of the sequence of images, and wherein the first confidence value indicates a likelihood that the first tracking system is tracking the object;

receive second tracking data and a second confidence value at the tracker selection system from a second tracking system, wherein the second tracking system is configured to track the object in the sequence of images using a second tracking algorithm different from the first tracking algorithm, wherein the second tracking algorithm is a non-adaptive tracking algorithm configured to not modify the object model when updating data representing the object, wherein the second tracking data indicates a second location of a second region where the second tracking system has tracked the object in the image, and wherein the second confidence value indicates a likelihood that the second tracking system is tracking the object; and

provide output data to the first tracking system and to the second tracking system, wherein the output data includes data selected from the first tracking data and the second tracking data based on a result of a comparison of the first confidence value and the second confidence value.

20. The non-transitory computer readable medium of claim 19, wherein the instructions, when executed by the processor, further cause the processor to:

determine whether the first tracking data is valid; and

determine whether the second tracking data is valid,

wherein the output data includes the first tracking data when the first tracking data is valid and the second tracking data is not valid.