System and method to improve object tracking using tracking fingerprints

• US 9,940,726 B2
• Filed: 12/19/2014
• Issued: 04/10/2018
• Est. Priority Date: 12/19/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Kim, Kyungnam, Jeong, Changsoo, Mundhenk, Terrell N., Owechko, Yuri
• Primary Examiner(s): FUJITA, KATRINA R
• Application Number:
  US14/577,983
• Publication Number:
  US 20160180546A1
• Time in Patent Office: 1,208 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

A method includes receiving image data at a first tracking system. The image data may represent a region in an image of a sequence of images. The method includes generating a first tracking fingerprint based on the image data. The method includes comparing the first tracking fingerprint and a second tracking fingerprint. The method further includes providing an output from the first tracking system to a second tracking system based on a result of the comparison of the first tracking fingerprint and the second tracking fingerprint.

Claims

1. A method comprising:

receiving image data at a first tracking system, wherein the image data represents a region in an image of a sequence of images;

generating a first tracking fingerprint based on the image data;

comparing the first tracking fingerprint and a second tracking fingerprint; and

providing a prevent instruction from the first tracking system to a second tracking system based on a result of the comparison of the first tracking fingerprint and the second tracking fingerprint, wherein the prevent instruction instructs the second tracking system to refrain from modifying an object model based on the image data.

2. The method of claim 1, wherein the first tracking fingerprint is generated by applying a mathematical model to feature data extracted from the image data.

3. The method of claim 1, wherein comparing the first tracking fingerprint and the second tracking fingerprint comprises determining a difference between the first tracking fingerprint and the second tracking fingerprint based on a distance measurement between the first tracking fingerprint and the second tracking fingerprint.

4. The method of claim 1, further comprising:

receiving second image data from a motion detector, wherein the second image data includes motion-triggered image data;

generating a first particular tracking fingerprint based on the second image data;

determining whether a difference between the first particular tracking fingerprint and a second particular tracking fingerprint is less than a first threshold value; and

receiving a confidence value from the second tracking system, wherein the confidence value indicates a likelihood that the second tracking system is correctly tracking an object.

5. The method of claim 4, further comprising:

determining whether the confidence value is less than a confidence threshold responsive to the difference being less than a first threshold value; and

providing a location update instruction and the motion-triggered image data to the second tracking system responsive to the confidence value being greater than or equal to the confidence threshold.

6. The method of claim 5, further comprising:

determining whether the difference is less than a second threshold value responsive to the confidence value being greater than the confidence threshold; and

providing an object model re-initialization instruction and the motion-triggered image data from the first tracking system to the second tracking system responsive to the difference being less than the second threshold value.

7. The method of claim 6, wherein the object model re-initialization instruction instructs the second tracking system to generate a new object model based on the motion-triggered image data.

8. The method of claim 4, further comprising:

determining whether the difference is less than a third threshold value; and

storing the first particular tracking fingerprint in a memory responsive to the difference being less than the third threshold value.

9. The method of claim 1, wherein comparing the first tracking fingerprint and the second tracking fingerprint comprises determining whether a difference between the first tracking fingerprint and the second tracking fingerprint is less than a fourth threshold value.

10. The method of claim 9, wherein the a prevent instruction is provided to the second tracking system responsive to the difference being greater than or equal to the fourth threshold value.

11. The method of claim 1, further comprising subsequent to providing the prevent instruction from the first tracking system to the second tracking system:

receiving second image data from a motion detector, wherein the second image data includes motion-triggered image data; and

providing an object model modification instruction from the first tracking system to the second tracking system, wherein the object model modification instruction instructs the second tracking system to modify the object model based on the motion-triggered image data.

12. The method of claim 1, further comprising providing a location update instruction from the first tracking system to the second tracking system, wherein the location update instruction instructs the second tracking system to set a tracking location based on location information received from the first tracking system.

13. A tracking system comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving image data representing a region in an image of a sequence of images;

generating a first tracking fingerprint based on the image data;

comparing the first tracking fingerprint and a second tracking fingerprint; and

providing a prevent instruction to a second tracking system based on a result of the comparison of the first tracking fingerprint and the second tracking fingerprint, wherein the prevent instruction instructs the second tracking system to refrain from modifying an object model based on the image data.

14. The tracking system of claim 13, wherein the sequence of images is received from an image capture device, and wherein the one or more processors, the memory, and the image capture device are integrated in an autonomous vehicle.

15. The tracking system of claim 14, wherein the second tracking system is configured to provide location data to a server, to another system of the autonomous vehicle, or a combination thereof.

16. A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

receiving image data at a first tracking system, wherein the image data represents a region in an image of a sequence of images;

generating a first tracking fingerprint based on the image data;

comparing the first tracking fingerprint and a second tracking fingerprint; and

providing a prevent instruction from the first tracking system to a second tracking system based on a result of the comparison of the first tracking fingerprint and the second tracking fingerprint, wherein the prevent instruction instructs the second tracking system to refrain from modifying an object model based on the image data.

17. The non-transitory computer readable medium of claim 16, wherein the operations further comprise providing an object model re-initialization instruction from the first tracking system to the second tracking system, wherein the object model re-initialization instruction instructs the second tracking system to generate a new object model.

18. The non-transitory computer readable medium of claim 17, wherein the object model re-initialization instruction enables the second tracking system to re-acquire an object in a subsequent image of the sequence of images after the object is at least partially occluded in one or more images of the sequence of images.