STATIC OCCLUSION HANDLING USING DIRECTIONAL PIXEL REPLICATION IN REGULARIZED MOTION ENVIRONMENTS

• US 20150310273A1
• Filed: 04/25/2014
• Published: 10/29/2015
• Est. Priority Date: 04/25/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Shreve, Matthew Adam, Li, Qun, Bernal, Edgar A., Loce, Robert P.
• Granted Patent: US 9,390,328 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

This disclosure provides a static occlusion handling method and system for use with appearance-based video tracking algorithms where static occlusions are present. The method and system assumes that the objects to be tracked move in according to structured motion patterns within a scene, such as vehicles moving along a roadway. A primary concept is to replicate pixels associated with the tracked object from previous frames to current or future frames when the tracked object coincides with a static occlusion, where the predicted motion of the tracked object is a basis for replication of the pixels.

Claims

1. A computer-implemented method of tracking an object in a scene captured with an image capturing device configured to provide a first frame including a plurality of pixels representative of the scene, the scene including an occlusion which occludes at least a portion of the tracked object as the tracked object travels along an approaching direction (D) associated with the scene, the method comprising:

a) locating the occlusion and associated occlusion pixel positions within the frame including the scene;

b) estimating the approaching direction (D) of the tracked object in the scene, relative to the located occlusion; and

c) tracking the object in the scene, where pixels from a neighboring region along direction (D) in a second frame are replicated to replace the occlusion pixels in the first frame while all or part of the tracked object is occluded by the occlusion.

2. The computer-implemented method of tracking an object in a scene according to claim 1, wherein the occlusion is a static occlusion.

3. The computer-implemented method of tracking an object in a scene according to claim 1, wherein at least a first and second image capturing devices are used to track one or more objects.

4. The method of claim 3, wherein pixels from the second frame acquired with the second image capturing device are replicated onto the first frame acquired with the first image capturing device.

5. The method of claim 4, wherein a geometric correction is applied prior to the pixel replication.

6. The computer-implemented method of tracking an object in a scene according to claim 1, wherein the second frame is a prior frame.

7. The computer-implemented method of tracking an object in a scene according to claim 1, wherein the second frame is the same as the first frame.

9. The computer-implemented method of tracking an object in a scene according to claim 1, wherein estimating the approaching direction (D) includes the use of one or more of:

a priori knowledge of the occlusion; and

motion analysis in regions nearby an occlusion.

10. The computer-implemented method of tracking an object in a scene according to claim 1, wherein pixels are replicated in step C) performing one of:

shifting line segments oriented in the D direction;

replicating the tracked object pixels associated with an area of the occlude shifted in the D direction;

replicating the tracked object pixels based on an intersection of the tracked object and the occlude; and

replicating the tracked object pixels based on a trajectory associated with the tracked object.

11. The computer-implemented method of tracking an object according to claim 1, wherein step c) performs pixel replication using one or both of a current frame and a stored image of the tracked object.

8. The method of claim 47, wherein the pixels that are replaced in the first frame are replicated from a prior frame based on at least one of the speed of the tracked object.

12. An image processing system comprising:

a controller configured to receive a digital image of a scene, the controller configured to execute instructions to perform a method of tracking an object in the scene captured with an image capturing device to provide a first frame including a plurality of pixels representative of the scene, the scene including an occlusion which occludes at least a portion of the tracked object as the object travels along an approaching direction (D) associated with the scene, the method comprising;

a) locating the occlusion and associated occlusion pixel positions within the frame including the scene;

b) estimating the approaching direction (D) of the tracked object in the scene, relative to the located occlusion; and

c) tracking the object in the scene, where pixels from a neighboring region along direction (D) in a second frame are replicated to replace the occlusion pixels in the first frame while all or part of the tracked object is occluded by the occlusion.

13. The image processing system according to claim 12, wherein the occlusion is a static occlusion.

14. The image processing system according to claim 12, wherein at least a first and second image capturing devices are used to track one or more objects.

15. The image processing system according to claim 14, wherein pixels from the second frame acquired with the second image capturing device are replicated onto the first frame acquired with the first image capturing device.

16. The image processing system according to claim 14, wherein a geometric correction is applied prior to the pixel replication.

17. The image processing system according to claim 12, wherein the second frame is a prior frame

18. The image processing system according to claim 12, wherein the second frame is the same as the first frame.

19. The image processing system according to claim 12, wherein the pixels that are replaced in the first frame are replicated from a prior frame based on the speed of the tracked object.

20. The image processing system according to claim 12, wherein estimating the approaching direction (D) includes the use of one or more of:

a priori knowledge of the occlusion; and

motion analysis in regions nearby an occlusion.

21. The image processing system according to claim 12, wherein pixels are replicated in step C) performing one of:

shifting line segments oriented in the D direction;

replicating the tracked object pixels associated with an area of the occlude shifted in the D direction;

replicating the tracked object pixels based on an intersection of the tracked object and the occlude; and

replicating the tracked object pixels based on a trajectory associated with the tracked object.

22. The image processing system according to claim 12, wherein step c) performs pixel replication using one or both of a current frame and a stored image of the tracked object.

23. A computer program product comprising:

a non-transitory computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a method of tracking an object in a scene captured with an image capturing device configured to provide a first frame including a plurality of pixels representative of the scene, the scene including an occlusion which occludes at least a portion of the tracked object as the tracked object travels along an approaching direction (D) associated with the scene, the method comprising;

a) locating the occlusion and associated occlusion pixel positions within the frame including the scene;

b) estimating the approaching direction (D) of the tracked object in the scene, relative to the located occlusion; and

c) tracking the object in the scene, where pixels from a neighboring region along direction (D) in a second frame are replicated to replace the occlusion pixels in the first frame while all or part of the tracked object is occluded by the occlusion.

24. The computer program product according to claim 23, wherein estimating the approaching direction (D) includes the use of one or more of:

a priori knowledge of the occlusion; and

motion analysis in regions nearby an occlusion.

25. The computer program product according to claim 23, wherein pixels are replicated in step C) performing one of:

shifting line segments oriented in the D direction;

replicating the tracked object pixels associated with an area of the occlude shifted in the D direction;

replicating the tracked object pixels based on an intersection of the tracked object and the occlude; and

replicating the tracked object pixels based on a trajectory associated with the tracked object.