Static occlusion handling using directional pixel replication in regularized motion environments

• US 9,390,328 B2
• Filed: 04/25/2014
• Issued: 07/12/2016
• Est. Priority Date: 04/25/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Shreve, Matthew Adam, Li, Qun, Bernal, Edgar A., Loce, Robert P.
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US14/262,276
• Publication Number:
  US 20150310273A1
• Time in Patent Office: 809 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

This disclosure provides a static occlusion handling method and system for use with appearance-based video tracking algorithms where static occlusions are present. The method and system assumes that the objects to be tracked move in according to structured motion patterns within a scene, such as vehicles moving along a roadway. A primary concept is to replicate pixels associated with the tracked object from previous frames to current or future frames when the tracked object coincides with a static occlusion, where the predicted motion of the tracked object is a basis for replication of the pixels.

Claims

1. A computer-implemented method of tracking an object in a scene captured with an image capturing device configured to provide a first frame including a plurality of pixels representative of the scene, the scene including an occlusion which occludes at least a portion of the tracked object as the tracked object travels along an approaching direction (D) associated with the scene, the method comprising:

a) locating the occlusion and associated occlusion pixel positions within the first frame including the scene;

b) estimating the approaching direction (D) of the tracked object in the scene, relative to the located occlusion;

c) identifying a neighboring region along direction (D) located adjacent to the occlusion pixel positions;

d) replicating pixels associated with the neighboring region; and

e) replacing pixels in a second frame located at the associated occlusion pixel positions with the replicated pixels, wherein all or part of the tracked object is occluded by the occlusion in the second frame.

2. The computer-implemented method of tracking an object in a scene according to claim 1, wherein the occlusion is a static occlusion.

3. The computer-implemented method of tracking an object in a scene according to claim 1, wherein at least a first and second image capturing devices are used to track one or more objects.

4. The method of claim 3, wherein pixels from the second frame acquired with the second image capturing device are replicated onto the first frame acquired with the first image capturing device.

5. The method of claim 4, wherein a geometric correction is applied prior to the pixel replication.

6. The computer-implemented method of tracking an object in a scene according to claim 1, wherein the second frame is a prior frame.

7. The computer-implemented method of tracking an object in a scene according to claim 1, wherein the second frame is the same as the first frame.

8. The method of claim 4, wherein the pixels that are replaced in the first frame are replicated from a prior frame based on at least one of the speed of the tracked object.

9. The computer-implemented method of tracking an object in a scene according to claim 1, wherein estimating the approaching direction (D) includes the use of one or more of:

a priori knowledge of the occlusion; and

motion analysis in regions nearby an occlusion.

10. The computer-implemented method of tracking an object in a scene according to claim 1, wherein pixels are replicated in step d) performing one of:

shifting line segments oriented in the D direction;

replicating the tracked object pixels associated with an area of the occlusion shifted in the D direction;

replicating the tracked object pixels based on an intersection of the tracked object and the occlusion; and

replicating the tracked object pixels based on a trajectory associated with the tracked object.

11. The computer-implemented method of tracking an object according to claim 1, wherein step d) performs pixel replication using one or both of a current frame and a stored image of the tracked object.

12. An image processing system for tracking an object, the system comprising:

a processor; and

,

a non-transitory computer readable memory storing instructions that are executable by the processor to;

a) receive a digital image of a scene captured with an image capturing device, the digital image including a plurality of pixels representative of the scene, the scene including an occlusion which occludes at least a portion of the tracked object as the object travels along an approaching direction (D) associated with the scene;

b) locate the occlusion and associated occlusion pixel positions within a first frame including the scene;

c) estimate the approaching direction (D) of the tracked object in the scene, relative to the located occlusion;

d) identify a neighboring region along direction (D) located adjacent to the occlusion pixel positions;

e) replicate pixels associated with the neighboring region; and

f) replace pixels in a second frame located at the associated occlusion pixel positions with the replicated pixels, wherein all or part of the tracked object is occluded by the occlusion in the second frame.

13. The image processing system according to claim 12, wherein the occlusion is a static occlusion.

14. The image processing system according to claim 12, wherein at least a first and second image capturing devices are used to track one or more objects.

15. The image processing system according to claim 14, wherein pixels from the second frame acquired with the second image capturing device are replicated onto the first frame acquired with the first image capturing device.

16. The image processing system according to claim 14, wherein a geometric correction is applied prior to the pixel replication.

17. The image processing system according to claim 12, wherein the second frame is a prior frame.

18. The image processing system according to claim 12, wherein the second frame is the same as the first frame.

19. The image processing system according to claim 12, wherein the pixels that are replaced in the first frame are replicated from a prior frame based on the speed of the tracked object.

20. The image processing system according to claim 12, wherein estimating the approaching direction (D) includes the use of one or more of:

a priori knowledge of the occlusion; and

motion analysis in regions nearby an occlusion.

21. The image processing system according to claim 12, wherein pixels are replicated in step e) performing one of:

shifting line segments oriented in the D direction;

replicating the tracked object pixels associated with an area of the occlusion shifted in the D direction;

replicating the tracked object pixels based on an intersection of the tracked object and the occlusion; and

replicating the tracked object pixels based on a trajectory associated with the tracked object.

22. The image processing system according to claim 12, wherein step e) performs pixel replication using one or both of a current frame and a stored image of the tracked object.

23. A computer program product comprising:

a non-transitory computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a method of tracking an object in a scene captured with an image capturing device configured to provide a first frame including a plurality of pixels representative of the scene, the scene including an occlusion which occludes at least a portion of the tracked object as the tracked object travels along an approaching direction (D) associated with the scene, the method comprising;

a) locating the occlusion and associated occlusion pixel positions within the frame including the scene;

b) estimating the approaching direction (D) of the tracked object in the scene, relative to the located occlusion; and

c) identifying a neighboring region along direction (D) located adjacent to the occlusion pixel positions;

d) replicating pixels associated with the neighboring region; and

e) replacing pixels in a second frame located at the associated occlusion pixel positions with the replicated pixels, wherein all or part of the tracked object is occluded by the occlusion in the second frame.

24. The computer program product according to claim 23, wherein estimating the approaching direction (D) includes the use of one or more of:

a priori knowledge of the occlusion; and

motion analysis in regions nearby an occlusion.

25. The computer program product according to claim 23, wherein pixels are replicated in step d) performing one of:

shifting line segments oriented in the D direction;

replicating the tracked object pixels associated with an area of the occlusion shifted in the D direction;

replicating the tracked object pixels based on an intersection of the tracked object and the occlusion; and

replicating the tracked object pixels based on a trajectory associated with the tracked object.