Method and system for partial occlusion handling in vehicle tracking using deformable parts model

• US 9,779,331 B2
• Filed: 02/26/2015
• Issued: 10/03/2017
• Est. Priority Date: 04/24/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Bulan, Orhan, Artan, Yusuf Oguzhan, Burry, Aaron Michael
• Primary Examiner(s): Shah, Utpal
• Assistant Examiner(s): LEMIEUX, IAN L
• Application Number:
  US14/632,237
• Publication Number:
  US 20150310624A1
• Time in Patent Office: 950 Days
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

Provided is a method and system of tracking partially occluded objects using an elastic deformation model. According to an exemplary method and system, partially occluded vehicles are detected and tracked in a scene including side-by-side drive-thru lanes. A method for updating an event sequence includes acquiring video data of a queue area from at least one image source; searching the frames for subjects located at least near a region of interest (ROI) of defined start points in the video data; tracking a movement of each detected subject through the queue area over a subsequent series of frames; using the tracking, determining if a location of the a tracked subject reaches a predefined merge point where multiple queues in the queue area converge into a single queue lane; in response to the tracked subject reaching the predefined merge point, computing an observed sequence of where the tracked subject places among other subjects approaching an end-event point; and, updating a sequence of end-events to match the observed sequence of subjects in the single queue lane.

Claims

1. A computer implemented method for updating an event sequence including tracking one or more partially occluded vehicles in a queue area captured with one or more image capturing devices, the method comprising:

a) acquiring video of the queue area from the one or more image capturing devices, the video including a plurality of video frames;

b) receiving a number of individual event requests from multiple subjects, each event request being received by a subject in a vehicle located in one of multiple queues in the queue area and at least one vehicle including an occluded portion and a non-occluded portion, the occluded portion occluded by another vehicle located in another queue at least near a predefined merge point associated with the queue area;

c) processing one or more of the video frames and at least one region of interest (ROI) associated with the video frames to detect vehicles located near at least one defined start point, the process using one or more elastic deformation models to detect the non-occluded portion of the vehicle including the occluded portion, each start point corresponding to one of the multiple queues in the queue area;

d) tracking a movement of each detected vehicle through the queue area over a subsequent series of frames using the elastic deformation model to track the non-occluded portion of the vehicle including the occluded portion;

e) determining if a location of a tracked vehicle corresponds to the predefined merge point associated with the queue area where the multiple queues in the queue area converge into a single queue lane;

f) in response to the tracked vehicle reaching the predefined merge point, computing an observed sequence of where the tracked vehicle is sequentially positioned relative to other vehicles past the merge point and approaching an end-event point in the single queue lane; and

g) updating a sequence of the events to match the observed sequence of vehicles in the single queue lane,

wherein the queue area includes a fast-food-drive-thru, the event-requests include food orders, and the end-events include one of goods and services.

2. The computer implemented method according to claim 1, wherein prior to step d), the method comprises:

checking if the detected non-occluded portion of the vehicle is already being tracked.

3. The computer implemented method according to claim 1, wherein prior to step b), the method comprises:

defining region of interest (ROI) associated with a field of view (FOV) associated with the image capturing device.

4. The computer implemented method according to claim 3, wherein prior to step b), the method comprises:

defining two or more ROIs for automated sequencing of vehicles in a side-by-side drive-thru configuration.

5. The computer implemented method according to claim 1, wherein prior to step c) candidate frames are identified from the acquired video using a background subtraction process including a Gaussian mixture model, eigenbackgrounds, principle component analysis or running average computation, the candidate frames including a detected vehicle within a predefined ROI.

6. The computer implemented method according to claim 1, wherein prior to step c) candidate frames are identified from the acquired video using a motion detection process with the ROI including a temporal difference process, a pixel-level optical flow process or a block-matching process, the candidate frames including a detected vehicle within a predefined ROI.

7. The computer implemented method according to claim 1, wherein the one or more deformable parts models are trained using positive video samples including a plurality of partially occluded vehicle models and a plurality of landmarks are assigned to the positive video samples which outline a non-occluded portion of the vehicle models.

8. An image processing system comprising:

a memory and a processor operatively associated with the memory configured to receive a digital image of a scene and execute instructions to perform a method for updating an event sequence including tracking one or more partially occluded vehicles in a queue area captured with one or more image capturing devices, the method comprising;

a) acquiring video of the queue area from the one or more image capturing devices, the video including a plurality of video frames;

b) receiving a number of individual event requests from multiple subjects, each event request being received by a subject in a vehicle located in one of multiple queues in the queue area and at least one vehicle including an occluded portion and a non-occluded portion, the occluded portion occluded by another vehicle located in another queue at least near a predefined merge point associated with the queue area;

c) processing one or more of the video frames and at least one region of interest (ROI) associated with the video frames to detect vehicles located near at least one defined start point, the process using one or more elastic deformation models to detect the non-occluded portion of the vehicle including the occluded portion, each start point corresponding to one of the multiple queues in the queue area;

d) tracking a movement of each detected vehicle through the queue area over a subsequent series of frames using the elastic deformation model to track the non-occluded portion of the vehicle including the occluded portion;

e) determining if a location of a tracked vehicle corresponds to the predefined merge point associated with the queue area where the multiple queues in the queue area converge into a single queue lane;

f) in response to the tracked vehicle reaching the predefined merge point, computing an observed sequence of where the tracked vehicle is sequentially positioned relative to other vehicles past the merge point and approaching an end-event point in the single queue lane; and

g) updating a sequence of the events to match the observed sequence of vehicles in the single queue lane,

wherein the queue area includes a fast-food-drive-thru, the event-requests include food orders, and the end-events include one of goods and services.

9. The image processing system according to claim 8, wherein prior to step d), the method comprises:

checking if the detected non-occluded portion of the vehicle is already being tracked.

10. The image processing system according to claim 8, wherein prior to step b), the method comprises:

defining region of interest (ROI) associated with a field of view (FOV) associated with the image capturing device.

11. The image processing system according to claim 10, wherein prior to step b), the method comprises:

defining two or more ROIs for automated sequencing of vehicles in a side-by-side drive-thru configuration.

12. The image processing system according to claim 8, wherein prior to step c) candidate frames are identified from the acquired video using a background subtraction process including a Gaussian mixture model, eigenbackgrounds, principle component analysis or running average computation, the candidate frames including a detected vehicle within a predefined ROI.

13. The image processing system according to claim 8, wherein prior to step c) candidate frames are identified from the acquired video using a motion detection process with the ROI including a temporal difference process, a pixel-level optical flow process or a block-matching process, the candidate frames including a detected vehicle within a predefined ROI.

14. The image processing system according to claim 8, wherein the one or more deformable parts models are trained using positive video samples including a plurality of partially occluded vehicle models and a plurality of landmarks are assigned to the positive video samples which outline a non-occluded portion of the vehicle models.

15. A system for updating an event sequence, the system comprising an automatic sequencing device including a memory and a processor in communication with the processor configured to:

acquire video data of a queue area from at least one image source, each image source associated with an image plane;

receive a number of individual event requests from multiple subjects, each event request being received when a subject in a vehicle is located in one of multiple queues in the queue area and at least one vehicle includes an occluded portion occluded by another vehicle located in another queue at least for a predefined merge point associated with the queue area;

process the video data using an elastic deformation model to detect the non-occluded portion of the vehicle including the occluded portion located near at least one of a plurality of start points, each start point corresponding to a location associated with one of the multiple queues in the video data;

track a movement of each detected vehicle through the queue area over a subsequent series of frames associated with the video data using the elastic deformation model to track the non-occluded portion of the vehicle including the occluded portion;

determine if a location of a tracked vehicle reaches the predefined merge point on the image plane where the multiple queues in the queue area converge into a single queue lane;

in response to the tracked vehicle reaching the predefined merge point, compute an observed sequence of where the tracked vehicle is sequentially located relative to other vehicles past the merge point and approaching an end-event point in the single queue lane; and

,

update a sequence of the events to match the observed sequence of vehicles in the single queue lane,

wherein the queue area includes a fast-food-drive-thru, the event-requests include food orders, and the end-events include one of goods and services.

16. The system of updating an event sequence according to claim 15, wherein the memory and processor is further configured to:

define an ROI in the queue area.

17. The system of updating an event sequence according to claim 15, wherein the processor is further configured to:

compute a statistic of tracked feature locations of the vehicle;

compare the statistic to a location of a merge point region; and

,

determine if a location of the tracked vehicle corresponds to the predefined merge point based on the comparison;

wherein the statistic is selected from a group consisting of;

a single point identifying a position of the tracked subject, a centroid of the tracked feature locations, a median center location of the tracked feature locations, a first point, a predetermined percentage of feature points, a last point, and a degree of overlap.

18. The system of updating an event sequence according to claim 15, wherein the memory and processor is further configured to:

process the acquired video data to determine candidate vehicles;

classify each candidate vehicle as belonging to one of a new vehicle and a currently tracked vehicle;

in response to the candidate vehicle being a new vehicle, assign a tracker to the new vehicle; and

in response to the candidate subject being a currently tracked vehicle, discard the candidate vehicle.

19. The system for updating an event sequence according to claim 18, wherein the candidate vehicles are determined from candidate video frames which are identified using one of a background subtraction process and a motion detection process.

20. The system of updating an event sequence according to claim 18, wherein the memory and processor is further configured to:

extract a feature descriptor from each detected candidate vehicle;

compare the extracted feature descriptor to one of a threshold and corresponding descriptors of current trackers; and

based on the comparing, classify each detected candidate vehicle as belonging to one of a new vehicle and a currently tracked vehicle.