Method and system for automated sequencing of vehicles in side-by-side drive-thru configurations via appearance-based classification

• US 10,176,384 B2
• Filed: 10/19/2016
• Issued: 01/08/2019
• Est. Priority Date: 04/24/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Bulan, Orhan, Bernal, Edgar A., Burry, Aaron M., Artan, Yusuf Oguzhan
• Primary Examiner(s): Williams, Jeffery A
• Assistant Examiner(s): Young, Patricia I
• Application Number:
  US15/297,572
• Publication Number:
  US 20170039432A1
• Time in Patent Office: 811 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06Q 50/12 Hotels or restaurants
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

This disclosure provides a method and system for automated sequencing of vehicles in side-by-side drive-thru configurations via appearance-based classification. According to an exemplary embodiment, an automated sequencing method includes computer-implemented method of automated sequencing of vehicles in a side-by-side drive-thru, the method comprising: a) an image capturing device capturing video of a merge-point area associated with multiple lanes of traffic merging; b) detecting in the video a vehicle as it traverses the merge-point area; c) classifying the detected vehicle associated with traversing the merge-point area as coming from one of the merging lanes; and d) aggregating vehicle classifications performed in step c) to generate a merge sequence of detected vehicles.

Claims

1. A computer-implemented method of automated sequencing of vehicles merging from multiple lanes into a single lane, the method comprising:

a) an image-capturing device capturing video of a merge-point area associated with multiple lanes of traffic merging;

b) detecting in the video a vehicle as it traverses the merge-point area;

c) classifying the detected vehicle associated with traversing the merge-point area as coming from one of the merging lanes; and

d) aggregating vehicle classifications performed in step c) to generate a merge sequence of detected vehicles,

wherein the multiple lanes of traffic merging correspond to a side-by-side drive-thru, and the side-by-side drive-thru includes an inner drive-thru lane and an outer drive-thru lane.

2. The computer-implemented method of automated sequencing of vehicles according to claim 1, wherein a detected vehicle is tracked until the detected vehicle is outside the merge-point area.

3. The computer-implemented method of automated sequencing of vehicles according to claim 1, wherein the vehicle detection is performed using one or more of temporal differencing of video frames, background estimation and subtraction, optical flow, and classifier-based vehicle detection.

4. The computer-implemented automated sequencing of vehicles according to claim 1, wherein step c) uses one or more classifiers trained with positive and negative video samples including vehicles approaching the merge point from a first lane, and positive and negative video samples including vehicles approaching the merge point area from a second lane.

5. The computer-implemented method of automated sequencing of vehicles according to claim 1, wherein step c) uses a first classifier trained with positive and negative video samples including vehicles approaching the merge point area from a first lane, and a second classifier trained with positive and negative video samples including vehicles approaching the merge point area from a second lane.

6. The computer-implemented method of automated sequencing of vehicles according to claim 1, wherein a detected vehicle is tracked until the detected vehicle is outside the merge point area and the detected vehicle is classified for a plurality of video frames while the detected vehicle is tracked.

7. The computer-implemented method of automated sequencing of vehicles according to claim 6, wherein a mathematical combination of classifications is used to generate the merge sequence of the detected vehicles.

8. An image processing system comprising:

a processor and a memory operatively associated with the processor, the processor and memory and execute instructions to perform a method of automated sequencing of vehicles merging from multiple lanes into a single lane, the method comprising;

a) an image capturing device capturing video of a merge-point area associated with multiple lanes of traffic merging;

b) detecting in the video a vehicle as it traverses the merge-point area;

c) classifying the detected vehicle associated with traversing the merge-point area as coming from one of the merging lanes; and

d) aggregating vehicle classifications performed in step c) to generate a merge sequence of detected vehicles,

wherein the multiple lanes of traffic merging correspond to a side-by-side drive-thru, and the side-by-side drive-thru includes an inner drive-thru lane and an outer drive-thru lane.

9. The image processing system according to claim 8, wherein a detected vehicle is tracked until the detected vehicle is outside the merge-point area.

10. The image processing system according to claim 8, wherein the vehicle detection is performed using one or more of temporal differencing of video frames, background estimation and subtraction, and optical flow.

11. The image processing system according to claim 8, wherein step c) uses one or more classifiers trained with positive and negative video samples including vehicles approaching the merge point from a first lane, and positive and negative video samples including vehicles approaching the merge point area from a second lane.

12. The image processing system according to claim 8, wherein step c) uses a first classifier trained with positive and negative video samples including vehicles approaching the merge point area from a first lane, and a second classifier trained with positive and negative video samples including vehicles approaching the merge point area from a second lane.

13. The image processing system according to claim 8, wherein a detected vehicle is tracked until the detected vehicle is outside the merge point area and the detected vehicle is classified for a plurality of video frames while the detected vehicle is tracked.

14. The image processing system according to claim 13, wherein a mathematical combination of classifications is used to generate the merge sequence of the detected vehicles.

15. A system for updating an event sequence, the system comprising an automatic sequencing device including a memory and a processor in communication with the processor configured to:

receive a number of individual order requests from the two order points, each order request being received when a vehicle is located in the one of the two order points;

acquire video data of a lane merge point area from at least one image source, the merge point area associated with the convergence of two vehicle queue lanes from two respective order points to a single queue lane;

process one of frames and at least one region of interest (ROI) in frames of the video data to detect a vehicle including at least a portion of the vehicle in the lane merge point area;

track a movement of each detected vehicle through the lane merge point area over a subsequent series of frames until the detected vehicle is outside the lane merge point area;

in response to the detected and tracked vehicle associated with the lane merge point area, classify one or more video frames as including a vehicle coming from an outer lane associated with order point A or a vehicle coming from an inner lane associated with order point B; and

update a sequence of the orders to match the determined sequence of vehicles in the single queue lane.

16. The system of claim 15, wherein the processor and memory is further configured to:

classify each detected vehicle as belonging to one of a new detected vehicle and a currently tracked vehicle;

in response to the detected vehicle being a new vehicle, assign a tracker to the new vehicle; and

,

in response to the detected vehicle being a currently tracked vehicle, discard the detected vehicle.

17. The system of claim 16, wherein the processor and memory is further configured to:

extract a feature descriptor from each detected vehicle;

compare the extracted feature descriptor to one of a threshold and corresponding descriptors of current vehicle trackers; and

based on the comparing, classify the detected vehicle as belonging to one of a new vehicle and a currently tracked vehicle.

18. The system of claim 15, wherein the order points and lane merge point area is associated with a fast-food drive-thru.

19. The system of claim 15, wherein the processor and memory is further configured to:

compute a statistic of tracked feature locations of the detected vehicle;

compare the statistic to a location of the lane merge point area; and

determine if a location of the tracked detected vehicle reaches the predefined lane merge point area based on the comparison,

wherein the statistic is selected from a group consisting of a single point identifying a position of the tracked detected vehicle, a centroid of the tracked feature locations, a median center location of the tracked feature locations, a first point, a predetermined percentage of feature points, a last point, and a degree of overlap.

20. The system of claim 15, wherein vehicle detection is performed using one or more of temporal differencing of video frames, background estimation and subtraction, and optical flow.