HEURISTIC-BASED APPROACH FOR AUTOMATIC PAYMENT GESTURE CLASSIFICATION AND DETECTION

• US 20140064566A1
• Filed: 08/12/2013
• Published: 03/06/2014
• Est. Priority Date: 08/29/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Shreve, Matthew Adam, Mongeon, Michael C., Loce, Robert P., Bernal, Edgar A.
• Granted Patent: US 9,165,194 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06Q 20/30 characterised by the use of specific devices or networks
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system and method for automatic classification and detection of a payment gesture are disclosed. The method includes obtaining a video stream from a camera placed above at least one region of interest, the region of interest classifying the payment gesture. A background image is generated from the obtained video stream. Motion is estimated in at least two consecutive frames from the video stream. A representation is created from the background image and the estimated motion occurring within the at least one region of interest. The payment gesture is detected based on the representation.

Claims

1. A method for automatic classification and detection of a payment gesture, comprising:

obtaining a video stream from a camera capturing video from at least one region of interest, the region of interest associated with the type of payment gesture;

detecting motion within the at least one region of interest;

creating a representation of the motion occurring within the at least one region of interest; and

detecting the payment gesture based on the representation.

2. The method of claim 1, wherein the detecting motion within the at least one region of interest is achieved by one of temporal frame differencing, background estimation and subtraction, optical flow, and frame differencing.

3. The method of claim 1, wherein the at least one region of interest is associated with one of an object hand-off, cash payment object left behind, debit card payment, and credit card payment gesture.

4. The method of claim 2, wherein the background estimation includes regenerating the background image when the motion contained in the at least two consecutive frames from the video stream is less than a threshold value, T.

5. The method of claim 2, wherein the optical flow technique is implemented through at least one of the Horn-Schunck, Lucas-Kanade, and Black flow method.

6. The method of claim 1, wherein the representation is one of a binary image, a histogram of motion vectors, and a set of spatio-temporal coordinates representing the trajectory of an object in motion.

7. The method of claim 1, wherein the detecting the payment gesture based on the representation further includes determining whether the magnitude of the vertical motion is greater than a downward threshold, T_cc1.

8. The method of claim 7, wherein the detecting the payment gesture based on the representation further includes determining whether the magnitude of the vertical motion is greater than an upward threshold, T_cc2.

9. The method of claim 1, wherein the at least one region of interest is associated with a credit card payment gesture and encompasses a swipe portion of a credit card machine.

10. The method of claim 9, further including setting a flag equal to 1 after detection of the credit card payment gesture, the flag allowing for detection of the at least one payment gesture within a different region of interest.

11. The method of claim 1, wherein the at least one region of interest is associated with a cash hand-off payment gesture and encompasses a payment counter.

12. The method of claim 11, wherein the representation includes a binary image and the detecting the payment gesture further includes determining whether a blob from the binary image has a height greater than a threshold, H.

13. The method of claim 11, wherein the threshold, H, is defined by the height of the at least one region of interest.

14. The method of claim 6, wherein the representation includes a two-bin histogram indicating vertical motion, and the detecting the payment gesture based on the representation further includes determining whether the length of the vertical motion vectors is larger than a maximum threshold, T_hand.

15. The method of claim 1, wherein the at least one region of interest classifies a debit card payment gesture and encompasses a number pad portion of a credit card machine.

16. The method of claim 15, wherein the representation includes a binary image and the detecting the payment gesture further includes determining whether a button pressing event has occurred.

17. The method of claim 15, wherein the button pressing event has occurred if more than a threshold, T_cc3, number of pixels in the binary image are equal to 1.

18. The method of claim 16, wherein the detecting the payment gesture further includes a counter, num_button, the counter incremented for each button pressing event which has occurred.

19. The method of claim 18, wherein the detecting further includes determining whether the counter, num_button, is greater than a threshold value, N_B.

20. A computer program product comprising tangible media encoding instructions, which when executed by a computer, perform a method comprising:

generating a background image from a video stream, the video stream capturing data from at least one region of interest classifying a payment gesture;

estimating the motion contained in N consecutive frames taken from the video stream;

creating a representation from the motion occurring within the at least one region of interest; and

detecting the payment gesture based on the representation.

21. The method of claim 20, wherein the representation is a binary image.

22. The method of claim 20, wherein the representation is at least one of a grayscale and color image.

23. The method of claim 20, wherein the representation is a two-bin histogram indicating vertical motion.

24. The method of claim 20, wherein the estimating the motion contained in N consecutive frames taken from the video stream is performed by optical flow.

25. The method of claim 20, wherein the estimating the motion contained in N consecutive frames taken from the video stream is performed by object tracking and trajectory analysis.

26. A system for automatically classifying and detecting a payment gesture, comprising:

a video capture unit which streams video from above a region of interest, the region of interest classifying a payment gesture;

a video processing module for generating a background image and estimating the motion contained in N consecutive frames from the video steam, wherein the processing module produces a representation based on the dynamic background image and the motion within the region of interest;

a detection module for analyzing the representation and determining whether the payment gesture has occurred; and

a computer processor which implements the video capture unit, the video processing module, and the detection module.

27. The system according to claim 26, wherein the video processing module generates a dynamic background image that is updated with a current frame of the video stream when the motion contained in the region of interest is less than a threshold value for N consecutive frames.

28. The system according to claim 26, wherein the detection module determines whether vertical motion indicated by the representation is above a threshold value.

29. The system according to claim 26, wherein the detection module performs connected component analysis to label a plurality of blobs contained in the representation.