Optimizing video stream processing

• US 9,569,672 B2
• Filed: 10/16/2015
• Issued: 02/14/2017
• Est. Priority Date: 02/01/2010
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Bobbitt, Russell P., Fan, Quanfu, Miyazawa, Sachiko, Pankanti, Sharathchandra U., Zhai, Yun
• Primary Examiner(s): Mejia, Anthony
• Application Number:
  US14/884,927
• Publication Number:
  US 20160034766A1
• Time in Patent Office: 487 Days
• Field of Search: 348/150, 382/100, 709/219, 709231-232, 706/47, 706/52
• US Class Current: 1/1
• CPC Class Codes: G06Q 20/202 Interconnection or interaction of plural electronic cash registers [ECR] or to host computer, e.g. network details, transfer of information from host to ECR or from ECR to ECR
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Transaction units of video data and transaction data captured from different checkout lanes are prioritized as a function of lane priority values of respective ones of the different checkout lanes from which the transaction units are acquired. Each of the checkout lanes has a different lane priority value. The individual transaction units are processed in the prioritized processing order to automatically detect irregular activities indicated by the transaction unit video and the transaction data of the processed individual transaction units.

Claims

1. A computer implemented method, comprising executing on a processor the steps of:

combining, via statistical sampling model processing, transaction data and video stream data into a plurality of individual transaction units that each comprise transaction data and video data corresponding to an item purchased by a customer at one of a plurality of different checkout lanes that are each operated by different ones of a plurality of cashiers; and

prioritizing, for the detection of irregular activities, each transaction unit of the plurality of individual transaction units as a function of differences in identifies of the cashiers and in transaction data that is captured from each of the individual transaction units, by;

assigning a highest priority value to the individual transaction units that are associated with a first one of the cashiers that is flagged by a manager;

assigning different ones of a plurality of priority values to the individual transaction units that are associated with at least one other of the cashiers, by assigning a first of the priority values to the individual transaction units that comprise a plurality of voided items, and a second of the priority values to the individual transaction units that comprise a voiding of all transaction items; and

assigning a lowest priority value to remaining others of the individual transaction units that have not been assigned one of the plurality of priority values.

2. The method of claim 1, further comprising:

integrating computer-readable program code into a computer system comprising the processor, a computer readable memory and a computer readable storage device;

wherein the computer readable program code is embodied on the computer readable storage device and comprises instructions for execution by the processor via the computer readable memory that cause the processor to perform the steps of combining the transaction data and video stream data into the plurality of individual transaction units, prioritizing each of the plurality of individual transaction units as the function of differences in the identifies of the cashiers and in the transaction data captured from each of the individual transaction units, by assigning the highest priority value to the individual transaction units associated with the first cashier flagged by the manager, assigning the different ones of a plurality of priority values to the individual transaction units that are associated with at least one other of the cashiers, by assigning the first priority value to the individual transaction units that comprise the plurality of voided items, the second priority value to the individual transaction units that comprise the voiding of all transaction items, and the lowest priority value to the remaining others of the individual transaction units that have not been assigned one of the plurality of priority values.

3. The method of claim 1, wherein the step of assigning different ones of the plurality of priority values to the individual transaction units that are associated with at least one other of the cashiers comprises assigning a third of the priority values to the individual transaction units that comprise a long time duration between item scans.

4. The method of claim 1, wherein the step of assigning different ones of the plurality of priority values to the individual transaction units that are associated with at least one other of the cashiers comprises:

assigning a fourth of the priority values to the individual transaction units that comprise a cash transaction; and

assigning a fifth of the priority values to the individual transaction units that are associated another of the cashiers that is associated with a plurality of managerial overwrites.

5. The method of claim 1, wherein the step of assigning different ones of the plurality of priority values to the individual transaction units that are associated with at least one other of the cashiers comprises:

assigning a sixth of the priority values to the individual transaction units that comprise a cash transaction; and

assigning a seventh of the priority values to the individual transaction units that comprise a plurality of high value items.

6. The method of claim 1, further comprising:

processing the individual transaction units in a processing order that is prioritized as a function of differences in their respective assigned priority values, to automatically detect irregular activities indicated by transaction unit video and the transaction data of the processed individual transaction units, by;

processing a high priority fraction of a total of the prioritized individual transaction units in the prioritized processing order; and

archiving a low priority remainder fraction of the total of the individual transaction units that each have a lower priority than the high priority fraction individual transaction units within the prioritized processing order.

7. The method of claim 6, wherein the step of archiving the low priority remainder fraction of the total of the individual transaction units comprises moving lowest ranked transaction units from the transaction priority queue to an archival queue in an order of relative values of their respective assigned priority values.

8. The method of claim 7, further comprising:

moving unprocessed transaction units from the transaction priority queue to the archival queue at regular intervals.

9. The method of claim 7, further comprising:

in response to determining that a computer processor has unused capacity, processing the low priority remainder fraction individual transaction units archived in the archival queue via the computer processor that has the unused capacity in order of their relative queued priority to automatically detect activities indicated by the transaction unit video and the transaction data of the processed transaction units.

10. The method of claim 9, further comprising:

obtaining streams of video and transaction data captured from each of the plurality of different checkout lanes; and

combining the streams of video and transaction data for each respective one of the different checkout lanes into the individual transaction units so that each of the individual transaction units comprises transaction video corresponding to a set of items purchased by a single customer at one of the different checkout lanes in a single span of time.

11. A system for the detection of irregular activities, comprising:

a processor;

a computer-readable memory in communication with the processor; and

a computer-readable storage device in communication with the processor;

wherein the processor executes program instructions stored on the computer-readable storage device via the computer-readable memory and thereby;

combines, via statistical sampling model processing, transaction data and video stream data into a plurality of individual transaction units that each comprise transaction data and video data corresponding to an item purchased by a customer at one of a plurality of different checkout lanes that are each operated by different ones of a plurality of cashiers; and

prioritizes each transaction unit of the plurality of individual transaction units as a function of differences in identifies of the cashiers and in transaction data that is captured from each of the individual transaction units, by;

assigning a highest priority value to the individual transaction units that are associated with a first one of the cashiers that is flagged by a manager;

assigning different ones of a plurality of priority values to the individual transaction units that are associated with at least one other of the cashiers, by assigning a first of the priority values to the individual transaction units that comprise a plurality of voided items, and a second of the priority values to the individual transaction units that comprise a voiding of all transaction items; and

assigning a lowest priority value to remaining others of the individual transaction units that have not been assigned one of the plurality of priority values.

12. The system of claim 11, wherein the processor executes the program instructions stored on the computer-readable storage device via the computer-readable memory, and thereby further:

assigns a third of the priority values to the individual transaction units that comprise a long time duration between item scans.

13. The system of claim 12, wherein the processor executes the program instructions stored on the computer-readable storage device via the computer-readable memory, and thereby further:

assigns a fourth of the priority values to the individual transaction units that comprise a cash transaction;

assigns a fifth of the priority values to the individual transaction units that are associated another of the cashiers that is associated with a plurality of managerial overwrites;

assigns a sixth of the priority values to the individual transaction units that comprise a cash transaction; and

assigns a seventh of the priority values to the individual transaction units that comprise a plurality of high value items.

14. The system of claim 11, wherein the processor executes the program instructions stored on the computer-readable storage device via the computer-readable memory, and thereby further:

processes the individual transaction units in a processing order that is prioritized as a function of differences in their respective assigned priority values, to automatically detect irregular activities indicated by transaction unit video and the transaction data of the processed individual transaction units, by;

processing a high priority fraction of a total of the prioritized individual transaction units in the prioritized processing order; and

archiving a low priority remainder fraction of the total of the individual transaction units that each have a lower priority than the high priority fraction individual transaction units within the prioritized processing order.

15. The system of claim 14, wherein the processor executes the program instructions stored on the computer-readable storage device via the computer-readable memory, and thereby archives the low priority remainder fraction of the total of the individual transaction units by moving lowest ranked transaction units from the transaction priority queue to an archival queue in an order of relative values of their respective assigned priority values.

16. An article of manufacture, comprising:

a computer readable storage device having computer readable program code embodied therewith, wherein the computer-readable storage device is not a transitory medium per se, the computer readable program code comprising instructions for execution by a processor that cause the processor to;

combine, via statistical sampling model processing, transaction data and video stream data into a plurality of individual transaction units that each comprise transaction data and video data corresponding to an item purchased by a customer at one of a plurality of different checkout lanes that are each operated by different ones of a plurality of cashiers; and

prioritize each transaction unit of the plurality of individual transaction units as a function of differences in identifies of the cashiers and in transaction data that is captured from each of the individual transaction units, by;

assigning a highest priority value to the individual transaction units that are associated with a first one of the cashiers that is flagged by a manager;

assigning different ones of a plurality of priority values to the individual transaction units that are associated with at least one other of the cashiers, by assigning a first of the priority values to the individual transaction units that comprise a plurality of voided items, and a second of the priority values to the individual transaction units that comprise a voiding of all transaction items; and

assigning a lowest priority value to remaining others of the individual transaction units that have not been assigned one of the plurality of priority values.

17. The article of manufacture of claim 16, wherein the computer-readable program code instructions for execution by the processor further cause the processor to:

assign a third of the priority values to the individual transaction units that comprise a long time duration between item scans.

18. The article of manufacture of claim 17, wherein the computer-readable program code instructions for execution by the processor, further cause the processor to:

assign a fourth of the priority values to the individual transaction units that comprise a cash transaction;

assign a fifth of the priority values to the individual transaction units that are associated another of the cashiers that is associated with a plurality of managerial overwrites;

assign a sixth of the priority values to the individual transaction units that comprise a cash transaction; and

assign a seventh of the priority values to the individual transaction units that comprise a plurality of high value items.

19. The article of manufacture of claim 18, wherein the computer-readable program code instructions for execution by the processor, further cause the processor to:

process the individual transaction units in a processing order that is prioritized as a function of differences in their respective assigned priority values, to automatically detect irregular activities indicated by transaction unit video and the transaction data of the processed individual transaction units, by;

processing a high priority fraction of a total of the prioritized individual transaction units in the prioritized processing order; and

archiving a low priority remainder fraction of the total of the individual transaction units that each have a lower priority than the high priority fraction individual transaction units within the prioritized processing order.

20. The article of manufacture of claim 19, wherein the computer-readable program code instructions for execution by the processor, further cause the processor to:

archive the low priority remainder fraction of the total of the individual transaction units by moving lowest ranked transaction units from the transaction priority queue to an archival queue in an order of relative values of their respective assigned priority values.