Systems and methods for expedited large file processing

• US 9,934,287 B1
• Filed: 07/25/2017
• Issued: 04/03/2018
• Est. Priority Date: 07/25/2017
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Bhatt, Japan
• Primary Examiner(s): Cao, Phuong Thao
• Application Number:
  US15/659,143
• Time in Patent Office: 252 Days
• Field of Search: 707737
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2458 Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

A system includes one or more memory devices storing instructions, and one or more processors configured to execute the instructions to perform steps of a method for processing a large file. The system may receive record data comprising a plurality of records having an identification value in a common field having a data format. The system may determine a plurality of focus values based on the data format and create a plurality of virtual processing units based on the plurality of focus values. Each of the plurality of virtual processing units may process a sub-group of the plurality of records that corresponds to the focus value associated with the respective virtual processing unit.

Claims

1. A system for processing a large file, comprising:

one or more processors; and

a memory in communication with the one or more processors and storing instructions that, when executed by the one or more processors, are configured to cause the system to;

receive record data comprising a plurality of records, each of the plurality of records comprising an identification value in a common field, the common field having a data format comprising a sequence of characters and each of the identification values corresponding to a license plate number;

determine, based on the data format, a plurality of focus values comprising at least a first focus value and a second focus value, each of the plurality of focus values corresponding to a sub-group of the plurality of records such that the first focus value corresponds to a first sub-group of the plurality of records and the second focus value corresponds to a second sub-group of the plurality of records, wherein the plurality of focus values comprise a set of 260 unique two-character sequences where each unique two-character sequence comprises a letter followed by a number with the plurality of focus values corresponding to a specified portion of the sequence of characters in the data format;

create a first virtual processing unit for processing the first sub-group of the plurality of records corresponding to the first focus value;

create a second virtual processing unit for processing the second sub-group of the plurality of records corresponding to the second focus value; and

process the first and second sub-groups of the plurality of records via the first and second virtual processing units, respectively.

2. The system of claim 1, wherein the data format comprises a sequence of seven characters, wherein a first, second, and third character of the sequence are associated with a letter data type and a fourth, fifth, sixth, and seventh character of the sequence are associated with a number data type.

3. The system of claim 2, wherein the specified portion of the data format is the third and fourth characters of the sequence of seven characters.