SYSTEM AND METHOD FOR MACHINE LEARNING AND CLASSIFYING DATA

• US 20140344195A1
• Filed: 05/20/2014
• Published: 11/20/2014
• Est. Priority Date: 05/20/2013
• Status: Active Grant

• Current Assignee: Southern Methodist University
• Original Assignee: Southern Methodist University
• Inventors: Drew, Jake, Hahsler, Michael, Moore, Tyler
• Granted Patent: US 10,579,661 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

The present invention relates in general to the field of parallel data processing, and more particularly to machine learning and classification of extremely large volumes of unstructured gene sequence data using Collaborative Analytics Gene Sequence Classification Learning Systems and Methods.

Claims

1. A computerized method for classifying data comprising:

(a) receiving the data;

(b) dividing the received data into two or more chunks;

(c) mapping each chunk into a token and storing the token in a token collection;

(d) hashing each token using two or more local sensitivity hashing functions, wherein each local sensitivity hashing function contains two or more random hashing seed numbers, determining a minimum hash value for each local sensitivity hashing function, and storing the minimum hash value for each local sensitivity hashing function in a minimum hash set collection;

(e) classifying the data using the minimum hash values for the tokens; and

wherein the foregoing steps are performed by one or more processors.

2. The method of claim 1, wherein:

step (c) is performed on multiple chunks simultaneously using two or more processors operating in parallel;

step (d) is performed on multiple tokens simultaneously using two or more processors operating in parallel;

or

step (e) is performed on multiple chunks simultaneously using two or more processors operating in parallel.

3. The method of claim 1, wherein the two or more chunks overlap one another by a stagger offset.

4. The method of claim 1, wherein steps (a) through (e) are used to construct a learning machine.

5. The method of claim 1, wherein the step of receiving the data comprises the step of extracting the data from one or more data sources.

6. The method of claim 1, wherein the data further comprises one or more categories associated with the data.

7. The method of claim 1, wherein the data comprises an unstructured gene sequence data.

8. The method of claim 7, wherein the unstructured gene data further comprises one or more categories, a gene sequence full name, a fasta identification number, a kingdom, a phylum, a class, an order, a family genus, a species or a combination thereof.

9. The method of claim 1, further comprising the step of validating the data.

10. The method of claim 1, further comprising the steps of:

receiving an input data comprising two or more sets of data;

separating the two or more sets of data from one another; and

determining whether to classify the two or more sets of data sequentially, parallel or a combination thereof.

11. The method of claim 1, further comprising the step of assigning a unique identifier to the data.

12. The method of claim 1, further comprising the step of maintaining a collection of the data, a collection of one or more categories associated with the data, a collection of totals of the categories, and a collection of category default frequencies.

13. A system for classifying data comprising:

at least one input/output interface;

a data storage;

one or more processors communicably coupled to the at least on input/output interface and the data storage; and

the one or more processors perform the steps of (a) receiving the data from the at least one input/output interface, (b) dividing the received data into two or more chunks, (c) mapping each chunk into a token and storing the token in a token collection within the data storage, (d) hashing each token using two or more local sensitivity hashing functions, wherein each local sensitivity hashing function contains two or more random hashing seed numbers, determining a minimum hash value for each local sensitivity hashing function, and storing the minimum hash value for each local sensitivity hashing function in a minimum hash set collection within the data storage, and (e) classifying the data using the minimum hash values for the tokens.

14. The system of claim 13, wherein the one or more processors perform:

step (c) on multiple chunks simultaneously using two or more processors operating in parallel;

step (d) on multiple tokens simultaneously using two or more processors operating in parallel;

or

step (e) on multiple chunks simultaneously using two or more processors operating in parallel.

15. The system of claim 13, wherein the two or more chunks overlap one another by a stagger offset.

16. The system of claim 13, wherein the one or more processors use steps (a) through (e) to construct a learning machine.

17. The system of claim 13, wherein the one or more processors perform the step of receiving the data by extracting the data from one or more data sources.

18. The system of claim 13, wherein the data further comprises one or more categories associated with the data.

19. The system of claim 13, wherein the data comprises an unstructured gene sequence data.

20. The system of claim 19, wherein the unstructured gene data further comprises one or more categories, a gene sequence full name, a fasta identification number, a kingdom, a phylum, a class, an order, a family genus, a species or a combination thereof.

21. The system of claim 13, wherein the one or more processors further perform the step of validating the data.

22. The system of claim 13, wherein the one or more processors further perform the steps of:

receiving an input data comprising two or more sets of data;

separating the two or more sets of data from one another; and

determining whether to classify the two or more sets of data sequentially, parallel or a combination thereof.

23. The system of claim 13, wherein the one or more processors further perform the step of assigning a unique identifier to the data.

24. The system of claim 13, wherein the one or more processors further perform the step of maintaining a collection of the data, a collection of one or more categories associated with the data, a collection of totals of the categories, and a collection of category default frequencies.

25. A computer program embodied on a non-transitory computer readable storage medium that is executed using one or more processors for classifying data comprising:

(a) a code segment for receiving the data;

(b) a code segment for dividing the received data into two or more chunks;

(c) a code segment for mapping each chunk into a token and storing the token in a token collection;

(d) a code segment for hashing each token using two or more local sensitivity hashing functions, wherein each local sensitivity hashing function contains two or more random hashing seed numbers, determining a minimum hash value for each local sensitivity hashing function, and storing the minimum hash value for each local sensitivity hashing function in a minimum hash set collection; and

(e) a code segment for classifying the data using the minimum hash values for the tokens.

26. A system for large-scale and rapid parallel processing, learning, and classification of extremely large volumes of gene sequence data comprising:

a plurality of processes operational on a plurality of interconnected processors;

the plurality of processes including a master process for coordinating the processing of a set of at least one gene sequence input data, a set of at least one application-independent Map Reduction Aggregation Methods, a set of at least one application-independent Classification Metric Functions, a set of at least one application-independent category managers, a set of at least one application-independent nested categorical key value pair collections, a set of at least one application-independent reduction operations, a set of at least one application-independent Blocking Mechanisms, a set of at least one transitional outputs collection and worker processes;

the master process performing the learning and/or classification processing coordination in response to a request to perform the learning and/or classification processing job, allocating portions of the gene sequences input data containing gene sequence text to at least one of the Map Reduction Aggregation Methods and allocating portions of the gene sequence input data containing gene sequence text and category associations to at least one of the category managers;

each of the Map Reduction Aggregation Methods including at least one Chunking Operations module comprising a first plurality of worker processes for receiving and mapping portions of the gene sequences input data into individual, independent units of transitional Sequence Chunk work comprising a consistently mapped data key and optional values that are conducive to simultaneous parallel Mapping Operations processing, wherein at least two of the first plurality of the worker processes perform Chunking Operations simultaneously in parallel;

each of the Map Reduction Aggregation Methods including at least one Mapping Operations modules comprising a second plurality of worker processes for receiving and mapping transitional Sequence Chunk outputs into individual, independent units of transitional Sequence Token work comprising a consistently mapped data key and optional values that are conducive to simultaneous parallel Locality Sensitive Hashing Operations processing, wherein at least two of the second plurality of the worker processes perform Mapping Operations simultaneously in parallel;

each of the Map Reduction Aggregation Methods including at least one Locality Sensitive Hashing modules comprising a third plurality of worker processes for receiving and performing Locality Sensitive Hashing operations on transitional Sequence Token outputs producing individual, independent units of transitional MinHash Set Items work comprising a collection of minimum hash value keys produced from a plurality of unique hashing functions and optional values that are conducive to simultaneous Reduction Operations processing and/or Classification Metric Functions, wherein at least two of the third plurality of the worker processes perform Locality Sensitive Hashing Operations simultaneously in parallel;

each of the Map Reduction Aggregation Methods including at least one transitional outputs collection allowing for shared thread-safe accesses by at least one worker process acting as a transitional outputs producer and at least one worker process acting as a transitional outputs consumer, wherein a Blocking Mechanism is utilized for managing accesses to the transitional outputs collection, wherein at least two of the worker processes perform production and consumption operations simultaneously in parallel;

each of the Map Reduction Aggregation Methods including at least one Blocking Mechanism that in response to notification when transitional outputs production has started manages the potential differences in the production and consumption speeds between at least one worker process acting as a transitional outputs producer and at least one worker process acting as a transitional outputs consumer, wherein each Blocking Mechanism in response to a complete depletion of transitional outputs by consumption worker processes before the production worker processes have completed production allows consumption worker processes to “

block”

or wait until additional transitional outputs are produced, wherein each Blocking Mechanism in response to an over-production of transitional outputs by the production worker processes exceeding a pre-defined transitional outputs capacity threshold allows consumption worker processes to “

block”

or wait until additional transitional outputs are consumed and the transitional outputs capacity threshold is no longer exceeded, wherein at least two of the worker processes perform Blocking Mechanism production and consumption operations simultaneously in parallel;

each reduction operations including at least one application specific Reduction Operations modules comprising a fourth plurality of worker processes for receiving and aggregating transitional MinHash Set Items output by reducing the minimum hash value keys and optional values eliminating the matching keys and aggregating the optional values into at least one nested categorical key value pair collection, wherein at least two of the fourth plurality of the worker processes perform Reduction Operations simultaneously in parallel;

each Classification Metric Function including one or more Classification Metric Function Operations modules comprising a fifth plurality of worker processes for performing frequency, similarity, or distance calculations, wherein each calculation is performed using the items within at least one Map Reduction Aggregation transitional output collection and/or using Map Reduction Aggregation outputs and associated categories consolidated into the Nested Categorical Key Value Pair Collection, wherein at least two of the fifth plurality of worker processes simultaneously perform the Classification Metric Functions Operations in parallel;

each Category Manager including one or more category and gene sequence management functions comprising at least one set of unique categories and Category IDs, gene sequences and Sequence IDs, default frequencies including all Sequence ID and Category ID associations, and category totals including relevant totals for all categories;

the Map Reduction Aggregation Methods applying Chunking Operations, Mapping Operations, and Locality Sensitive Hashing Operations to the retrieved input data to produce transitional MinHash Set Item outputs corresponding to a reduced set of minimum hash values representing the unique characteristics of each individual gene sequence text provided within the gene sequence input data; and

the Classification Metric Functions applying Classification Metric Functions Operations to the transitional MinHash Set Item outputs to produce Classification Totals and/or Penetration Totals corresponding to the similarity, distance, or classification between each individual gene sequence text provided within the gene sequence input data for classification and at least one other MinHash Set Item output and/or using Map Reduction Aggregation outputs and associated categories consolidated into the Nested Categorical Key Value Pair Collection.

27. The system of claim 26, wherein:

the final outputs of one or more similar Map Reduction Aggregation Methods are placed into a consolidated collection of nested key value pairs where the key for each key-value pair includes a minimum hash value or gene token text key which maps to a second or nested set of “

categorical”

keys or key-value pairs as its value;

the master process performing the Reduction Operations coordination in response to a request to perform consolidation of Map Reduction Aggregation transitional outputs, wherein the Reduction Operations perform the consolidation;

the second or nested set of “

categorical”

keys or key-value pairs including categorical associations of the minimum hash value or gene token key across any number of categorical keys contained in the nested set, wherein a key-value pair'"'"'s optional values may include any numerical frequency information or other data relevant for performing classification metric function operations;

the keys within each consolidated collection of nested key value pairs being partitioned or otherwise divided by each unique hashing function when Locality Sensitive Hashing Operations are deployed; and

the keys within each consolidated collection of nested key value pairs also being further partitioned or otherwise divided by each Map Reduction Aggregation Method when at least two non-similar Map Reduction Aggregation Methods are deployed within the same system.

28. The system of claim 26, wherein:

each Category Manager in response to receiving a set of at least one gene sequence'"'"'s text and a set of at least one category associated with each gene sequence'"'"'s text, assigning any new categories a unique Category ID and assigning any new gene sequences a unique Sequence ID;

each Category Manager in response to receiving a set of at least one Sequence IDs associated with a set of at least one Category IDs creating a set of default frequencies comprising a Sequence ID key and at least one Category ID key or key value pair including an optional value that contains a default frequency value and/or other relevant values, wherein each unique set of default frequencies contains at least one entry for each unique Category ID associated with a unique Sequence ID;

each Category Manager in response to requests for categories or default frequency values associated with a Sequence ID providing the Sequence ID'"'"'s default frequency set to the requestor;

each Category Manager maintaining a list of Category Totals representing a list of frequency and/or other relevant totals for each unique Category ID within the consolidated collection of nested key value pairs or a known Min Hash Set item; and

each Category Manager in response to requests for category totals associated with a Category ID providing the appropriate Category ID total to the requestor.

29. The system of claim 26, wherein:

at least one Reduction Operations consolidate MinHash Set Item'"'"'s minimum hash values or Sequence Token'"'"'s gene token text keys into a Nested Categorical Key Value Pair Collection eliminating matching minimum hash values or gene token text keys and aggregating the MinHash Set Item or Sequence Token values consistent with the specified Reduction Operations;

the master process performing the Reduction Operations coordination in response to a request to perform a learning and/or classification processing job;

each Reduction Operations in response to identifying a new or unique keys making a copy of the default frequencies set associated with the MinHash Set Item or Sequence Token'"'"'s Sequence ID and using the default frequencies as the second or nested collection of “

categorical”

key-value pairs in the new Nested Categorical Key Value Pair Collection entry; and

each Reduction Operations in response to modifying the Nested Categorical Key Value Pair Collection maintaining and updating the Category Manager'"'"'s Category Totals for each category total impacted.

30. The system of claim 26, wherein:

the Classification Metric Functions maintain a collection of Classification Totals and/or a collection of Category Penetration Totals to perform similarity or distance calculations between at least one Map Reduction Aggregation transitional output collection and/or using Map Reduction Aggregation outputs and associated categories consolidated into a Nested Categorical Key Value Pair Collection;

the master process performing the Classification Metric Function coordination in response to a request to perform classification of the input data;

the Classification Metric Functions maintaining and updating the Classification Totals consistent with a specified Length Weighted Frequency Jaccard embodiment or another custom distance or similarity function aggregating totals for all required categories associated with each matching key identified; and

the Classification Metric Functions maintaining and updating the Category Penetration Totals in response to notification that Classification Totals have completed and calculating a specified Length Weighted Frequency Jaccard embodiment or another custom distance or similarity function'"'"'s final values for each category contained within the Classification Totals.

31. The system of claim 26, wherein the Locality Sensitive Hashing Operations comprises:

the plurality of processes further include a second master process for coordinating the processing of at least one set of gene sequence'"'"'s Sequence Tokens input data, a set of at least one MinHash Initialization modules, a set of at least one MinHash Producer modules, a set of at least one MinHash Set Item outputs, and worker processes;

the second master process performing the Locality Sensitive Hashing processing coordination in response to a request to perform the Locality Sensitive Hashing operations, allocating each of the Sequence Token inputs containing small variable length samples of gene sequence text to at least one of the MinHash producer'"'"'s worker processes;

each of the Locality Sensitive Hashing operations including at least one MinHash Initialization module using a predefined Universe Size value to generate non-negative random numbers up to the specified Universe Size used during the creation of a specified number of unique hashing functions each containing a plurality of random numbers used as hashing seeds, wherein each unique hashing function is stored in at least one MinHash delegates collection;

each of the Locality Sensitive Hashing operations including at least one MinHash Producer module comprising a first plurality of worker processes for receiving and hashing each Sequence Token'"'"'s text or a pre-defined hash value for each Sequence Token'"'"'s text one time for each unique hashing function contained within the MinHash Delegates collection, wherein at least two of the first plurality of the worker processes perform Locality Sensitive Hashing operations simultaneously in parallel;

each of the Locality Sensitive Hashing operations including at least one SkipDups collection for ensuring that duplicate text values contained within a Gene Token'"'"'s text are hashed only one time for each unique gene sequence by each of the unique hashing functions contained within the MinHash Delegates collection;

each of the SkipDups collections using SkipDup composite keys comprising a Sequence Token'"'"'s Sequence ID and text or a pre-defined hash value for a Sequence Token'"'"'s text to identify duplicate Sequence Tokens within the same gene sequence;

each of the Locality Sensitive Hashing operations including a collection of MinHash Set Items for retaining the minimum hash values produced by each of the unique hashing functions contained within the MinHash delegates collection for each unique gene sequence'"'"'s Gene Tokens that Locality Sensitive Hashing operations are performed against;

each MinHash Set Item containing one entry for each unique hashing function contained within the MinHash delegates collection or in alternative embodiments only a smaller predefined number of hashing operations can be performed to identify candidate matches before full MinHash Sets are produced; and

each MinHash Producer in response to receiving a Sequence Token input creating a SkipDup key, determining if MinHashing has been performed, hashing each Sequence Token'"'"'s text or a pre-defined hash value for a Sequence Token'"'"'s text one time for each hashing function contained within the MinHash delegates collection or a candidate set of hashing functions in alternative embodiments, and retaining the minimum hash values produced for each unique hashing function within each unique gene sequence in the collection of MinHash Set Item outputs.

32. A system for large-scale and rapid parallel Locality Sensitive Hashing of gene sequence Sequence Tokens input data comprising:

a plurality of processes operational on a plurality of interconnected processors;

the plurality of processes including a master process for coordinating the processing of at least one set of gene sequence'"'"'s Sequence Tokens input data, a set of at least one MinHash Initialization modules, a set of at least one MinHash Producer modules, a set of at least one MinHash Set Item outputs, and worker processes;

the master process performing the Locality Sensitive Hashing processing coordination in response to a request to perform the Locality Sensitive Hashing operations, allocating each of the Sequence Token inputs containing small variable length samples of gene sequence text to at least one of the MinHash producer'"'"'s worker processes;

each of the Locality Sensitive Hashing operations including at least one MinHash Initialization module using a predefined Universe Size value to generate non-negative random numbers up to the specified Universe Size used during the creation of a specified number of unique hashing functions each containing a plurality of random numbers used as hashing seeds, wherein each unique hashing function is stored in at least one MinHash delegates collection;

each of the Locality Sensitive Hashing operations including at least one MinHash Producer module comprising a first plurality of worker processes for receiving and hashing each Sequence Token'"'"'s text or a pre-defined hash value for each Sequence Token'"'"'s text one time for each unique hashing function contained within the MinHash Delegates collection, wherein at least two of the first plurality of the worker processes perform Locality Sensitive Hashing operations simultaneously in parallel;

each of the Locality Sensitive Hashing operations including at least one SkipDups collection for ensuring that duplicate text values contained within a Gene Token'"'"'s text are hashed only one time for each unique gene sequence by each of the unique hashing functions contained within the MinHash Delegates collection;

each of the SkipDups collections using SkipDup composite keys comprising a Sequence Token'"'"'s Sequence ID and text or a pre-defined hash value for a Sequence Token'"'"'s text to identify duplicate Sequence Tokens within the same gene sequence;

each of the Locality Sensitive Hashing operations including a collection of MinHash Set Items for retaining the minimum hash values produced by each of the unique hashing functions contained within the MinHash delegates collection for each unique gene sequence'"'"'s Gene Tokens that Locality Sensitive Hashing operations are performed against;

each MinHash Set Item containing one entry for each unique hashing function contained within the MinHash delegates collection or in alternative embodiments only a smaller predefined number of hashing operations can be performed to identify candidate matches before full MinHash Sets are produced; and

each MinHash Producer in response to receiving a Sequence Token input creating a SkipDup key, determining if MinHashing has been performed, hashing each Sequence Token'"'"'s text or a pre-defined hash value for a Sequence Token'"'"'s text one time for each hashing function contained within the MinHash delegates collection or a candidate set of hashing functions in alternative embodiments, and retaining the minimum hash values produced for each unique hashing function within each unique gene sequence in the collection of MinHash Set Item outputs.

33. A method for mapping gene sequences into Gene Sequence Chunks containing portions of the gene sequence'"'"'s text broken into individual, independent units of transitional work conducive to simultaneous parallel downstream processing comprising:

receiving gene sequences input data containing at least one gene sequence text and possibly one or more categories associated with the gene sequence text;

creating one unique Sequence ID per gene sequence means identifying each gene sequence'"'"'s text with a unique number operable for referring to the gene sequence in one or more associations throughout the system while maintaining only one copy of the gene sequence'"'"'s text and possibly many copies of the Sequence ID utilized in many associations;

creating one unique Category ID per unique category associated with any of the gene sequence'"'"'s text means identifying each category with a unique number operable for referring to the category in one or more associations throughout the system while maintaining only one copy of the category'"'"'s text and possibly many copies of the Category ID utilized in many associations; and

creating a set of default frequencies means a set of key value pairs comprising one entry for each unique Category ID associated with a unique Sequence ID, wherein a Sequence ID key and at least one value containing a Category ID or nested key value pair including an optional value that contains a default frequency value and/or other relevant values for perform Classification Metric Functions.

34. The method of claim 33, further comprising:

performing Simple Chunking, if Stagger Chunking has not been deployed;

each Simple Chunking operation in response to receiving a gene sequence'"'"'s text, setting a Start Position equal to the current Start Position+1, wherein the StartPosition'"'"'s default value is equal to 0;

setting an End Position equal to the Start Position+a pre-defined Maximum Token Length, or the last position in the gene sequence'"'"'s text, whichever is closer;

creating a Sequence Chunk equal to the text between the current Start and End Positions; and

continuing the specified Simple Chunking operations until the Start Position is less than the gene sequence'"'"'s text length-minimum token length.

35. The method of claim 33, further comprising:

performing Stagger Chunking First Offset Operations, if Stagger Chunking is deployed;

each Stagger Chunking operation in response to receiving a gene sequence'"'"'s text performs First Offset Operations, setting a Start Position equal to the current End Position+1, wherein the Start Position'"'"'s default starting value is equal to 0;

setting an End Position equal to the Start Position+the First Offset value, or the last position in the gene sequence'"'"'s text, whichever is closer;

creating a Sequence Chunk equal to the text between the current Start and End Positions; and

repeating First Offset Operations while the current Start Position is less than the gene sequence text length.

36. The method of claim 33, further comprising:

performing Stagger Chunking Stagger Offset Operations, if Stagger Chunking is deployed;

each Stagger Chunking operation in response to receiving a gene sequence'"'"'s text performs Stagger Offset Operations, setting a Start Position equal to the current End Position+1, wherein the Start Position'"'"'s default starting value is equal to the First Offset/2 truncated or rounded to a whole number;

setting an End Position equal to the Start Position+the First Offset value, or the last position in the gene sequence'"'"'s text, whichever is closer;

creating a Sequence Chunk equal to the text between the current Start and End Positions; and

repeating Stagger Offset Operations while the current Start Position is less than the gene sequence text length.

37. A method for mapping gene sequences into Gene Sequence Tokens containing small samples of a gene sequence'"'"'s text broken into individual, independent units of transitional work conducive to simultaneous parallel downstream processing comprising:

receiving gene sequence input data containing at least one gene sequence text or the starting and ending positions of a gene sequence'"'"'s text, and a gene'"'"'s Sequence ID;

setting the Start Position and Ending Position equal to the first position within the gene sequence'"'"'s text, or setting the Ending Position equal to the Minimum Token Length, if a Minimum Token Length is used;

creating a Sequence Token equal to the text between the current Start and End Positions;

setting the End Position equal to the End Position+1; and

repeating the specified operations until the Maximum Token Length or the end of gene sequence'"'"'s text is reached, whichever occurs first.

38. The method of claim 37, further comprising:

performing Stagger Mapping Operations, if Stagger Chunking is deployed;

setting the Start Position to the last Start Position+1 and the Ending Position equal to the last Start Position+1 or the Minimum Token Length, whichever is greater;

creating a Sequence Token equal to the text between the current Start and End Positions;

setting the End Position equal to the End Position+1; and

repeating the specified operations end of gene sequence'"'"'s text−

the minimum token length is reached, whichever occurs first.

39. A method for the locality sensitive hashing of gene sequences comprising:

receiving gene sequences input data containing at least one gene sequence text;

mapping gene sequences into transitional Gene Sequence Chunks by (a) receiving gene sequences input data containing at least one gene sequence text and possibly one or more categories associated with the gene sequence text, (b) creating one unique Sequence ID per gene sequence means identifying each gene sequence'"'"'s text with a unique number operable for referring to the gene sequence in one or more associations throughout the system while maintaining only one copy of the gene sequence'"'"'s text and possibly many copies of the Sequence ID utilized in many associations, (c) creating one unique Category ID per unique category associated with any of the gene sequence'"'"'s text means identifying each category with a unique number operable for referring to the category in one or more associations throughout the system while maintaining only one copy of the category'"'"'s text and possibly many copies of the Category ID utilized in many associations, and (d) creating a set of default frequencies means a set of key value pairs comprising one entry for each unique Category ID associated with a unique Sequence ID, wherein a Sequence ID key and at least one value containing a Category ID or nested key value pair includes an optional value that contains a default frequency value and/or other relevant values for performing Classification Metric Functions;

mapping Gene Sequence Chunks into transitional Gene Sequence Tokens by (a) receiving gene sequence input data containing at least one gene sequence text or the starting and ending positions of a gene sequence'"'"'s text, and a gene'"'"'s Sequence ID, (b) setting the Start Position and Ending Position equal to the first position within the gene sequence'"'"'s text, or setting the Ending Position equal to the Minimum Token Length, if a Minimum Token Length is used, (c) creating a Sequence Token equal to the text between the current Start and End Positions, (d) setting the End Position equal to the End Position+1, and (e) repeating the specified operations until the Maximum Token Length or the end of gene sequence'"'"'s text is reached, whichever occurs first;

receiving each transitional Gene Sequence Token;

creating a set of unique hashing functions used for the locality sensitive hashing operations;

creating a SkipDup key comprising the Gene Sequence Token'"'"'s Sequence ID and text or pre-determined text hash value;

maintaining a SkipDup key set ensuring that only unique Gene Sequence Token'"'"'s text or pre-determined text hash values are hashed one time for each unique gene Sequence ID and one time for each of the unique hashing functions used in the locality sensitive hashing operations;

maintaining a MinHash Set Item for each unique Sequence ID for which locality sensitive hashing operations are performed;

each time a unique Gene Sequence Token and SkipDup key are encountered, determining if a MinHash Set Item exists for the Gene Sequence Token'"'"'s Sequence ID and creating a new MinHash Set Item for Sequence IDs when needed;

each time a unique hashing function produces a minimum hash value, retaining the value within the MinHash Set Item for each unique hashing function; and

providing a MinHash Set Item as locality sensitive hashing operations output for each unique Gene Sequence ID for which locality sensitive hashing operations are performed.