×

Identifying corrupted text segments

  • US 10,318,650 B2
  • Filed: 06/05/2018
  • Issued: 06/11/2019
  • Est. Priority Date: 03/03/2016
  • Status: Active Grant
First Claim
Patent Images

1. A computer system comprising:

  • one or more computer processors;

    one or more computer readable storage device;

    program instructions stored on the one or more computer readable storage devices for execution by at least one of the one or more computer processors, the stored program instructions comprising;

    program instructions to select a set of web pages containing text associated with a single language grouping;

    program instructions to determine a set of text segments within the set of web pages;

    program instructions to determine a language affinity indicator corresponding to each text segment in the set of text segments, the language affinity indicator being a comparison value of a text segment with a set of predefined rules corresponding to the single language grouping;

    program instructions to, responsive to each language affinity indicator indicating an affinity to the single language grouping, identify a set of text artefacts within the text segments;

    program instructions to generate an indexing repository based on the set of text artefacts;

    program instructions to create an occurrence table from the indexing repository;

    program instructions to determine a compliance threshold value for the occurrence table;

    program instructions to identify an individual occurrence value for each unique text artefact in the set of text artefacts, the individual occurrence value being the probability that a text artefact occurs within the occurrence table based on the single language grouping; and

    program instructions to determine a compliance value for the set of text segments by, for each text segment in the set of text segments;

    program instructions to compute a compliance sum value for a first text segment in the set of text segments;

    program instructions to adjust the compliance sum value according to the individual occurrence values of a subset of text artefacts occurring in the first text segment;

    program instructions to determine a segment length value associated with the first text segment; and

    program instructions to adjust the compliance sum value according to the segment length value;

    program instructions to, responsive to computing a set of compliance sum values for each text segment in the set of text segments, compute the compliance value based on an average value of the set of compliance sum values;

    program instructions to compute a compliance indicator for the set of text segments by comparing the compliance value and the compliance threshold; and

    program instructions to, responsive to the compliance indicator indicating that the compliance value is less than the compliance threshold, take a corrective action;

    wherein;

    the corrective action is an action selected from the group consisting of;

    notifying a user of a corrupted set of text segments in the selected set of web pages; and

    preventing the selected set of web pages from being accessed again.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×