Content quality apparatus, systems, and methods

• US 7,716,228 B2
• Filed: 09/25/2007
• Issued: 05/11/2010
• Est. Priority Date: 09/25/2007
• Status: Active Grant

• Current Assignee: Aurea Software Incorporated (ESW Capital, LLC)
• Original Assignee: FirstRain, Inc. (Ignite Technologies Incorporated)
• Inventors: Mohanty, Binay, Rewari, Gaurav, Sahasrabudhe, Sadanand, Yu, Xiang
• Primary Examiner(s): Rones; Charles
• Assistant Examiner(s): Khoshnoodi; Fariborz
• Application Number:
  US11/861,141
• Publication Number:
  US 20090083251A1
• Time in Patent Office: 959 Days
• Field of Search: 707/5, 707/4, 707/3, 707/101
• US Class Current: 707/748
• CPC Class Codes: G06Q 40/06 Asset management; Financial planning or analysis

Abstract

Embodiments herein receive a set of content quality threshold values, a search string, and a content data stream at a content quality metric (CQM) apparatus. Content segments associated with the content data stream are scored and/or graded according to a set of content relevance scales. The content data stream is then filtered to include only passing content segments and intermediate calculation values used to determine whether a content segment is passing. Other embodiments are described and claimed.

Claims

1. An apparatus, comprising:

one or more processors;

a scoring and grading module to score, using the one or more processors, at least one content segment of a set of content segments associated with a content data stream according to a set of scorable content relevance scales to obtain at least one relevance score, the at least one relevance score corresponding to a numerical value at a point along an associated scorable relevance scale, and to grade at least one content segment of the set of content segments according to a set of gradable content relevance scales to obtain at least one relevance grade, the at least one relevance grade corresponding to a zone along an associated gradable relevance scale;

a content segment filter communicatively coupled to the scoring and grading module to filter the set of content segments to include only passing content segments, a content segment defined as passing based on a mathematical function of the at least one relevance score being related to a mathematical function of a set of content quality threshold values associated with the at least one relevance score according to a selected scoring threshold relationship or based on a mathematical function of the at least one relevance grade being related to a mathematical function of a set of content quality threshold values associated with the at least one relevance grade according to a selected grading threshold relationship;

a historical content segment database coupled to the scoring and grading module to store at least one of a set of previously-received content segments or an index of storage locations associated with the set of previously-received content segments; and

a syntactic comparison module coupled to the scoring and grading module to compare a sentence associated with a newly-received content segment to each of a plurality of sentences associated with a concatenated set of previously-received content segments to identify sentences in the newly-received content segment that are syntactically different from each of the plurality of sentences associated with the concatenated set of previously-received content segments, to compare a paragraph associated with the newly-received content segment to each of a plurality of paragraphs associated with the concatenated set of previously-received content segments to identify paragraphs in the newly-received content segment that are syntactically different from each of the plurality of paragraphs associated with the concatenated set of previously-received content segments, and to obtain a syntactic difference score based upon a number of syntactically different sentences and a number of syntactically different paragraphs found.

2. The apparatus of claim 1, further comprising:

a user requirement input module communicatively coupled to the scoring and grading module and to the content segment filter to accept at least one of a search string or the set of content quality threshold values.

3. The apparatus of claim 2, further comprising:

a content quality feedback module coupled to the user requirement input module to accept feedback information from at least one content consumer and to adjust the set of content quality threshold values in order to dynamically adjust subsequent output to match user requirements.

4. The apparatus of claim 1, wherein the mathematical function of the at least one relevance score, the mathematical function of the at least one relevance grade, and the mathematical function of the set of content quality threshold values comprise at least one of a numerical function, a boolean function, or an alphanumeric string comparison function.

5. The apparatus of claim 1, configured to calculate the mathematical function of the at least one relevance score using a single one of the set of content relevance scales and a single content segment and configured to calculate the mathematical function of the at least one relevance grade using a single one of the set of content relevance scales and a single content segment.

6. The apparatus of claim 1, further including:

a composite scoring module to calculate a composite mathematical function of the at least one relevance score using a plurality of content relevance scales and a single content segment and to calculate a composite mathematical function of the at least one relevance grade using a plurality of content relevance scales and a single content segment.

7. The apparatus of claim 1, further including:

a multi-segment scoring module to calculate a multi-segment mathematical function of the at least one relevance score using a single content relevance scale and a plurality of content segments and to calculate a multi-segment mathematical function of the at least one relevance grade using a single content relevance scale and a plurality of content segments.

8. The apparatus of claim 1, configured to calculate a composite multi-dimensional relevance score using a plurality of content relevance scales and configured to calculate a multi-dimensional relevance grade using a plurality of content relevance scales and a plurality of content segments.

9. The apparatus of claim 1, further comprising:

a content source lookup table coupled to the scoring and grading module, each record of the content source lookup table to relate at least one of a standard content source identifier or a standard content source class identifier to at least one of an entity-specific content source relevance score, a topic-specific content source relevance score, a standard content source relevance score, an entity-specific content source relevance grade, a topic-specific content source relevance grade, or a standard content source relevance grade.

10. The apparatus of claim 9, wherein the content source lookup table is configured to use the standard content source class identifier to retrieve at least one of the entity-specific content source relevance score, the topic-specific content source relevance score, the standard content source relevance score, the entity-specific content source relevance grade, the topic-specific content source relevance grade, or the standard content source relevance grade in an absence of the standard content source identifier.

11. The apparatus of claim 9, wherein a market entity record of the content source lookup table further comprises a market entity identifier and wherein the content source lookup table is configured to retrieve at least one of the entity-specific content source relevance score or the entity-specific content source relevance grade if the market entity identifier is found in a query issued to the apparatus.

12. The apparatus of claim 9, wherein a market topic record of the content source lookup table further comprises a market topic identifier and wherein the content source lookup table is configured to retrieve at least one of the topic-specific content source relevance score or the topic-specific content source relevance grade based on the market topic identifier being found in a query issued to the apparatus.

13. The apparatus of claim 1, further comprising:

a content type lookup table coupled to the scoring and grading module, each record of the content type lookup table to relate a standard content type identifier to a content type relevance score and to a standard content type relevance grade.

14. The apparatus of claim 1, further comprising:

a semantic comparison module coupled to the scoring and grading module to perform a semantic comparison between content associated with the newly-received content segment and content associated with the concatenated version of the previously-received content segments to determine whether the content associated with the newly-received content segment contains new information.

15. The apparatus of claim 1, further comprising:

a connectedness metric lookup table coupled to the scoring and grading module, each record of the connectedness metric lookup table to relate a content segment identifier to a content segment connectedness metric value and to relate an identifier associated with a source of the content segment to a content source connectedness metric value.

16. The apparatus of claim 1, further comprising:

a specialty topic weighting lookup table coupled to the scoring and grading module, each record of the specialty topic weighting lookup table to relate a source of the at least one content segment and a specialty topic to a weighting factor, the weighting factor to be used to decrease an obscurity score delivered to a market information subscriber with expertise in the specialty topic.

17. The apparatus of claim 1, further comprising:

an entity-topic knowledgebase (ETK) coupled to the scoring and grading module including an entity dataset, a topic dataset, and a set of semantic rules associated with the entity dataset and with the topic dataset, the set of semantic rules to be used to identify a reference to at least one of an entity or a topic in a content segment.

18. The apparatus of claim 17, further comprising:

an ETK administrative graphical user interface (GUI) communicatively coupled to the ETK to receive a market entity dataset, a market topic dataset, and a set of semantic rules;

a market entity loading module coupled to the ETK to load the market entity dataset and a subset of semantic rules associated with a plurality of market entity representations contained in the market entity dataset; and

a market topic loading module coupled to the ETK to load the market topic dataset and a subset of semantic rules associated with a plurality of market topic representations contained in the market topic dataset.

19. The apparatus of claim 1, further comprising:

an impact lookup table coupled to the scoring and grading module, each record of the impact lookup table to relate at least one of an entity, a topic, a keyword, or a combination of the entity, the topic or the keyword to an impact metric value associated with the entity, the topic, the keyword, or the combination.

20. A method of computing content relevance, comprising:

receiving a set of content quality threshold values and a search string at a content quality metric (CQM) apparatus;

receiving a content data stream at the CQM apparatus;

scoring at least one content segment associated with the content data stream according to a set of scorable content relevance scales to achieve a set of relevance scores for the at least one content segment;

grading, using one or more processors, the at least one content segment according to a set of gradable content relevance scales to obtain a set of relevance grades for the at least one content segment;

filtering the content data stream to include passing content segments and intermediate calculation values used to determine whether a content segment is passing, a content segment defined as passing based on a mathematical function of at least one of the set of relevance scores or the set of relevance grades being related to a mathematical function of a set of content threshold values according to a selected relationship;

determining a subset of previously-received content segments stored in an historical content segment database;

concatenating the subset of previously-received content segments to create a concatenated content segment; and

performing a syntactic text comparison of a newly-received content segment to the concatenated content segment to identify portions of the newly-received content segment that are different from the concatenated content segment and to calculate a syntactic difference score.

21. The method of claim 20, wherein at least one of the set of scorable content relevance scales or the set of gradable content relevance scales comprises at least one of a source type metric, a content type metric, an incremental content metric, an obscurity metric, an applicability to user query metric, or an impact metric.

22. The method of claim 20, further including:

obtaining at least one of a standard content source identifier or a standard content source class identifier associated with the at least one content segment;

retrieving at least one content source record from a content source lookup table using the standard content source identifier based on the standard content source identifier being available;

retrieving the at least one content source record using the standard content source class identifier based on the standard content source identifier being not available;

determining whether a market entity identifier associated with the at least one content source record exists in a user query associated with content segment retrieval operations;

based on the market entity identifier associated with the at least one content source record existing in the user query, retrieving at least one of an entity-specific content source relevance score or an entity-specific content source relevance grade associated with the at least one record;

determining whether a market topic identifier associated with the at least one content source record exists in a user query associated with the content segment retrieval operations;

based on the market topic identifier associated with the at least one content source record existing in the user query, retrieving at least one of a topic-specific content source relevance score or a topic-specific content source relevance grade associated with the at least one record; and

based on neither the market entity identifier nor the market topic identifier existing in the user query, retrieving at least one of a standard content source relevance score or a standard content source relevance grade.

23. The method of claim 20, further including:

obtaining a standard content type identifier associated with the at least one content segment;

retrieving at least one content type record from a content type lookup table using the standard content type identifier; and

retrieving at least one of a standard content type relevance score or a standard content type relevance grade associated with the standard content type identifier.

24. The method of claim 20, further including:

returning an incremental content score of zero and a lowest possible incremental content grade based on the syntactic difference score being less than or equal to a selected incremental content threshold score;

performing a semantic comparison operation on the newly-received content segment and on the concatenated content segment to obtain a semantic difference score; and

assigning at least one of an incremental content score and an incremental content grade to the newly-received content segment based upon an amount of content that is both syntactically different and semantically different between the newly received content segment and the concatenated content segment.

25. The method of claim 20, further including:

retrieving a connectedness record from a connectedness metric lookup table using a content segment identifier associated with the at least one content segment;

retrieving a content segment connectedness metric value and a content source connectedness value from the connectedness metric lookup table;

calculating at least one of an obscurity score or an obscurity grade associated with the at least one content segment, wherein the obscurity score comprises a function of the content segment connectedness metric value and the content source connectedness value and wherein the obscurity grade comprises a function of the content segment connectedness metric value and the content source connectedness value;

retrieving at least one specialty topic weighting value from a specialty topic weighting table using at least one specialty topic associated with a user profile and with a source of the at least one content segment based on the at least one specialty topic being associated with the at least one content segment; and

performing a weighting operation on the obscurity score using the at least one specialty topic weighting value.

26. The method of claim 20, further including:

parsing a user query into at least one of an entity, a topic, a keyword, or a key phrase, wherein the user query is used to retrieve the at least one content segment;

retrieving at least one of an entity-indicative keyword or an entity-indicative key phrase from an entity-topic knowledge base (ETK) using the entity;

retrieving at least one of a topic-indicative keyword or a topic-indicative key phrase from the ETK using the topic;

calculating at least one of a set of individual applicability to user query scores or a set of individual applicability to user query grades, wherein at least one of each individual applicability to user query score or each applicability to user query grade is associated with the keyword, the key phrase, the entity-indicative keyword, the entity-indicative key phrase, the topic-indicative keyword, or the topic-indicative key phrase, wherein at least one of each individual applicability to user query score or each applicability to user query grade is based upon at least one of a frequency of occurrence of the keyword, the key phrase, the entity-indicative keyword, the entity-indicative key phrase, the topic-indicative keyword, or the topic-indicative key phrase in the at least one content segment, a prominence of location of the keyword, the key phrase, the entity-indicative keyword, the entity-indicative key phrase, the topic-indicative keyword, or the topic-indicative key phrase in the at least one content segment, a prominence of textual attributes associated with the keyword, the key phrase, the entity-indicative keyword, the entity-indicative key phrase, the topic-indicative keyword, or the topic-indicative key phrase in the at least one content segment, an occurrence of the keyword, the key phrase, the entity-indicative keyword, the entity-indicative key phrase, the topic-indicative keyword, or the topic-indicative key phrase in a picture caption within the at least one content segment, or an occurrence of the keyword, the key phrase, the entity-indicative keyword, the entity-indicative key phrase, the topic-indicative keyword, or the topic-indicative key phrase in anchor text associated with a hypertext transport protocol (HTTP) link pointing to the at least one content segment;

calculating a composite applicability to user query score associated with the at least one content segment, wherein the composite applicability to user query score comprises a mathematical function of the set of individual applicability to user query scores; and

calculating a composite applicability to user query grade associated with the at least one content segment, wherein the composite applicability to user query grade comprises a mathematical function of the set of individual applicability to user query scores.

27. The method of claim 20, further including:

identifying at least one of a segment entity, a segment topic, or a segment keyword in the at least one content segment;

querying an impact lookup table for a segment impact metric value associated with the segment entity, a segment impact metric value associated with the segment topic, and a segment impact metric value associated with the segment keyword and retrieving the segment impact metric value associated with the segment entity, the segment impact metric value associated with the segment topic, and the segment impact metric value associated with the segment keyword;

calculating at least one of an overall segment impact score or an overall segment impact grade as a function of the segment impact metric value associated with the segment entity, the segment impact metric value associated with the segment topic, and the segment impact metric value associated with the segment keyword;

parsing a user query into at least one of a query entity, a query topic, or a query keyword, wherein the user query is used to retrieve the at least one content segment;

querying the impact lookup table for a query impact metric value associated with the query entity, a query impact metric value associated with the query topic, and a query impact metric value associated with the query keyword and retrieving the query impact metric value associated with the query entity, the query impact metric value associated with the query topic, and the query impact metric value associated with the query keyword;

calculating at least one of an overall query impact score or an overall query impact grade as a function of the query impact metric value associated with the query entity, the query impact metric value associated with the query topic, and the query impact metric value associated with the query keyword;

calculating a composite impact score as a function of the overall segment impact score and the overall query impact score; and

calculating a composite impact grade as a function of the overall segment impact grade and the overall query impact grade.

28. A computer-readable medium having instructions, wherein the instructions, when executed, result in at least one processor performing:

receiving a set of content quality threshold values and a search string at a content quality metric (CQM) apparatus;

receiving a content data stream at the CQM apparatus;

scoring at least one content segment associated with the content data stream according to a set of scorable content relevance scales to achieve a set of relevance scores for the at least one content segment;

grading the at least one content segment according to a set of gradable content relevance scales to obtain a set of relevance grades for the at least one content segment;

filtering the content data stream to include passing content segments and intermediate calculation values used to determine whether a content segment is passing, a content segment defined as passing based on a mathematical function of at least one of the set of relevance scores or the set of relevance grades being related to a mathematical function of a set of content threshold values according to a selected relationship;

determining a subset of previously-received content segments stored in an historical content segment database;

concatenating the subset of previously-received content segments to create a concatenated content segment; and

performing a syntactic text comparison of a newly-received content segment to the concatenated content segment to identify portions of the newly-received content segment that are different from the concatenated content segment and to calculate a syntactic difference score.