Efficient fact checking method and system

• US 8,990,234 B1
• Filed: 03/11/2014
• Issued: 03/24/2015
• Est. Priority Date: 02/28/2014
• Status: Active Grant

• Current Assignee: Palo Alto Networks Incorporated
• Original Assignee: Lucas Myslinski
• Inventors: Myslinski, Lucas J.
• Primary Examiner(s): CHEEMA, AZAM M
• Application Number:
  US14/203,899
• Time in Patent Office: 378 Days
• US Class Current: 707/758
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/2457 with adaptation to user needs
• CPC Class Codes: G06F 16/2468 Fuzzy queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/226 Validation
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 5/048 Fuzzy inferencing
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

An efficient fact checking system analyzes and determines the factual accuracy of information and/or characterizes the information by comparing the information with source information. The efficient fact checking system automatically monitors information, processes the information, fact checks the information efficiently and/or provides a status of the information.

Claims

1. A method programmed in a non-transitory memory of a device comprising:

a. automatically analyzing broadcast information;

b. automatically fact checking the broadcast information by comparing the broadcast information with source information to generate a result, wherein comparing includes;

i. searching for an exact match of the broadcast information in the source information and returning the exact match search result of the exact match search if the exact match is found;

ii. otherwise, if the exact match is not found, then utilizing pattern matching for fact checking and returning the result of the pattern matching fact check if a pattern matching result confidence score is above a pattern matching result confidence threshold;

iii. otherwise, if the pattern matching result confidence score is not above the pattern matching result confidence threshold, then utilizing a natural language search for fact checking and returning the result of the natural language fact check if a natural language result confidence score is above a natural language result confidence threshold; and

iv. otherwise, returning a negative result value as the result; and

c. automatically presenting a status of the broadcast information in real-time based on the result of the comparison of the broadcast information with the source information,

wherein searching for the exact match begins searching the source information located on a fastest access time hardware device, then using the source information located on a second fastest access time hardware device, and then using the source information located on slower access time hardware devices until a device list has been exhausted;

wherein utilizing pattern matching begins utilizing the source information located on the fastest access time hardware device, then using the source information located on the second fastest access time hardware device, and then using the source information located on the slower access time hardware devices until the device list has been exhausted;

wherein the natural language search begins searching the source information located on the fastest access time hardware device, then using the source information located on the second fastest access time hardware device, and then using the source information located on the slower access time hardware devices until the device list has been exhausted;

wherein searching for the exact match begins searching the source information located in a designated fact checking database, then goes to a broader set of source information, and repeatedly goes to broader sets of source information until a broadest source information set has been exhausted;

wherein utilizing pattern matching begins utilizing the source information located in the designated fact checking database, then goes to the broader set of source information, and repeatedly goes to broader sets of source information until the broadest source information set has been exhausted; and

wherein the natural language search begins searching the source information located in the designated fact checking database, then going to the broader set of source information, and repeatedly going to broader sets of source information until the broadest source information set has been exhausted.

2. The method of claim 1 wherein most popular, most recent and most common information is stored in the fastest access time hardware device, and less popular, less recent and less common information is stored in the slower access hardware devices.

3. The method of claim 1 wherein searching for the exact match begins searching the source information classified by a plurality of keywords found in the broadcast information, then using the source information classified by a single keyword found in the broadcast information, and then using the source information classified by keywords related to the keywords found in the broadcast information;

wherein utilizing pattern matching begins utilizing the source information classified by the plurality of keywords found in the broadcast information, then using the source information classified by the single keyword found in the broadcast information, and then using the source information classified by the keywords related to the keywords found in the broadcast information; and

wherein the natural language search begins searching the source information classified by the plurality of keywords found in the broadcast information, then using the source information classified by the single keyword found in the broadcast information, and then using the source information classified by the keywords related to the keywords found in the broadcast information.

4. The method of claim 1 further comprising parsing the broadcast information into segments and prioritizing the segments, so that a highest priority segment is fact checked first, wherein priority is based on the relatedness of the segment to a current topic being discussed and when the segment was presented, wherein if the segment is not fact checked before a timeout threshold, then the segment is removed from a fact check queue.

5. The method of claim 4 wherein a plurality of fact check queues are implemented, wherein a first fact check queue contains the segments to be fact checked in real-time, and the second fact check queue contains the segments to be fact checked in non-real-time.

6. The method of claim 1 further comprising parsing the broadcast information into segments and prioritizing the segments, so that a highest priority segment is fact checked first, wherein priority is based on a presenter of the broadcast information, wherein if the presenter of the broadcast information has a validity rating below a first validity threshold, then the segments of the broadcast information from the presenter are prioritized in a highest priority group, and if the presenter of the broadcast information has the validity rating above the first threshold and below a second threshold, then the segments of the broadcast information from the presenter are prioritized in a second highest priority group which has a lower priority than the highest priority group, and if the presenter of the broadcast information has the validity rating above the second threshold, then the segments of the broadcast information from the presenter are prioritized in a third highest priority group which has a lower priority than the second highest priority group.

7. The method of claim 1 further comprising parsing the broadcast information into phrases, parsing the phrases into words, counting the number of words in each phrase, and comparing each phrase with the source information containing the same number of words as the number of words in the phrase being compared.

8. The method of claim 7 wherein the source information includes only source information in a classification matching a keyword detected in the broadcast information.

9. The method of claim 1 further comprising parsing the broadcast information into phrases, parsing the phrases into words, counting the number of words in each phrase, and comparing each phrase with the source information containing one more word than the number of words in the phrase being compared.

10. The method of claim 1 further comprising parsing the broadcast information into segments, and only fact checking randomly selected segments.

11. The method of claim 1 further comprising utilizing a plurality of fact checking implementations initially, wherein each fact checking implementation utilizes a different set of source information, and comparing results of each fact checking implementation, and iteratively eliminating a fact checking implementation of the plurality of fact checking implementations with a lowest confidence score until a single fact checking implementation remains.

12. The method of claim 11 wherein utilizing the plurality of fact checking implementations initially occurs in the first minute of a television program.

13. The method of claim 11 wherein the single fact checking implementation determined by eliminating other fact checking implementations is utilized for a specific type of content and is reused for future content that is the same type of content as the specific type of content.

14. The method of claim 11 wherein utilizing a plurality of fact checking implementations initially occurs in parallel while a previously determined best fact checking implementation is utilized for fact checking the broadcast information.

15. The method of claim 1 further comprising utilizing a plurality of fact checking implementations initially, wherein each fact checking implementation utilizes a different set of source information, and comparing results of each fact checking implementation, and iteratively eliminating any fact checking implementations of the plurality of fact checking implementations with a confidence score below a threshold until two fact checking implementations remain and are used in a continuous, competitive fact checking implementation wherein the results of the two remaining fact checking implementations are compared to determine a final fact check result.

16. The method of claim 1 wherein searching for the exact match begins searching the source information controlled by a media company, then using crowdsourced data as the source information, and then using world wide web data for fact checking;

wherein utilizing pattern matching begins utilizing the source information controlled by the media company, then using the crowdsourced data as the source information, and then using the world wide web data for fact checking; and

wherein the natural language search begins searching the source information controlled by the media company, then using the crowdsourced data as the source information, and then using the world wide web data for fact checking.

17. A method programmed in a non-transitory memory of a device comprising:

a. automatically analyzing broadcast information;

b. automatically fact checking the broadcast information by comparing the broadcast information with source information to generate a result, wherein comparing includes;

i. implementing a first fact check implementation for fact checking the broadcast information using the source information and returning a first fact check result of the first fact check implementation if the first fact check result of the first fact check is above a first confidence threshold;

ii. otherwise, if the first fact check result is not above the first confidence threshold, then implementing a second fact check implementation for fact checking the broadcast information using the source information and returning a second fact check result of the second fact check implementation if the second fact check result of the second fact check is above a second confidence threshold;

iii. otherwise, if the second fact check result is not above the second confidence threshold, then implementing a third fact check implementation for fact checking the broadcast information using the source information and returning a third fact check result of the third fact check implementation if the third fact check result of the third fact check is above a third confidence threshold; and

iv. otherwise, returning a negative result value as the result; and

c. automatically presenting a status of the broadcast information in real-time based on the comparison of the broadcast information with the source information,

wherein implementing the first fact check implementation begins searching the source information located on a fastest access time hardware device, then using the source information located on a second fastest access time hardware device, and then using the source information located on slower access time hardware devices until a device list has been exhausted;

wherein implementing the second fact check implementation begins utilizing the source information located on the fastest access time hardware device, then using the source information located on the second fastest access time hardware device, and then using the source information located on the slower access time hardware devices until the device list has been exhausted;

wherein implementing the third fact check implementation begins searching the source information located on the fastest access time hardware device, then using the source information located on the second fastest access time hardware device, and then using the source information located on the slower access time hardware devices until the device list has been exhausted;

wherein implementing the first fact check implementation begins searching the source information located in a designated fact checking database, then goes to a broader set of source information, and repeatedly goes to broader sets of source information until a broadest source information set has been exhausted;

wherein implementing the second fact check implementation begins utilizing the source information located in the designated fact checking database, then goes to the broader set of source information, and repeatedly goes to broader sets of source information until the broadest source information set has been exhausted; and

wherein implementing the third fact check implementation begins searching the source information located in the designated fact checking database, then going to the broader set of source information, and repeatedly going to broader sets of source information until the broadest source information set has been exhausted.

18. A device comprising:

a. a non-transitory memory for storing an application for automatically performing the following steps;

i. analyzing broadcast information;

ii. fact checking the broadcast information by comparing the broadcast information with source information to generate a result, wherein comparing includes;

(1) searching for an exact match of the broadcast information in the source information and returning the exact match search result of the exact match search if the exact match is found;

(2) otherwise, if the exact match is not found, then utilizing pattern matching for fact checking and returning the result of the pattern matching fact check if a pattern matching result confidence score is above a pattern matching result confidence threshold;

(3) otherwise, if the pattern matching result confidence score is not above the pattern matching result confidence threshold, then utilizing a natural language search for fact checking and returning the result of the natural language fact check if a natural language result confidence score is above a natural language result confidence threshold; and

(4) otherwise, returning a negative result value as the result; and

iii. presenting a status of the broadcast information in real-time based on the result of the comparison of the broadcast information with the source information,

wherein searching for the exact match begins searching the source information located on a fastest access time hardware device, then using the source information located on a second fastest access time hardware device, and then using the source information located on slower access time hardware devices until a device list has been exhausted;

wherein utilizing pattern matching begins utilizing the source information located on the fastest access time hardware device, then using the source information located on the second fastest access time hardware device, and then using the source information located on the slower access time hardware devices until the device list has been exhausted;

wherein the natural language search begins searching the source information located on the fastest access time hardware device, then using the source information located on the second fastest access time hardware device, and then using the source information located on the slower access time hardware devices until the device list has been exhausted;

wherein searching for the exact match begins searching the source information located in a designated fact checking database, then goes to a broader set of source information, and repeatedly goes to broader sets of source information until a broadest source information set has been exhausted;

wherein utilizing pattern matching begins utilizing the source information located in the designated fact checking database, then goes to the broader set of source information, and repeatedly goes to broader sets of source information until the broadest source information set has been exhausted; and

wherein the natural language search begins searching the source information located in the designated fact checking database, then going to the broader set of source information, and repeatedly going to broader sets of source information until the broadest source information set has been exhausted; and

b. a processor for processing the application.

19. The device of claim 18 wherein most popular, most recent and most common information is stored in the fastest access time hardware device, and less popular, less recent and less common information is stored in the slower access hardware devices.

20. The device of claim 18 wherein searching for the exact match begins searching the source information classified by a plurality of keywords found in the broadcast information, then using the source information classified by a single keyword found in the broadcast information, and then using the source information classified by keywords related to the keywords found in the broadcast information;

wherein utilizing pattern matching begins utilizing the source information classified by the plurality of keywords found in the broadcast information, then using the source information classified by the single keyword found in the broadcast information, and then using the source information classified by the keywords related to the keywords found in the broadcast information; and

wherein the natural language search begins searching the source information classified by the plurality of keywords found in the broadcast information, then using the source information classified by the single keyword found in the broadcast information, and then using the source information classified by the keywords related to the keywords found in the broadcast information.