Crowd-source as a backup to asynchronous identification of a type of form and relevant fields in a credential-seeking web page

• US 10,482,167 B2
• Filed: 09/24/2015
• Issued: 11/19/2019
• Est. Priority Date: 09/24/2015
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, LLC
• Inventors: Lupien, Nicolas, Lakhia, Michael, Gagnon-Lamonde, Hubert
• Primary Examiner(s): Khan, Shahid K
• Application Number:
  US14/864,448
• Publication Number:
  US 20170091163A1
• Time in Patent Office: 1,517 Days
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/226 Validation
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

A technique allows a credential manager application on a client computer system to identify fields and forms on a web page. An analysis server may automatically crawl web pages and identify the fields and form, then push the information to the client computer system for use by the credential manager. If the credential manager discovers the information is not available, the credential manager may analyze the web form to discover the fields and form information, then provide the discovered information to the analysis server for providing to other client computers. The analysis server may use crowd-sourcing for asynchronous verification of field and form information discovered by the analysis server or provided by the client computer.

Claims

1. A storage device comprising instructions that, when executed by a machine, cause the machine to at least:

discover field and form information for a web page of a web site, the web page accessed over a network by an analysis server;

create an annotated image of the web page to mark a position for a field of the field and form information, the annotated image created without displaying the web page to a user, the annotated image including the field and form information;

transmit the annotated image with the field and form information to a crowd-sourcing service;

accept the field and form information as validated field and form information responsive to a positive result from the crowd-sourcing service, the positive result indicating that the marked position of the field is correct;

receive corrected field and form information from a human reviewer responsive to a negative result from the crowd-sourcing service, the negative result indicating that the marked position of the field is incorrect;

send the annotated image with the field and form information to a human arbiter responsive to a mixed result from the crowd-sourcing service, and receive either corrected field and form information from the human arbiter responsive to a negative decision from the human arbiter or validated field and form information responsive to a positive decision from the human arbiter, the mixed result indicating that a minority of votes received by the crowd-sourcing service identify the marked position of the field as incorrect;

send the at least one of validated or corrected field and form information from the analysis server to a credential manager application;

instruct the credential manager application to cache the at least one of the validated or corrected field and form information, and the marked position of the field, in a cache memory; and

instruct the credential manager application to insert credentials into a web page based on the at least one of the validated or corrected field and form information stored in the cache memory.

2. The storage device of claim 1, wherein the instructions, when executed, cause the machine to instruct the credential manager application to invalidate some or all of the cached field and form information maintained by the credential manager application.

3. The storage device of claim 1, wherein the instructions, when executed, cause the machine to:

request at least three crowd-source responses from the crowd-sourcing service; and

consider the at least three crowd-source responses as a voting result.

4. The storage device of claim 1, wherein the instructions, when executed, cause the machine to:

use computer vision to view the annotated image of the web page annotated corresponding to the field and form information; and

return a decision based on the computer vision that indicates a confidence level that the field and form information is correct.

5. The storage device of claim 4, wherein the instructions, when executed, cause the machine to pass the annotated image to the human reviewer responsive to the confidence level being lower than a threshold.

6. The storage device of claim 1, wherein the instructions, when executed, cause the machine to employ a web crawler to examine web pages.

7. The storage device of claim 6, wherein the web crawler is to crawl a set of web pages.

8. A computer system for determining web form information in a web page for a web site, the computer system comprising:

one or more processors; and

memory to store instructions that, when executed by the one or more processors, cause at least one of the one or more of the processors to at least;

discover field and form information for a web page of a web site, the web page accessed over a network by an analysis server;

create an annotated image of the web page to mark a position for a field of the field and form information, the annotated image created without displaying the web page to a user, the annotated image including the field and form information;

transmit the annotated image including the field and form information to a crowd-sourcing service;

accept the field and form information as validated field and form information responsive to a positive result from the crowd-sourcing service, the positive result indicating that the marked position of the field is correct;

receive corrected field and form information from a human reviewer responsive to a negative result from the crowd-sourcing service, the negative result indicating that the marked position of the field is incorrect;

send the annotated image with field and form information to a human arbiter responsive to a mixed result from the crowd-sourcing service, and receive either corrected field and form information from the human arbiter responsive to a negative decision from the human arbiter or validated field and form information responsive to a positive decision from the human arbiter, the mixed result indicating that a minority of votes received by the crowd-sourcing service identify the marked position of the field as incorrect;

send the at least one of the validated or corrected field and form information from the analysis server to a credential manager application;

instruct the credential manager application to cache the at least one of the validated or corrected field and form information, and the marked position of the field, in a cache memory; and

instruct the credential manager application to insert credentials into a web page based on the at least one of the validated or corrected field and form information stored in the cache memory.

9. The computer system of claim 8, wherein the at least one of the one or more processors is to instruct the credential manager application to invalidate at least a portion of the cached field and form information maintained by the credential manager application.

10. The computer system of claim 8, wherein at least one of the one or more processors is to at least:

request at least three crowd-source responses from the crowd-sourcing service; and

consider the at least three crowd-source responses as a voting result.

11. The computer system of claim 8, wherein at least one of the one or more processors is to at least:

use computer vision to view the annotated image of the web page annotated corresponding to the field and form information; and

return a decision based on the computer vision that indicates a confidence level that the field and form information is correct.

12. The computer system of claim 11, wherein at least one of the one or more processors is to pass the annotated image to the human reviewer responsive to the confidence level being lower than a threshold.

13. The computer system of claim 8, wherein at least one of the one or more processors is to receive the web page by employing a web crawler to examine web pages.

14. The computer system of claim 13, wherein the web crawler is to crawl a set of web pages.

15. A method for determining web form information in a web page for a web site, the method comprising:

discovering field and form information for a web page of a web site, the web page accessed over a network by an analysis server;

creating an annotated image of the web page to mark a position for a field of the field and form information, the annotated image created without displaying the web page to a user, the annotated image including the field and form information;

transmitting the annotated image with the field and form information to a crowd-sourcing service;

accepting the field and form information as validated field and form information responsive to a positive result from the crowd-sourcing service, the positive result indicating that the marked position of the field is correct;

receiving corrected field and form information from a human reviewer responsive to a negative result from the crowd-sourcing service, the negative result indicating that the marked position of the field is incorrect;

sending the annotated image with the field and form information to a human arbiter responsive to a mixed result from the crowd-sourcing service, and receive either corrected field and form information from the human arbiter responsive to a negative decision from the human arbiter or validated field and form information responsive to a positive decision from the human arbiter, the mixed result indicating that a minority of votes received by the crowd-sourcing service identify the marked position of the field as incorrect;

sending the at least one of the validated or corrected field and form information from the analysis server to a credential manager application;

instructing the credential manager application to cache the at least one of the validated or corrected field and form information, and the position of the field, in a cache memory; and

instructing the credential manager application to insert credentials into a web page based on the at least one of the validated or corrected field and form information stored in the cache memory.

16. The method of claim 15, further including instructing the credential manager application to invalidate at least a portion of the cached field and form information maintained by the credential manager application.

17. The method of claim 15, wherein the validating of the field and form information using crowd-sourcing service includes:

requesting at least three crowd-source responses from the crowd-sourcing service; and

considering the at least three crowd-source responses as a voting result.

18. The method of claim 15, further including:

using computer vision to view the annotated image of the web page annotated corresponding to the field and form information; and

return a decision based on the computer vision that indicates a confidence level that the field and form information is correct.

19. The method of claim 15, wherein the receiving of the web page includes employing a web crawler to examine web pages.