System and method to determine the validity of an interaction on a network

• US 20020002445A1
• Filed: 01/19/2001
• Published: 01/03/2002
• Est. Priority Date: 02/14/2000
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Go To.com Incorporated
• Inventors: Doliov, Stephan
• Granted Patent: US 7,136,860 B2
• Time in Patent Office: Days
• US Class Current: 703/2
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: G06F 2221/2135 Metering
• CPC Class Codes: G06Q 10/067 Enterprise or organisation modelling
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/04 Billing or invoicing
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 63/126 the source of the received data
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/14 Session management for real-time applications in data packet communications networks H04L65/1066
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 : Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 : Object-oriented database structure processing

Abstract

The methods and systems of the invention utilize limited data to yield information about the validity of any given interaction with a website. Once validity information is available, an operator can determine whether or not to continue offering interactions to a given user. The determination could also relate to whether to report website interaction statistics based on undesired interactions, how to handle billing or payment for such undesired interactions, and what type of content to send to users who are interacting with the website in an undesirable manner.

Claims

1. A method of identifying illegitimate interactions of a presumed user on a network, the method comprising:

collecting data from user interactions on a network, the data including aggregate measure data and unique feature data about the presumed user;

storing the data in a database;

building predictive models with the aggregate measure and unique feature data to identify illegitimate interactions with the network; and

identifying the illegitimate interactions in the database using the predictive models.

2. The method of claim 1 further including:

applying a mathematical model of interactions derived from at least one probabilistic approach;

applying a mathematical model of interactions based on at least one stochastic approach; and

determining the legitimacy of the interactions using the stochastic and probabilistic mathematical equations.

3. The method of claim 2 where the at least one probabilistic approach includes logistic regression.

4. The method of claim 2 where the at least one stochastic approach includes linear discriminant analysis.

5. The method of claim 2 further including:

assigning a classification to the value created by the probabilistic and stochastic derived mathematical models of interaction;

computing values of the probabilistic and stochastic derived mathematical models of interaction;

for the probabilistic mathematical models, placing an interaction into a class based on a computed probability of belonging to that class; and

for the stochastic mathematical models, placing an interaction into the class with the highest computed value.

6. The method of claim 5 further including:

periodically reassessing the probabilistic and stochastic mathematical models against a standard;

applying a retrospective validating process; and

applying a prospective validating process.

7. The method of claim 1 further comprising:

evaluating multiple network interactions;

creating a classification table; and

determining a model of legitimate and illegitimate interactions from the classification table.

8. The method of claim 1 further including:

creating a database of illegitimate interactions.

9. The method of claim 8 further including:

evaluating the database to ensure that the probabilistic and stochastic derived mathematical models of interaction are reliable; and

discarding unreliable probabilistic and stochastic mathematical models of interaction.

10. The method of claim 9 further including:

collecting additional aggregate measure and unique feature data; and

deriving new probabilistic and stochastic derived mathematical models of interaction based on existing collected data and the additional collected aggregate measure and unique feature data.

11. The method of claim 1 where the collection of aggregate measure data comprises:

collecting data on a number of clicks per internet protocol address for a given time period;

collecting data on a number of unique queries per user session;

collecting data on a number of network clicks for a given time period; and

collecting data on a number of distinct referral partners who could access the network.

12. The method of claim 1 where the collection of unique feature data comprises:

collecting data on an origin of the presumed user;

collecting data on a time of the interactions;

collection data on a type of the interactions; and

collecting data on presumed measures of uniqueness of the presumed user.

13. A method for creating reports according to website interactions after determining the validity of the website interactions, the method comprising:

collecting aggregate measure data and unique feature data about the website interaction;

building predictive models with the aggregate measure and unique feature data to identify undesirable interactions with the website; and

removing the undesirable interactions from the reports, once identified.

14. The method of claim 13 further including:

applying statistical data analysis techniques to identify undesirable interactions with the website before removing the undesirable interactions from the reports.

15. The method of claim 14 wherein the statistical data analysis is used to create predictive models of undesirable interactions.

16. The method of claim 15 wherein the predictive models are used to produce at least one of a probability value and a behavior group.

17. The method of claim 16 further including:

formulating a business rule about which probability value or behavior group constitutes undesirable interactions with the Website.

18. The method of claim 15 further including:

creating a database of undesirable interactions.

19. The method of claim 18 further including:

evaluating the database to ensure that the predictive models are reliable; and

discarding unreliable predictive models.

20. The method of claim 19 further including:

collecting additional aggregate measure and unique feature data; and

deriving new predictive models based on existing collected data and the additional collected data.

21. The method of claim 13, wherein the predictive model is biased in favor of a customer.

22. A system for creating reports according to website interactions after determining the validity of the website interactions, the method comprising:

a first processor to collect aggregate measure data and unique feature data about a website user;

a second processor to create predictive models with the aggregate measure and unique feature data to identify undesirable interactions with the website, wherein the undesirable interactions are removed from the reports, once identified.

23. The system of claim 22 wherein statistical data analysis techniques are used to identify undesirable interactions with the website before removing the undesirable interactions from the reports.

24. The system of claim 23 wherein the statistical data analysis is used to create predictive models of undesirable interactions.

25. The system of claim 24 wherein the predictive models are used to produce at least one of a probability value and a behavior group.

26. The system of claim 25 wherein a business rule is formulated about which probability value or behavior group constitutes undesirable interactions with the Website.

27. The system of claim 24 wherein a database is created of undesirable interactions.

28. The system of claim 27 wherein the database is evaluated to ensure that the predictive models are reliable and unreliable predictive models are discarded.

29. The system of claim 28 wherein new data is collected and new predictive models are derived based on existing collected data and the new collected data.

30. The system of claim 22 wherein the predictive model is biased in favor of a customer.

31. A method of rating user interactions on a network, the method comprising:

collecting data from user interactions on a network, the data including aggregate measure data and unique feature data about the user;

storing the data in a database; and

building predictive models with the aggregate measure and unique feature data to rate the interactions with the network.

32. The method of claim 31 further including identifying illegitimate interactions on the network.

33. The method of claim 31 further including identifying invalid interactions on the network.

34. The method of claim 31 further including identifying unauthorized interactions on the network.

35. The method of claim 31 further including:

applying a probabilistic approach to mathematical modeling of the interactions;

applying a stochastic approach to derive mathematical models of the interaction; and

identifying interactions on the network using the probabilistic and stochastic mathematical models of interaction.

36. The method of claim 35 further including:

assigning a class to values computed from the probabilistic and stochastic derived mathematical equations;

computing the values of the probabilistic and the stochastic mathematical models of interaction;

for the probabilistic mathematical models, placing an interaction into a class based on the computed probability of belonging to the class; and

for the stochastic models, placing an interaction into the class with the highest computed value.

37. The method of claim 36 further including:

periodically reassessing the probabilistic and stochastic mathematical equations against a standard;

applying a retrospective validating process; and

applying a prospective validating process.

38. The method of claim 37 further comprising:

evaluating multiple network interactions;

creating a classification table; and

determining a model of interactions from the classification table.

39. The method of claim 35 further including:

creating a database of interactions.

40. The method of claim 39 further including:

evaluating the database to ensure that the probabilistic and the stochastic mathematical models of interaction are reliable; and

discarding unreliable probabilistic and stochastic mathematical models of interaction.

41. The method of claim 40 further including:

collecting additional aggregate measure and unique feature data; and

deriving new probabilistic and stochastic mathematical models of interaction based on existing collected data and the additional collected data.

42. The method of claim 31 where the collection of aggregate measure data comprises:

collecting data on a number of clicks per internet protocol address for a given time period;

collecting data on a number of unique queries per user session;

collecting data on a number of network clicks for a given time period; and

collecting data on a number of distinct referral partners who could access the network.

43. The method of claim 31 where the collection of unique feature data comprises:

collecting data on an origin of the user;

collecting data on a time of the interactions;

collection data on a type of the interactions; and

collecting data on presumed measures of uniqueness of the user.