Predictive modeling of consumer financial behavior using supervised segmentation and nearest-neighbor matching

• US 6,839,682 B1
• Filed: 10/03/2000
• Issued: 01/04/2005
• Est. Priority Date: 05/06/1999
• Status: Expired due to Term

• Current Assignee: Callahan Cellular LLC (Intellectual Ventures LLC)
• Original Assignee: Fair Isaac Corporation
• Inventors: Blume, Matthias, Lazarus, Michael A., Peranich, Larry S., Vernhes, Frederique, Caid, William R., Dunning, Ted E., Russell, Gerald R., Sitze, Kevin L.
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): Heck, Michael C.
• Application Number:
  US09/679,022
• Time in Patent Office: 1,554 Days
• Field of Search: 705/10
• US Class Current: 705/7.31
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0202 Market predictions or forecasting for commercial activities
• CPC Class Codes: G06Q 30/0207 Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0255 based on user history
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]

Abstract

Predictive modeling of consumer financial behavior, including determination of likely responses to particular marketing efforts, is provided by application of consumer transaction data to predictive models associated with merchant segments. The merchant segments are derived from the consumer transaction data based on co-occurrences of merchants in sequences of transactions. Merchant vectors represent specific merchants, and are aligned in a vector space as a function of the degree to which the merchants co-occur more or less frequently than expected. Consumer vectors are developed within the vector space, to represent interests of particular consumers by virtue of relative vector positions of consumer and merchant vectors. Various techniques, including clustering, supervised segmentation, and nearest-neighbor analysis, are applied separately or in combination to generate improved predictions of consumer behavior.

Claims

1. A method of predicting financial behavior of consumers, comprising:

obtaining a set of input transactions for a plurality of consumers with respect to a plurality of merchants;

defining at least one merchant segment, each merchant being associated with at least one of the defined merchant segments; and

for at least one consumer, applying the input transactions of the consumer by a computer to each of at least one merchant segment predictive model, each merchant segment predictive model defining for a merchant segment a prediction function between input transactions in a past time interval and financial behavior in a subsequent time interval, to produce for each consumer a predicted behavior in each of at least a subset of the merchant segments.

2. The method of claim 1, wherein the predicted behavior comprises a likelihood of positive response to an offer.

3. The method of claim 1, wherein the predicted behavior comprises a spending level with respect to a merchant.

4. The method of claim 1, further comprising:

generating a consumer vector for each of at least a subset of the consumers;

generating a merchant vector for each of at least a subset of the merchants;

wherein defining at least one merchant segment comprises performing supervised segmentation on the merchant vectors.

5. The method of claim 4, wherein performing supervised segmentation comprises applying a learning vector quantization algorithm to the merchant vectors.

6. The method of claim 4, wherein performing supervised segmentation comprises:

initializing a set of segment vectors;

accepting at least one segment label for at least one of the merchants; and

for each of at least a subset of the labeled merchants;

selecting at least one segment vector for a merchant having a merchant vector;

determining whether the selected segment vector matches the segment label for the merchant; and

responsive to the determination, adjusting zero or more of the segment vectors.

7. The method of claim 6, wherein selecting at least one segment vector for a merchant comprises selecting a segment vector that is closest to the merchant vector corresponding to the merchant.

8. The method of claim 6, wherein selecting at least one segment vector for a merchant comprises selecting at least one segment vector having a tolerance range that includes the value of the merchant vector.

9. The method of claim 1, further comprising:

training a predictive model using the predicted behavior of a plurality of consumers in at least a subset of the merchant segments, and additional observed behavior for the plurality of consumers with regard to a target segment not included in the subset of merchant segments; and

for at least one target consumer;

providing as input to the trained predictive model predicted behavior of the target consumer in at least a subset of the merchant segments; and

obtaining from the trained predictive model a predicted behavior of the target consumer with respect to the target segment.

10. The method of claim 1, further comprising:

for at least one consumer, associating the consumer with the merchant segment for which the consumer had the highest predicting spending relative to other merchant segments.

11. The method of claim 1, further comprising:

generating a consumer vector for each of at least a subset of the consumers;

generating a merchant vector for each of at least a subset of the merchants;

for at least one merchant segment, determining a segment vector as a summary vector of merchant vectors of merchants associated with the segment; and

for at least one consumer, associating the consumer with the merchant segment having the greatest dot product between the segment vector of the segment and a consumer vector of the consumer.

12. The method of claim 1, further comprising:

for at least one merchant segment;

ranking the consumers by their predicted spending in the merchant segment; and

determining for at least one consumer a percentile ranking in the merchant segment; and

for each consumer;

determining the merchant segment in which the consumer'"'"'s percentile ranking is the highest, to uniquely associate each consumer with one merchant segment; and

for at least one merchant segment, determining summary transaction statistics for the consumers uniquely associated with the merchant segment.

13. The method of claim 1, further comprising:

for at least one merchant segment;

ranking the consumers by their predicted spending in the merchant segment;

determining for at least one consumer a percentile ranking in the merchant segment;

selecting as a population, the consumers having a percentile ranking in excess of predetermined percentile threshold; and

determining summary transaction statistics for selected population of consumers.

14. The method of claim 1, further comprising:

establishing for at least one merchant in the transaction data a merchant vector; and

updating the merchant vector of at least one merchant relative to the merchant vectors of other merchants according to co-occurrences of each merchant in the transaction data.

15. The method of claim 14, further comprising:

updating the merchant vector of at least one merchant based upon an unexpected amount deviation in a frequency of co-occurrence of the merchant with other merchants.

16. The method of claim 14, further comprising:

determining a co-occurrence frequency for at least one merchant with at least one other merchant in the transaction data;

determining for at least one pair of merchants, a relationship strength between the pair of merchants based on how much the determined co-occurrence frequency deviates from an expected co-occurrence frequency;

for at least one pair of merchant vectors, mapping the relationship strength into a vector space as a desired dot product between respective merchant vectors the merchants in the pair; and

updating at least one merchant vector so that the actual dot products between each pair of merchant vectors approximate the desired dot product between the merchant vectors.

17. The method of claim 16, wherein determining for at least one pair of merchants a relationship strength between the pair of merchants further comprises:

determining the relationship strength by $r_{\mathrm{ij}}=\frac{T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}}{{\mathrm{\sigma <br><br>}}_{\mathrm{ij}}}\approx <br><br>\frac{T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}}{\sqrt{{\hat{T}}_{\mathrm{ij}}}}$

where

r_ij is the relationship strength between merchant_i and merchant_j in a pair of merchants;

T_ij is the actual co-occurrence frequency of merchant_i and merchant_j in the transaction data; and

{circumflex over (T)}_ij is the expected co-occurrence frequency of merchant_i and merchant_j in the transaction data.

18. The method of claim 16, wherein determining for at least one pair of merchants a relationship strength between the pair of merchants further comprises:

determining the relationship strength by

r_ij=sign(T_ij−

{circumflex over (T)}_ij)·

√

{square root over (2ln λ

)}

where

r_ij is the relationship strength between merchant_i and merchant_j in a pair of merchants;

λ

is a log-likelihood ratio;

T_ij is the actual co-occurrence frequency of merchant_i and merchant_j in the transaction data; and

{circumflex over (T)}_ij is the expected co-occurrence frequency of merchant_i and merchant_j in the transaction data.

19. The method of claim 16, wherein determining for at least one pair of merchants a relationship strength between the pair of merchants further comprises:

determining the relationship strength by $r_{\mathrm{ij}}=\mathrm{sign}<br><br>\left(T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}\right)·<br><br>\sqrt{\frac{2<br><br>\ln <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\lambda <br><br>}}{\sqrt{{\hat{T}}_{\mathrm{ij}}}}}=\mathrm{sign}<br><br>\left(T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}\right)·<br><br>\sqrt{2<br><br>\ln <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\lambda <br><br>}}·<br><br>{\hat{T}}_{\mathrm{ij}}^{\frac{1}{4}}$

where

r_ij is the relationship strength between merchant_i and merchant_j in a pair of merchants;

λ

is a log-likelihood ratio;

T_ij is the actual co-occurrence frequency of merchant_i and merchant_j in the transaction data; and

{circumflex over (T)}_ij is the expected co-occurrence frequency of merchant_i and merchant_j in the transaction data.

20. The method of claim 16, wherein updating at least one merchant vector so that the actual dot products between the at least one pair of merchant vectors approximates the desired dot product between the merchant vectors comprises a gradient descent update that updates the merchant vectors according to whether the actual dot product between them is greater or lesser than the desired dot product.

21. The method of claim 16, wherein updating at least one merchant vector so that the actual dot products between the at least one pair of merchant vectors approximates the desired dot product between the merchant vectors comprises determining for at least one merchant vector an error weighted average of the desired positions of the merchant vector from current position of at least one other merchant vector and the desired dot product between the merchant vector and at least one other merchant vector.

22. The method of claim 1, further comprising:

generating a consumer vector for each of at least a subset of the consumers;

generating a merchant vector for each of at least a subset of the merchants;

determining for at least one merchant name in the transaction data a merchant vector;

clustering the merchant vectors to form a plurality of merchant segments, wherein at least one merchant vector is associated with one and only one merchant segment; and

for at least one merchant segment, determining from the transactions of consumers at the associated merchants of the merchant, statistical measures of consumer transactions in the segment.

23. The method of claim 1, further comprising:

selecting a plurality of consumers associated with at least one merchant segment, the selected plurality selected according to their predicted spending in the merchant segment; and

providing promotional offers to the selected plurality of consumers.

24. The method of claim 1, further comprising:

training at least one of the merchant segment predictive models to predict spending in a predicted time period based upon transaction statistics of the consumer'"'"'s transactions in a past time period.

25. The method of claim 24, wherein the transaction statistics comprises variables describing the recency of the consumer'"'"'s transactions in one or more merchant segments, the frequency of the consumer'"'"'s transactions in one or more merchant segments, and the amount of the consumer'"'"'s transactions in one or more merchant segments.

26. A system for predicting financial behavior of consumers, comprising:

a database for storing a set of input transactions for a plurality of consumers with respect to a plurality of merchants;

at least one merchant segment, each merchant being associated with at least one of the defined merchant segments;

at least one merchant segment predictive model, for defining for a merchant segment a prediction function between input transactions in a past time interval and financial behavior in a subsequent time interval, to produce for each consumer a predicted behavior in each of at least a subset of the merchant segments.

27. The system of claim 26, wherein the predicted behavior comprises a likelihood of positive response to an offer.

28. The system of claim 26, wherein the predicted behavior comprises a spending level with respect to a merchant.

29. The system of claim 26, further comprising:

a merchant vector build module, coupled to the database, for generating a merchant vector for each of at least a subset of the merchants;

a consumer vector build module, coupled to the database, for generating a consumer vector for each of at least a subset of the consumers; and

a segmentation module, coupled to the merchant vector build module, for performing supervised segmentation on the merchant vectors.

30. The system of claim 29, wherein the at least one merchant segment predictive model applies a learning vector quantization algorithm to the merchant vectors.

31. The system of claim 26, wherein:

the merchant vector build module determines, for at least one merchant segment, a segment vector as a summary vector of merchant vectors of merchants associated with the segment; and

the consumer vector build module associates at least one consumer with the merchant segment having the greatest dot product between the segment vector of the segment and a consumer vector of the consumer.

32. A computer-readable medium comprising computer-readable code for predicting financial behavior of consumers, the computer-readable medium comprising:

computer-readable code adapted to obtain a set of input transactions for a plurality of consumers with respect to a plurality of merchants;

computer-readable code adapted to define at least one merchant segment, each merchant being associated with at least one of the defined merchant segments; and

computer-readable code adapted to, for at least one consumer, apply the input transactions of the consumer to each of at least one merchant segment predictive model, each merchant segment predictive model defining for a merchant segment a prediction function between input transactions in a past time interval and financial behavior in a subsequent time interval, to produce for each consumer a predicted behavior in each of at least a subset of the merchant segments.

33. The computer-readable medium of claim 32, wherein the predicted behavior comprises a likelihood of positive response to an offer.

34. The computer-readable medium of claim 32, wherein the predicted behavior comprises a spending level with respect to a merchant.

35. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to generate a consumer vector for each of at least a subset of the consumers;

computer-readable code adapted to generate a merchant vector for each of at least a subset of the merchants;

wherein the computer-readable code adapted to define at least one merchant segment comprises computer-readable code adapted to perform supervised segmentation on the merchant vectors.

36. The computer-readable medium of claim 35, wherein the computer-readable code adapted to perform supervised segmentation comprises computer-readable code adapted to apply a learning vector quantization algorithm to the merchant vectors.

37. The computer-readable medium of claim 35, wherein the computer-readable code adapted to performing supervised segmentation comprises:

computer-readable code adapted to initialize a set of segment vectors;

computer-readable code adapted to accept at least one segment label for at least one of the merchants; and

computer-readable code adapted to, for each of at least a subset of the labeled merchants;

select at least one segment vector for a merchant having a merchant vector;

determine whether the selected segment vector matches the segment label for the merchant; and

responsive to the determination, adjust zero or more of the segment vectors.

38. The computer-readable medium of claim 37, wherein the computer-readable code adapted to select at least one segment vector for a merchant comprises computer-readable code adapted to select a segment vector that is closest to the merchant vector corresponding to the merchant.

39. The computer-readable medium of claim 37, wherein the computer-readable code adapted to select at least one segment vector for a merchant comprises computer-readable code adapted to select at least one segment vector having a tolerance range that includes the value of the merchant vector.

40. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to train a predictive model using the predicted behavior of a plurality of consumers in at least a subset of the merchant segments, and additional observed behavior for the plurality of consumers with regard to a target segment not included in the subset of merchant segments; and

computer-readable code adapted to, for at least one target consumer;

provide as input to the trained predictive model predicted behavior of the target consumer in at least a subset of the merchant segments; and

obtain from the trained predictive model a predicted behavior of the target consumer with respect to the target segment.

41. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to, for at least one consumer, associate the consumer with the merchant segment for which the consumer had the highest predicting spending relative to other merchant segments.

42. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to generate a consumer vector for each of at least a subset of the consumers;

computer-readable code adapted to generate a merchant vector for each of at least a subset of the merchants;

computer-readable code adapted to, for at least one merchant segment, determine a segment vector as a summary vector of merchant vectors of merchants associated with the segment; and

computer-readable code adapted to, for at least one consumer, associate the consumer with the merchant segment having the greatest dot product between the segment vector of the segment and a consumer vector of the consumer.

43. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to, for at least one merchant segment;

rank the consumers by their predicted spending in the merchant segment; and

determine for at least one consumer a percentile ranking in the merchant segment; and

computer-readable code adapted to, for each consumer;

determine the merchant segment in which the consumer'"'"'s percentile ranking is the highest, to uniquely associate each consumer with one merchant segment; and

for at least one merchant segment, determine summary transaction statistics for the consumers uniquely associated with the merchant segment.

44. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to, for at least one merchant segment;

rank the consumers by their predicted spending in the merchant segment;

determine for at least one consumer a percentile ranking in the merchant segment;

select as a population, the consumers having a percentile ranking in excess of predetermined percentile threshold; and

determine summary transaction statistics for selected population of consumers.

45. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to establish for at least one merchant in the transaction data a merchant vector; and

computer-readable code adapted to update the merchant vector of at least one merchant relative to the merchant vectors of other merchants according to co-occurrences of each merchant in the transaction data.

46. The computer-readable medium of claim 45, further comprising:

computer-readable code adapted to update the merchant vector of at least one merchant based upon an unexpected amount deviation in a frequency of co-occurrence of the merchant with other merchants.

47. The computer-readable medium of claim 45, further comprising:

computer-readable code adapted to determine a co-occurrence frequency for at least one merchant with at least one other merchant in the transaction data;

computer-readable code adapted to determine for at least one pair of merchants, a relationship strength between the pair of merchants based on how much the determined co-occurrence frequency deviates from an expected co-occurrence frequency;

computer-readable code adapted to, for at least one pair of merchant vectors, map the relationship strength into a vector space as a desired dot product between respective merchant vectors the merchants in the pair; and

computer-readable code adapted to update at least one merchant vector so that the actual dot products between each pair of merchant vectors approximates the desired dot product between the merchant vectors.

48. The computer-readable medium of claim 47, wherein the computer-readable code adapted to determine for at least one pair of merchants a relationship strength between the pair of merchants further comprises:

computer-readable code adapted to determine the relationship strength by $r_{\mathrm{ij}}=\frac{T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}}{{\mathrm{\sigma <br><br>}}_{\mathrm{ij}}}\approx <br><br>\frac{T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}}{\sqrt{{\hat{T}}_{\mathrm{ij}}}}$

where

r_ij is the relationship strength between merchant_i and merchant_j in a pair of merchants;

T_ij is the actual co-occurrence frequency of merchant_i and merchant_j in the transaction data; and

{circumflex over (T)}_ij is the expected co-occurrence frequency of merchant_i and merchant_j in the transaction data.

49. The computer-readable medium of claim 47, wherein the computer-readable code adapted to determine for at least one pair of merchants a relationship strength between the pair of merchants further comprises:

computer-readable code adapted to determine the relationship strength by

r_ij=sign(T_ij−

{circumflex over (T)}_ij)·

√

{square root over (2ln λ

)}

where

r_ij is the relationship strength between merchant_i and merchant_j in a pair of merchants;

λ

is a log-likelihood ratio;

T_ij is the actual co-occurrence frequency of merchant_i and merchant_j in the transaction data; and

{circumflex over (T)}_ij is the expected co-occurrence frequency of merchant_i and merchant_j in the transaction data.

50. The computer-readable medium of claim 47, wherein the computer-readable code adapted to determine for at least one pair of merchants a relationship strength between the pair of merchants further comprises:

computer-readable code adapted to determine the relationship strength by $r_{\mathrm{ij}}=\mathrm{sign}<br><br>\left(T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}\right)·<br><br>\sqrt{\frac{2<br><br>\ln <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\lambda <br><br>}}{\sqrt{{\hat{T}}_{\mathrm{ij}}}}}=\mathrm{sign}<br><br>\left(T_{\mathrm{ij}}-{\hat{T}}_{\mathrm{ij}}\right)·<br><br>\sqrt{2<br><br>\ln <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\lambda <br><br>}}·<br><br>{\hat{T}}_{\mathrm{ij}}^{\frac{1}{4}}$

where

r_ij is the relationship strength between merchant_i and merchant_j in a pair of merchants;

λ

is a log-likelihood ratio;

T_ij is the actual co-occurrence frequency of merchant_i and merchant_j in the transaction data; and

{circumflex over (T)}_ij is the expected co-occurrence frequency of merchant_i and merchant_j in the transaction data.

51. The computer-readable medium of claim 47, wherein the computer-readable code adapted to update at least one merchant vector so that the actual dot products between the at least one pair of merchant vectors approximates the desired dot product between the merchant vectors comprises computer-readable code adapted to perform a gradient descent update that updates the merchant vectors according to whether the actual dot product between them is greater or lesser than the desired dot product.

52. The computer-readable medium of claim 47, wherein the computer-readable code adapted to update at least one merchant vector so that the actual dot products between the at least one pair of merchant vectors approximates the desired dot product between the merchant vectors comprises computer-readable code adapted to determine for at least one merchant vector an error weighted average of the desired positions of the merchant vector from current position of at least one other merchant vector and the desired dot product between the merchant vector and at least one other merchant vector.

53. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to generate a consumer vector for each of at least a subset of the consumers;

computer-readable code adapted to generate a merchant vector for each of at least a subset of the merchants;

computer-readable code adapted to determine for at least one merchant name in the transaction data a merchant vector;

computer-readable code adapted to cluster the merchant vectors to form a plurality of merchant segments, wherein at least one merchant vector is associated with one and only one merchant segment; and

computer-readable code adapted to, for at least one merchant segment, determine from the transactions of consumers at the associated merchants of the merchant, statistical measures of consumer transactions in the segment.

54. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to select a plurality of consumers associated with at least one merchant segment, the selected plurality selected according to their predicted spending in the merchant segment; and

computer-readable code adapted to provide promotional offers to the selected plurality of consumers.

55. The computer-readable medium of claim 32, further comprising:

computer-readable code adapted to train at least one of the merchant segment predictive models to predict spending in a predicted time period based upon transaction statistics of the consumer'"'"'s transactions in a past time period.

56. The computer-readable medium of claim 55, wherein the transaction statistics comprises variables describing the recency of the consumer'"'"'s transactions in one or more merchant segments, the frequency of the consumer'"'"'s transactions in one or more merchant segments, and the amount of the consumer'"'"'s transactions in one or more merchant segments.