METHOD AND APPARATUS FOR TESTING AUTOMATED VALUATION MODELS

• US 20120011075A1
• Filed: 07/13/2011
• Published: 01/12/2012
• Est. Priority Date: 12/08/2004
• Status: Active Grant

• Current Assignee: Corelogic Solutions LLC (CoreLogic Incorporated)
• Original Assignee: CoreLogic Information Solutions Incorporated (CoreLogic Incorporated)
• Inventors: Graboske, Benjamin, Walker, Robert L.
• Granted Patent: US 8,370,239 B2
• Time in Patent Office: Days
• US Class Current: 705/306
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/0278 Product appraisal
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 40/12 Accounting
• CPC Class Codes: G06Q 50/16 Real estate

Abstract

A method and apparatus for real time testing of automated valuation using various indicators of accuracy. These indicators are then weighted according to their value as indicators of accuracy using individualized weighting factors or an equation. A ranking is then computed based upon the factors and their weights. This method is preformed continuously, so as to achieve real-time or periodically updated automated valuation model accuracy rankings.

Claims

1-37. -37. (canceled)

38. A system for determining the accuracy of automated valuation models (AVMs) that provide automated valuations for real estate properties, the system comprising:

physical data storage configured to store (1) a reference value for a subject property and (2) a plurality of automated valuations of the subject property, each automated valuation of the subject property generated by an automated valuation model (AVM) without use of the reference value for the subject property, wherein different automated valuations correspond to different AVMs; and

a computer system in communication with the physical data storage, the computer system comprising computer hardware, the computer system programmed to;

calculate one or more indicators of accuracy for each AVM, the indicators of accuracy for each AVM based at least in part on the AVM'"'"'s automated valuation of the subject property and the reference value of the subject property; and

determine a ranking of the plurality of AVMs based at least in part on the one or more indicators of accuracy.

39. The system of claim 38, wherein the reference value for the subject property comprises at least one of:

a sale value for the subject property, an appraisal for the subject property, an AVM valuation of the subject property, and a foreclosure value for the subject property.

40. The system of claim 38, wherein an AVM generates the automated valuation of the subject property before being updated with the reference value for the subject property.

41. The system of claim 38, wherein at least one of the indicators of accuracy comprises an error value calculated based at least in part on the automated valuation of the subject property by an AVM and the reference value of the subject property.

42. The system of claim 41, wherein the error value comprises at least one of:

an absolute mean error, a median error, and a mean squared error.

43. The system of claim 38, wherein at least one of the indicators of accuracy is an AVM accuracy indicator based at least in part on the number of automated valuations of the subject property that are within a percentage of the reference value for the subject property.

44. The system of claim 38, wherein at least one of the indicators of accuracy is an outlier indicator based at least in part on the number of automated valuations of the subject property that are outside a percentage of the reference value for the subject property.

45. The system of claim 38, wherein at least one of the indicators of accuracy is a hit rate indicator based at least in part on the number of real estate properties for which the AVM is able to return an automated valuation.

46. The system of claim 38, wherein the computer system is programmed to:

determine an accuracy score for each of the plurality of AVMs based at least in part on the one or more indicators of accuracy for each AVM; and

determine the ranking of the plurality of AVMs based at least in part on the accuracy scores.

47. The system of claim 46, wherein the computer system is programmed to determine the accuracy score by weighting at least some of the indicators of accuracy.

48. The system of claim 38, wherein the computer system is programmed to determine the ranking of the plurality of AVMs based at least in part on costs to request the automated valuations from the plurality of AVMs.

49. The system of claim 38, wherein the computer system is programmed to determine the ranking of the plurality of AVMs in one or more categories for the real estate properties.

50. The system of claim 49, wherein the one or more categories comprise at least one of:

an economic tier, a property type, and a geographic area.

51. A system for determining the accuracy of automated valuation models that provide automated valuations for real estate properties, the system comprising:

physical data storage configured to store reference values for real estate properties and automated valuations of the real estate properties; and

a computer system in communication with the physical data storage, the computer system comprising computer hardware, the computer system programmed to;

determine first accuracy rankings for a first plurality of automated valuation models (AVMs), the first accuracy rankings based at least in part on the reference values for the real estate properties and automated valuations provided by the first plurality of automated valuation models for at least some of the real estate properties;

determine a primary automated valuation model (primary AVM) based at least in part on the first accuracy rankings;

determine second accuracy rankings for a second plurality of AVMs, the second plurality of AVMs including at least some of the first plurality of AVMs but not including the primary AVM, the second accuracy rankings based at least in part on the reference values of the real estate properties and automated valuations provided by the second plurality of AVMs for real estate properties for which the primary AVM was not able to return an automated valuation; and

determine a secondary automated valuation model (secondary AVM) based at least in part on the second accuracy rankings.

52. The system of claim 51, wherein at least one of the automated valuations of a real estate property provided by the first plurality of AVMs or by the second plurality of AVMs is generated without use of the reference value of the real estate property.

53. The system of claim 51, wherein the second plurality of AVMs includes the first plurality of AVMs except for the primary AVM.

54. The system of claim 51, wherein the primary AVM and the secondary AVM are determined for at least one of:

a geographic area, an economic tier, and a property type for the real estate properties.

55. A system for determining the accuracy of automated valuation models (AVMs) that provide automated valuations for real estate properties, the system comprising:

physical data storage configured to store at least one indicator of accuracy for each of a plurality of automated valuation models (AVMs), the at least one indicator of accuracy for an AVM based at least in part on (1) a reference value for a subject property and (2) a valuation generated by the AVM for the subject property, wherein the valuation is generated without use of the reference value for the subject property; and

a computer system in communication with the physical data storage, the computer system comprising computer hardware, the computer system programmed to;

determine a ranking of the plurality of AVMs based at least in part on the indicators of accuracy for the AVMs.

56. The system of claim 55, wherein at least one of the indicators of accuracy comprises an error value calculated based at least in part on the automated valuation of the subject property by an AVM and the reference value of the subject property.

57. The system of claim 56, wherein the error value comprises at least one of:

an absolute mean error, a median error, and a mean squared error.

58. The system of claim 55, wherein the computer system is programmed to:

determine an accuracy score for each of the plurality of AVMs based at least in part on the at least one indicator of accuracy for each AVM; and

determine the ranking of the plurality of AVMs based at least in part on the accuracy scores.

59. The system of claim 58, wherein the computer system is programmed to determine the accuracy score by weighting at least some of the indicators of accuracy.

60. The system of claim 55, wherein the computer system is programmed to:

determine a primary AVM based on the ranking of the plurality of AVMs; and

determine a secondary AVM based at least in part on indicators of accuracy for real estate properties for which the primary AVM was not able to return an automated valuation.