Document View

Risk-mitigation decisions based on a combination of fraud detection and performance data analysis hold greater promise for the mortgage industry than fraud detection alone. The future of the industry is in performance data and evaluating the performance of a fraud system against the hundreds of thousands of loan files run through it. Only this will allow a fraud-detection tool to be tuned so that it can accurately predict the performance and mitigate loss during the origination process. Performance data will allow the tool to provide customers with stratification to our scoring range. To provide customers with the most robust set of analytics, it will be necessary to utilize various types of statistical analysis. But an enterprise risk-mitigation solution must encompass more than just performance data and analysis. By using heuristics in combination with performance data, it is possible to continually tune statistical fraud-detection methods to ensure the methods keep pace with ever-changing market conditions and fraud patterns.

Enlarge 200% Enlarge 400%

"Loan No. 16523 contains questionable information in 4 categories, has a score of 223 and an 85 percent to 95 percent chance of not performing." * Sound like a reading from a crystal ball? Welcome to the very real future of risk mitigation. * Risk-mitigation decisions based on a combination of fraud detection and performance data analysis hold greater promise for the mortgage industry than fraud detection alone. It's one thing to detect mortgage fraud after it has occurred. It's quite another to detect possible fraudulent activity prefunding based on commonalities and inconsistencies that should not exist. It's quite another matter still to be able to predict the likelihood of losses based on the addition of performance data. * In other words, if there were a way to accurately predict the level of losses that would occur if certain loans were funded, wouldn't it make sense to use that tool on 100 percent of volume to truly protect company assets? * We at Interthinx believe the future of the industry is in performance data and evaluating the performance of a fraud system against the hundreds of thousands of loan files run through it. Only this will allow a fraud-detection tool to be tuned so that it can accurately predict the performance and mitigate loss during the origination process.

The advantages are clear. Performance data will allow the tool to provide customers with stratification to our scoring range. For example, we will be able to use our score to assign a probability of fraud and provide a revolutionary decision tool for the industry. Then, based on the lender's cost per fraudulent loan, it will be possible to calculate the savings that can be realized by leveraging closed-loop data.

The analysis of historical performance data and real-time application data will provide the ability to customize risk-mitigation tools. By examining how the score performs and factoring in the lender's risk tolerance, performance data enable solutions to be customized based on each lender's profile. Further, pre-funding workflow can be streamlined by providing seasoned employees the more difficult loan files that have been separated not only by score but by predictive performance analytics.

To provide customers with the most robust set of analytics, it will be necessary to utilize various types of statistical analysis. These include, but are not limited to, the following:

* Survival analysis: This is an area of statistical modeling that predicts how long something (e.g., a loan) will survive or perform. This can answer questions such as how likely is it that a loan with X characteristics will perform for at least six months.

* Competing risks: This is part of survival analysis, in which our models account for the fact that a loan might face several different sources of risk at once (e.g., fraud risk, bankruptcy risk, early payoff risk).

* Measurement of detection rate and false-alarm rate: This gets at the fundamental yes/no decision of whether a loan should be funded, by allowing lenders to quantitatively balance the cost of funding a bad loan against the cost of rejecting a good loan.

* Power analysis: In statistical terms, power means the probability that you would be able to detect a particular effect using a certain sample size.

* Logistic-regression models: This is a generalized form of linear regression that is appropriate for use when the outcome is a yes/no variable. For example, we could use logistic-regres sion models to examine how multiple loan characteristics simultaneously relate to the probability of being rated as "investigate" by our tools.

But an enterprise risk-mitigation solution must encompass more than just performance data and analysis. By using heuristics (rules created and used by expert human beings) in combination with performance data, it is possible to continually tune statistical fraud-detection methods to ensure the methods keep pace with ever-changing market conditions and fraud patterns. As a result, the system is aligned in real-time with new fraud schemes and "hot" fraud areas.

But let's first take a step back and look at the logical argument leading to the application of performance data analysis for effective risk mitigation.

Early-payment default statistics are not enough

Some providers use early-payment defaults (EPDs) as the sole sign of fraud, and then sell tools that are marketed as "fraud tools" to predict EPDs. As it turns out, some loss from fraud is felt in early-payment defaults. But since EPDs can be caused by other reasons, they should not be mistaken as a sure sign of fraud.

For example, EPDs could be the result of credit risk, life event or property calamity. Using EPDs as a fraud proxy not only overcounts some types of nonfraud losses, it undercounts many true fraudbased losses.

Numbers alone don't tell the story

Whether they are called neural networks or something else, any fraud tool that relies exclusively on statistical methods is just another one-trick pony. The argument for these methods is "from raw data, losses can be inferred, without any other technique." But fraud detection is simply more complicated than this. Without heuristics, numbers are just . . . numbers. Heuristics are rules of thumb based on human experience that can tell you what the data actually mean.

Drawing on my experience in military intelligence, the analogy that comes to mind is from the Cold War era and involves the use of artificial intelligence (AI) techniques to determine whether the Soviet Union was going to attack through the Fulda Gap in Germany.

The Soviets used to run military exercises in East Germany. Whenever they ran the exercises, the North Atlantic Treaty Organization (NATO) used to watch and listen to everything. As a result, a great deal of raw data were accumulated. Like fraudsters inflating creditworthiness by adding good credit lines (credit boosting), the Soviets would try to camouflage what they were doing by covering tanks and reducing radar.

Was that the sound of 25 tanks rolling toward Berlin? Do those radar signatures indicate the presence of Soviet fighter planes? Are those missiles anti-aircraft weapons or short-range ballistic (nuclear) missiles?

Just as with fraud detection, the raw data were not enough to determine exactly what was going on. NATO would try to determine what the Soviets were doing, along with the risk of attack. To do that, the team at NATO would interpret the risk it represented.

As might be imagined, surveillance photos and radar are not very helpful without the proper tools to assess risk. Our military team had thousands of pictures and lots of radar. Analyzing statistics wouldn't have helped to determine the likelihood of an attack. At best, if we detected additional activity, we might have suspected the Soviets were running military exercises-but it would not have predicted losses.

To help make useful predictions, we built systems to aggregate the data and create a meaningful picture of what was happening. If many tanks were moving west, aircraft was being loaded with weapons, and radar-jamming equipment was redirected to point west all at the same time, then the system would alert us about a problem.

If, on the other hand, tanks were moving back toward Moscow, the system would conclude that everything was OK. Having such a system in place was crucial, because NATO did not want to go to war if the Soviets were simply repairing equipment.

Similarly, in loan fraud, simple statistics are not enough. Property values in the neighborhood and loan-to-value (LTV) ratios are just data. They do not indicate whether there is going to be a fraud loss on a particular property.

What is needed are heuristics or the type of reasoning a risk manager, underwriter, quality-assurance (QA) analyst or fraud investigator would do. Heuristics allow a system to organize vast amounts of data into patterns of meaningful information.

From these meaningful patterns, statistics can be applied to determine the likelihood that fraudulent behavior has occurred. It is not enough to just have data coming in. The application of statistics to results is key to truly understanding the problem at hand.

Heuristics also tell you what to do. The heuristic, or rule developed by military experts, is to check for patterns of activity to determine possible risk. With fraud detection, the heuristics will tell you if an application is suspect or not and-perhaps as important-why.

A system that does not apply heuristics cannot reason about the underlying conditions. Both heuristics and statistics are vital to predicting actual loss.

Lastly, heuristics have another value: They can explain why a decision was reached.

Artificial intelligence, just like the natural kind, is not always correct. A score or a statistical summary cannot explain why a specific recommendation was made. Heuristics give you the power to understand why the decision was reached, and therefore you can intelligently override it. For example, a borrower may claim on a refinance that a property is owner-occupied. Heuristics would know to compare the property address to several data sources, such as a Social Security number trace and title records. A discrepancy would indicate there may be fraud. Other heuristics could be used to validate the occupancy from other data sources such as reverse directory and reverse address searches. The heuristics would be drawn together and presented to the user to validate or bring into question the statement of owner occupancy.

Re-inventing risk assessment with performance data

So, combining heuristics and statistics is a great start, but it still leaves something out. Unless you use real-world data, it is nearly impossible to know if each heuristic is correct and each statistic is useful.

Fine-tuning a fraud system is like the military experts adding new activity patterns and weapons to the system to determine the likelihood of an attack.

When lenders suffer losses, they often do not know why the loss occurred. One reason for this is they have been unable to test their system statistics with any degree of accuracy. Without access to large pools of actual loan performance data, the statistics are simply not valid.

However, access to large volumes of performance data is not the only issue. Data quality, which is difficult to accurately measure, is a crucial element in the success of large-scale, closed-loop feedback. This is an issue because a lender's assessment of loan performance often presents a biased picture. A lender may not know all the loans that were indeed fraudulent and may not have all of the relevant information about what problems have occurred.

Performance data from lenders may be based on unknown biases. The lender's view of fraud usually comes from two sources-internal QC and loans repurchased from investors. Neither source provides random selections, because QC groups typically focus their investigations on known fraud and loan repurchases. Such investigations only happen after a loan goes bad. Because both of these two feedback mechanisms tend to be subjective or inconsistent, loan performance data built from this information will not be reliable.

For example, if 10 percent of loans have some kind of problem with fraud, but a lender uses performance data that show only 2 percent with problems, any fraud-detection tools based on the 2 percent sample will underestimate the true level of fraud and fail to detect about 8 percent of loans that will eventually exhibit problems.

When that 8 percent of the loan pool comes back for repurchase, the underestimate becomes a direct hit to the bottom line. Of course, it is also possible to overestimate. If a lender has a sample showing 20 percent of loans with a particular failing, but in reality only 10 percent of the whole population suffers from that problem, the overestimate will cause the lender to deny good loans and lose revenue.

Innovation and the right performance data

Instead of using small samples of performance data that may or may not be biased, Interthinx has created a new model based on the loan performance data from 100 percent of the hundreds of thousands of loan applications evaluated monthly through our system and then funded. The advantages are enormous, because they allow us to determine how effective each rule of thumb is and the accuracy of each set of statistics.

Fraud is complex, as most lenders know, with hundreds of variations on every type of fraud scheme. Smart fraudsters keep trying fraud schemes until they stop working. As soon as a vendor picks up a fraud scheme, the fraudsters adapt and change until they find one that is not being detected currently by fraud-prevention vendors.

Using a limited feedback loop of performance data, a vendor cannot quickly respond to new fraud schemes because it may take months or even years before these schemes show up in the performance data.

What is the best approach for comparing performance data with the results of the fraud-detection system? First, it is important to look at the heuristics in relation to each loan's performance. By determining which heuristics predicted a loan would suffer a loss, which heuristics predicted it would not, and how the heuristic rules interact and work together, it is possible to see how well the heuristics were used to predict loan performance.

Next, scores and other summary measures should be validated against actual loan performance. These steps should be taken for all loans-the ones that actually had losses and the ones that did not.

By comparing both bad and good loans, it is possible to create a system that simultaneously does two things: 1) catches the bad loans (high detection rate), and 2) labels the good loans as good (low false-alarm rate).

The two fundamental measures of a prediction system are the detection rate and the false-alarm rate. The detection rate is the percentage of bad loans that are caught. The false-alarm rate is the percentage of good loans that are incorrectly labeled as bad. Both of these numbers are important; bad loans cause losses directly, while false alarms are missed opportunities to make good loans.

Last word

Proceed carefully when selecting and integrating a companywide risk-mitigation solution. Fraud systems that only look at statistical data are fundamentally flawed. They may see certain trends, but those trends tend to be inaccurate in finding fraud. Further, since these systems look only at data and do not apply intelligent methods for aggregating data (such as heuristics), they miss trends within a single loan file or among groups of fraudulent loans.

The use of performance data to validate statistics is the future of fraud detection. By using heuristics in concert with closedloop data, we can fine-tune our statistical tactics to ensure that our methods stay in tempo with fraud patterns and volatile market conditions. Ultimately, the system becomes aligned with new fraud schemes and hot fraud areas in real-time.

While risk factors may tell you the buyer's ability to pay or willingness to pay, only a fraud model that uses real performance data can tell you other critical things. These things include whether the buyer is who he or she says, whether a property is worth the appraisal price, whether the third parties are part of a known fraud ring, and whether actual loan-payment history supports the application data.

To go beyond simple mortgage fraud detection and positively impact the bottom line, innovative new approaches are required that take risk mitigation from detection to prediction and, ultimately, to prevention. That is the promise of using a system that incorporates real-time performance data with statistics and heuristics. We believe it is the future of risk mitigation and the future of our industry.