The Models Are Out of Whack

SMITH BRAIN TRUST – Recent COVID-prompted Federal Reserve actions, fiscal policy responses, forbearances and foreclosures moratoriums have affected the reliability and stability of mortgage finance data models -- and the industry more broadly.

Those models had revolutionized the industry in terms of assessing risk to assist borrowers and to project interest rates, mortgage rates, house prices, unemployment rates, defaults and prepayments, and other key outputs that determine business success and viability.

But those models also are based on historical information and not equipped for today’s COVID-affected environment, says Professor of the Practice and Executive-in-Residence Clifford Rossi at the University of Maryland’s Robert H. Smith School of Business. “You need a balance of art and science to manage models.”

Rossi, who’s authored “Coronavirus is a problem data models cannot comprehend” for The Hill, made the above comments in a recent podcast discussion with Michael Bradley, Freddie Mac senior vice president of modeling, econometrics, data science and analytics.

“The models are only as good as the information that’s fed into them,” Rossi said, then asked: “Who’s seen a COVID-19 type of pandemic, coupled with the monumental amount of monetary and fiscal stimulus that’s been piled on to it since then [and] the consumer responses – or non-responses – from those kinds of actions? …We haven’t.”

“You can go back to the 1918 pandemic, so our data on that is pretty thin.”

Given the uncertainty, Rossi said companies should run a series of scenarios under a variety of assumed outcomes: “Like what happens if there’s an announcement of a weak vaccine—maybe an ineffective vaccine in Q1 or Q2 – or pushed out to Q3 – or a worst-case scenario like no vaccine. What impact would that have?” Under each scenario, he said, look at the economic risk. “And business impacts net of any stimulus would need to be factored in to see how it impacts things like loan loss reserving or forecasts.”

Ultimately, he said, these models “must be guided by good judgement and qualitative adjustment.”

Rossi, who noted his early-career work as a risk executive included a 10-year stint for both Fannie Mae and Freddie Mac “on the single-family side of risk,” put the current challenge in historical context: “Freddie Mac’s [Loan Prospector] revolutionized mortgage underwriting.  Then Fannie Mae came along with Desktop Underwriter [and Desktop Originator]. It was a major leap forward to applying what we more broadly refer to as artificial intelligence-type techniques – as crude as it may seem today -- in loan decisioning and what continues in the form of automated collateral evaluation processes.”

That’s the good side of automation, he added. The downside: “It can incent the wrong practices.”

Things can go wrong “as you move from full-documentation types of loans where you know the income is rock solid to the classic low- or no-documentation loan types… ”The data gets polluted and adversely affects the modelling. So, you have to be careful with that…”

Being careful, means for example, “to better address folks with nontraditional credit histories over the years,” Rossi said. “You have a lot of borrowers who for whatever reason have thin files, thin information. They don’t use credit very much or use no credit at all.” This is problematic, he says, for serving those consumers when you’re used to using credit scores in those automated underwriting models. “So, looking at rent payments, utility payments or mobile home payments can really help, along with other types of information.”

Listen to more from the discussion between Rossi and Bradley, via Freddie Mac’s “Makes MEDA (Modeling, Econometrics, Data Science and Analytics) Sense” podcast, from which Rossi leaves listeners with “a key takeaway”:  “Sophisticated analytics and data are here to stay. They continue to be harnessed for business and risk decisions and they are not going away. But there’s an art and science to it. I tell my graduate students all day long that I can turn you intogreat technicians. But to be a great analyst -- and using and harnessing the best of data models -- really comes with both the judgement and the technology…That’s the balance between art and science.”

Rossi also is slated to participate in forthcoming, Center for Financial Policy webinars on financial and mortgage risk:

Digital Mortgage Risk Summit – Chief Risk Officer Panel, 1 p.m. Wednesday, Oct. 14, 2020: Rossi joins chief risk officers of Fannie Mae, Freddie Mac, the Federal Housing Administration, Freedom Mortgage, and Bank of America’s Consumer Lending group for a wide-ranging conversation regarding the COVID impact on the mortgage industry, the evolving regulatory landscape, and challenges ahead in 2021 as the country continues to emerge from a pandemic-induced recession.

Scenario Analysis in the Age of COVID, 1 p.m. Friday, Oct. 16, 2020: Rossi and Sridhar "Subra" Subramanian, head of US Consumer Risk for BMO Harris Bank, discuss challenges associated with developing reasonable scenarios for stress testing bank portfolios during unprecedented times, and data issues and key assumptions that go into building robust scenario analysis.