What the FCA found when analyzing 1 million production changes

A recent FCA report shows that the financial services industry needs to reimagine its approach to change management. By analyzing data from over 1 million production changes, they found out what works and what doesn’t work in the land of regulated change. Let’s dig in…🕵️‍♀️

On the 5th of February the Financial Conduct Authority (FCA) published its Implementing Technology Change report. It focuses on the way financial firms manage technology changes and the impact of failures.

And something leaps right off the early pages.

While most financial institutions award themselves a mature rating for change management capabilities, change management failures are the most common cause of incidents reported to the FCA.

In other words, the industry thinks it has change management licked when the exact opposite is true. 🤯

What is happening?

The reason for this contradiction is the misplaced trust the industry has in Change Advisory Boards (CABs). Institutions throw a ton of time and money at these boards whose job it is to approve changes. Here’s what the FCA has to say about them:

“One of the key assurance controls firms used when implementing major changes was the Change Advisory Board (CAB). However, we found that CABs approved over 90% of the major changes they reviewed, and in some firms the CAB had not rejected a single change during 2019. This raises questions over the effectiveness of CABs as an assurance mechanism.”

When the regulator reaches for diplomatic phrasing like “raises questions” you can tell they’re a bit flustered. And that’s because CABs are an exercise in risk theatre designed to create the illusion of effective change management. It’s a powerful illusion because even the institutions believe in it.

This is an important finding: adherence to traditional change management processes doesn’t work to manage the risk of changes.

The science of DevOps backs this up. Here’s the unvarnished truth on external approvals and CABs based on research by Dr. Nicole Forsgren, Jez Humble, and Gene Kim in their 2018 book, Accelerate: Building and Scaling High Performing Technology Organizations.

“We found that external approvals were negatively correlated with lead time, deployment frequency, and restore time, and had no correlation with change fail rate. In short, approval by an external body (such as a change manager or CAB) simply doesn’t work to increase the stability of production systems, measured by the time to restore service and change fail rate. However, it certainly slows things down. It is, in fact, worse than having no change approval process at all.”

Worse than no change approval process at all.

So what does work then?

The FCA identified several practices that contributed to change success. Unsurprisingly, having well defined processes and a majority of the IT budget on delivering change will help.

But, in terms of actually delivering the software, they also found this:

“Frequent releases and agile delivery can help firms to reduce the likelihood and impact of change related incidents:

Overall, we found that firms that deployed smaller, more frequent releases had higher change success rates than those with longer release cycles. Firms that made effective use of agile delivery methodologies were also less likely to experience a change incident.”

The practices described here are the foundation of DevOps - frequent deployments, agile development, defined processes, and change as the normal way of working. So, if you’re already practicing DevOps, you’re well on the way to finding a better way to manage change. If you’re not - you should probably start. 😇

Winner winner chicken dinner

The thread that runs through all of Dr. Forsgren’s work is that DevOps maturity correlates very, very closely with business performance. If you’re going to corner your market and outpace the competition you need a strong DevOps game.

Some industry leaders have understood the importance of technological performance for quite a while. In 2014, Richard Fairbank, Capital One CEO, said “ultimately, the winners in banking will have the capabilities of a world class software company.” He might not have known it at the time, but he was describing DevOps organizations.

Being a mature DevOps outfit is especially effective in regulated verticals because you’ve got this annoying change management stuff that no one really wants to deal with. With DevOps you can automate your change and release control in the pipelines.

Imagine that. Compliant software on demand. And no CABs either. This is what the real winners in fintech and other regulated spaces will look like.

Now, can someone call a taxi for the CAB please? Peep peep peep! 🚕