Ask Me Anything Keynote: FinTech

FinTech experts Matthew Ellis and Chenxi Wang took questions during All Day DevOps last keynote of 2020. Matthew works at Macquarie, a global, digital-only financial services company without any branch network and Chenxi is a founder and general partner at Rain Capital, a cybersecurity firm for financial institutions. 

DevOps Exposure

Matthew talks about his experience working at a trading desk. He had the opportunity to sit next to his users, writing software. And when you’re sitting that close with your users, the feedback loop is really tight. Software struggles to keep up with users’ expectations. That was Matthew’s first exposure to DevOps and it’s potential. 

Feedback Loops

When talking about feedback loops, the question comes up: is it better for the customer to move fast and have quick feedback loops, or is it better to have slower, more deliberate feedback loops? 

Chenxi, with no hesitation, claims that fast feedback loops are much better. You need levels of testing and checking, but moving fast will help teams understand and stitch together security of their systems. The only way to understand a system is to have a mechanism to fail fast, learn, and roll back to a known state.

Matthew agrees. Everything is speeding up, but it’s not, as some believe, at the expense of quality. You just need good telemetry and good observability around your environment when you move fast. Otherwise, how will you notice when things go wrong? But the idea of speed and quality are not opposites. 

Chenxi brings up an example of a monolithic application that is old and full of vulnerabilities. Yet her company won’t touch the application because it’s producing maybe $8 million a minute. But if the team had a DevOps process and toolchain, they could extract modules from the app and make changes over time, learning and failing fast while still making its millions of dollars a minute.

Then and Now

It’s easier now than ever to build these feedback loops. It even looks silly these days to think about quarterly releases, where you’re likely to only get two out of four correct in a year. Chenxi saw where the security team could see real time attacks from their traffic and respond, patching it up in hours. This used to be an unknown thing, but now it’s becoming mainstream.

Thinking about the current state, Matt is looking beyond software to apply DevOps. Even the term FinTech still is siloed to technology and software. But there is value, for example, in sitting at call centers and getting feedback on what the customer support is dealing with. The operator on the phone is part of the whole value chain. 

Policy as Code

To adapt to the DevOps mindset, the role of developers in the organization has changed.

Instead of transcribing requirements, developers find value in expressing and deploying policies. This lets security teams test policies and manage them safely before they get deployed live. 

Knowing the impact of a policy before it gets deployed is so powerful. For example, Chenxi’s friend in Amazon SRE had security people keep coming and asking to install a certain script. But they were worried it could bring down a part of the system. With policy as code, that friend could stage it and test in a sandbox environment, looking at how the policy behaves.

To achieve policy as code, it’s better to build guardrails than gates. Where gates block progress, guardrails simply put limits on the impact on any given policy. Guardrails allow you to express guidance on how things should be. (But you must have some sparingly used gates for the rare moment a very critical error happens.) Even internal auditing is moving to guardrails. 

Gates are point-in-time, not continuous, but guardrails let you apply continuous policies. 

When you have controls as code, you’re not just releasing; you’re collecting evidence. It makes it very visible to auditors. There are just a few places they can look to review security and compliance. 

FinTech with COVID

The global pandemic has changed the way we are doing work, including in FinTech. For Macquarie, it didn’t change much of their goal. They already had a five-year plan to migrate completely to cloud services. They even now have a vibrant, multi-cloud ecosystem. 

Many of the impacts Matthew has seen have been personal to individuals. Some employees have kids who may try to kick the door down to get to their working parent. And there are other challenges. Macquarie already had flexible work scheduling so they were set up to tackle the issues of working from home.

The pandemic has had various mental impacts on software workers. Macquarie leaders ensure they have a pulse on the morale of their employees, checking in regularly. Chenxi points out that the pandemic has forced most companies to support working from home, and that has given employees more choices. It has even brought up the question of “Why do I only work for one company?” Maybe in the future, people center around missions: for mission A they may work with company X that day. For mission B they may work with company Y. 

Speed vs. Quality vs. Security

When asked what’s more important—speed, quality, or security—Matthew responds that there is not as much of an opposition among these three things as many people think. However, speed comes first. It is better to get that fast feedback loop so that you can learn and have quality come up from behind. 

For Chenxi this is a very tough question. Though she may be expected, as a security professional, to choose security, she doesn’t think it’s the most important. After all, if there’s no product to secure, security has no value. So she puts speed first, then security, hoping that quality comes with it as an artifact of making things secure. 

DevOps in FinTech 

The experts in FinTech show that it is like many other software organizations. It is paramount to have speed. Fast feedback loops are key to failing fast, and thus key to learning your system. This feeds into security and quality by letting developers roll back and improve the software. 

An important element for bringing up the quality in these feedback loops is policy as code. It’s powerful to be able to test policy changes before they get deployed. 

Finally, FinTech was affected by COVID in a way that’s similar to other orgs: FinTech organizations were forced to be remote-first. This has given employees more choice in how they work. 

This session was summarized by Mark Henke. Mark has spent over 10 years architecting systems that talk to other systems, doing DevOps before it was cool, and matching software to its business function. Every developer is a leader of something on their team, and he wants to help them see that.