Being Budget Conscious in a Continuously Automated World

One way to bring more business alignment with IT in DevOps is through continuous verification with things like security and budgets. Oftentimes these things are an afterthought until the last minute, but we can bring them into the pipeline and continuously think about them as we deliver software. Tim Davis (@vtimd), cloud advocate, shares his thoughts.

What Is Continuous Verification?

Continuous verification is the weaving of budget and other concerns directly into our DevOps pipeline.

AKS cluster questions from Tim Davis’s “Being Budget Conscious in a Continuously Automated World” presentation.

In the public cloud, you can quickly find yourself spending more money than you thought you were. It’s so easy to purchase new services that overspending can sneak up on you. Traditionally, this can snake up through the finance department, resulting in you getting flooded with a slew of angry emails.

Instead, we can keep an eye on things, ensuring we’re consistently within budget. There are many tools that can help us do this. CloudHealth from VMWare is great for cost. Tools like Clair and SecureState help us with security and compliance.

Why Should it be CI/CD/CV?

It’s extremely important to be doing these verifications inside your pipeline because it keeps it on the top of your mind. At all times, you’re preventing these issues from becoming large ones.

Demo of Continuous Verification

There is a great demo of continuous verification that you can find at https://gitlab.com/vtimd/addo-script

You can see, through GitLab, both a budget and security check. Later in the pipeline, you can see a traffic check to ensure performance is stable. In the demo, you can see that the Wavefront performance check failed.

Failure of WavefrontCheck from Tim Davis’s “Being Budget Conscious in a Continuously Automated World” presentation.

The great part here is that this pipeline lets us fail faster. If we run out of budget or don’t meet performance needs, we cancel that pipeline. We don’t continue to push and overspend.

In the demo, the entire pipeline is configured through one file. It is easy to turn off and on parts of the pipeline to speed it up and understand how it works.

The demo uses a ch-gitlab-script.py in Python that does most of the heavy lifting. The deployment config uses much for this script’s functions to run each step in the pipeline.

ch-gitlab-script.py from Tim Davis’s “Being Budget Conscious in a Continuously Automated World” presentation.

The scripts that run are neither VMWare- nor GitLab-proprietary. Anyone can do such scripting. Note also how things like “Bill Tokens” and other variables are parameterized. We want to inject these into the scripts and not hardcode them, as we want to be able to run this across environments and keep sensitive information secure.

Summary

To summarize, doing continual verification is not technically hard. You don’t have to familiarize yourself with large amount of new tools, either. You can do this with many of the tools you’re already familiar with. But the value you get from continually verifying budget, security, and performance is immeasurable.

This post was written by Mark Henke. Mark has spent over 10 years architecting systems that talk to other systems, doing DevOps before it was cool, and matching software to its business function. Every developer is a leader of something on their team, and he wants to help them see that.

Photo by Franck V.