As more enterprises embrace DevOps, organizational disconnects often are created between what controls DevOps teams have in place and what IT controls auditors believe need to be in place. And if the right controls are, in fact, in place they absolutely need to be communicated to the auditing teams. Bridging these gaps is often painful for both IT teams and the auditors.
Helping enterprises to better understand what auditors need from them to do their work is the top-line goal of the DevOps Audit Defense Toolkit, the aim of which is to “define the authoritative guidance of how management and auditors should conduct audits where DevOps practices are in place,” according to the homepage for the effort. “By doing this, the DevOps Audit Defense Toolkit will elevate the state of the management practice, defining how to understand risks to business objectives, correctly scope and substantiate the of effectiveness of controls, which reduces the costs of audits and increases effectiveness of audits.”
The review draft of the DevOps Audit Defense Toolkit was release earlier this month for review and comment. The authors, James DeLuccia, Jeff Gallimore, Gene Kim, and Byron Miller, expect the final version to be available in coming months.
To get a deeper understanding of the Toolkit, we grabbed a few minutes of time with Gene Kim, author of The Phoenix Project.
Why do you think something like the DevOps Audit Defense Toolkit is necessary?
I think at its heart DevOps is transformative. It’s the way to increase the flow of work through the DevOps value stream. DevOps increases reliability, stability, and security. And it also helps the organization win in the marketplace.
Yet, when organizations actually embark upon this journey, probably the number one obstacle they encounter is that their “compliance guys will never let us do this.”
I think there’s some truth to this. Because of so much that’s being proposed when you embark upon the DevOps journey, you actually take away some of the key controls that traditional IT organizations have used – like separation of duty, like change approval processes.
So when they hear, “Hey, I’m going to be letting our developers deploys whenever they want, without necessarily getting a change approval order from the change advisory board,” auditors, compliance managers, and security people often freak out.
And you have first-hand experience of seeing this in action, right?
It’s why I think it’s a very important effort. The more I talk to people, the more I hear about the friction between the audit teams and audit groups in organizations trying to move this way.
How do you see the DevOps Audit Defense Toolkit achieving this?
The DevOps Audit Defense Toolkit was designed to train. Since we can’t bring DevOps to the auditors, we must bring auditors to DevOps. What I mean is the goal is to train practitioners in DevOps how to think like an auditor and actually walk through what an auditor will do when examining a process and systems. That’s starting from the very highest level of the top-down risk assessment into the details, just as an auditor would go through the list asking what can go wrong.
The DevOps Audit Defense Toolkit shows how an enterprise has the environment that can prevent bad things from happening, and if we can’t prevent it, we can detect and correct for it. Then, it shows how we evidenced it. I think by doing all of that, one is doing exactly what the best auditors who examine DevOps organizations do themselves.
By training people in in the DevOps organization to think like auditors, we can actually bring the auditors along, show our work, connect the dots, and hopefully have the auditor become DevOps’ best friend along the way.
So, this is training IT how to think like an auditor and view the systems and processes the way that auditor would?
In many ways, yes. This is kind of a structured thinking process that any world-class auditor would do to actually create an audit plan and then audit fieldwork.
The toolkit asks questions like: Why is this being audited? Describe the environment Walk through the process of how code moves to production, and where the development environment comes from. It also asks: How do you prevent developers from introducing backdoor code into production; how do we ensure that untended code doesn’t make it into production?
I’ve learned a lot just working with the other team members, in terms of showing how we have a basis to say that we know what we’re doing, and that we’re really thinking through how to have an effective control environment.
That takes it from sort of the high-level platitudes and claims down to well, no, really, this is how we enforce peer reviews of code. We ensure that all deploys are put into the build systems. We run all the automated tests. Every time someone commits code, we run all the security tests before it can ever get into the staging environment. We can confirm that what’s deployed actually passed all the tests.
We communicate most effectively when we write down and show our work. And we’re exposing all that is required for an auditor to be able to look at it and say, “You know what? This is actually better than my best auditors would’ve done in the field, and you know what you’re doing.”
I would imagine that simply learning the language of auditors and how they speak is very valuable itself.
Yes, we’re now using a language that auditors use among themselves. Even just exposing what that language looks like, I think, is helpful to connect our own dots to be able to say, “Oh, I know why it’s being phrased this way. Tons of value, just by providing that.
Did you learn anything along the way as you created the DevOps Audit Defense Toolkit that surprised you?
It’s funny how much I learned by working on this. You would think the person who co-wrote the Phoenix Project would know what this would look like? But I learned quite a bit when I was looking at what were the actual business flows, and where that regulated data could reside, and what specifically are the applications in-scope for the compliance domains.
The second thing was going through the management risk analysis – to walk through the structured thinking process and look at how a company is really doing the diligence that is required, deciding what are its top business risks and if it really has a control environment that can prevent, detect, and correct.
Of course, we all know that DevOps makes everything better, but we need to know why and how to prove it, By what standards are you holding developers accountable? You talk about peer reviewing, but where is that documented? What is the standard? How is it enforced?
You want to be in compliance, and that’s important. But you also don’t want people to insert backdoors. You don’t want systems to crash when changes are deployed. So you really need to name all the dots, connect them, and then have that processed challenged. I have learned so much in this project.
