devopsy.kinsta.cloud

Author: Reuven Harrison

Security and Speed: Why DevOps and Security Need to Play Nicely

It isn’t news that DevOps and IT security teams often struggle to align their departments and maintain a coherent balance between keeping a business secure and developing new applications to maintain customer interest. While security processes are a necessity, they can be deemed by DevOps teams to be manual and cumbersome, blocking the agility that makes them so effective in bringing their solutions to market. IT teams conversely feel their counterparts are prepared to sacrifice security in the name of innovation and revenue.

Even if both teams do respect the other’s intentions, any conflict could lead to delays in both of their processes. For example, an IT team may need to make crucial updates to the network security and warn different teams they may experience some downtime during this crucial implementation. However, DevOps have typically been given more leeway in how they operate as they are so important in today’s software-driven world, and may ask for the update to be delayed so they can complete tasks or meet deadlines, leaving the IT team waiting and losing time rescheduling their own work.

This has, unfortunately, led to a myth that DevOps teams choose to ignore security. In reality, developers are keen to know that their apps and the environment they work in are secure—but at the same time, they don’t want security to get in the way of them quickly delivering valuable new products and software features.

So, is there a way for DevOps teams—one of the most important resources in many modern businesses—to embrace security without impacting agility? Can the integration of DevOps and security be done in a way that alleviates tensions and promotes collaboration, while actually improving both security and agility in the process?

Yes. The secret is automation.

Reconciliation Through Automation

As C-suite executives are now more likely to focus on security, due to the obvious financial and reputational consequences of a breach, DevOps teams should define how they protect and secure their multiple projects and production environments. Automating security as part of the CI/CD process allows DevOps teams to easily follow company security policies because they will be embedded into the automation pipeline.

This process can remain running with little concern, effectively minimizing stress about security. This still automates policy changes and activities so that there is a significantly reduced chance of error. Although the automation solution remains hidden, it can still be utilized at any point to view data on the vulnerabilities, compliance requirements, security policies and network connectivity, via its continuous scanning abilities.

Additionally, DevOps teams are already familiar with automated tools in their daily operations and communications—and they are likely to be accepting of switching to a security solution that integrates with their existing processes.

Automation is the key to creating reliable, effective and connected “DevSecOps” teams, as it makes the secure option the easy option. It combines DevOps’ existing use of automated tools to achieve their ultimate goal of continuous, on-time and on-budget deployments with security’s focus of reducing human error and maintaining continuous visibility into potential vulnerabilities.

Encouraging Adoption

A guiding principle of DevOps is collaboration, which is often equated with the idea of shared responsibility. To successfully embed security into the DevOps process, security teams and developers must work together and establish shared responsibility. But how?

Some organizations may assign a security representative in each development team. This person acts as a pivotal link between the two teams, improving communication and building a balanced process that considers everyone’s mutual interests. A continuous flow of knowledge-sharing among both teams ensures a level of maturity that allows a business to secure applications and services with an automated solution.

Security teams can begin to define “guardrail policies” that allow development teams to deploy continuously, with the caveat of having to obey security and compliance policies. This is critical for both teams. This new way of working means developers will be able to test their security posture at every step in the CI/CD pipeline and correct things when necessary, and security teams can comprehensively ensure security and compliance throughout the development process.

Embracing Collaboration

Any belief that there is common discord between DevOps and IT security teams is unfounded. While it cannot be denied that both teams affect each other, this is not due to conflict—it’s due to business needs. If the two teams work together, they can both achieve their goals and be part of a secure, innovative and profitable organization. The first step is to accept collaboration is a necessity and by embracing security instead of being concerned by it, DevOps teams can stay in control of how their needs work around IT teams’ processes. Then, an automated security solution can be deployed to improve the efficiency and outcomes of both departments—and, in turn, the entire organization. It’s time for DevOps to embrace DevSecOps.

— Reuven Harrison

January 4, 2019
Automation domination (for security automation it’s a path)

Let’s face it, by nature security professionals are not the trusting type – in the least bit whatsoever – to put it bluntly. The concept of automating security makes them jumpy, testy and downright irritable in some cases.

Why?

Well, as we know, security teams are charged with maintaining control over processes. Introducing automation requires trust, a lot of it in fact. But as automation IS the future of IT, like it or not, how can security pros learn to love automation domination?

Let’s tackle the elephant in the room first. When it comes to getting security teams on board with automation, the stopgap tends to be around trust. The security culture is one of control (yes, we’re a bunch of control freaks) and by nature security professionals don’t trust people and they don’t trust systems.

The sound of the word Automation makes some security people jump out of their seat, but it’s not only because of security reasons. The biggest concern is actually, causing an outage, an event that tends to be noticeable much faster than an overly permissive security configuration which may go unnoticed forever.

For automation to take place, there has to be a hand off around security decisions, a loss of control so to speak, and in order to do that there has to be unfounded trust in the process about to be automated.

Where to Start: Today, automation in IT is predominant; sys admins use automation for application deployments as well as managing infrastructure on premise and in the cloud. It is increasingly becoming the norm. However when you move from IT into security automation, that is where the norm ends. So if automation of security processes is new to your team or organization, where might be a good place to start?

First, security teams are open to automating tasks that are not inline. Compliance audits are one good example. While it is a security-related activity, it cannot induce risk (except for the risk of failing the audit). So anything that doesn’t introduce risk and that can be automated, security professionals are open to – especially if it’s a dull, time consuming task.

Another area where security automation is abundant is with those processes that are just too overwhelming for security teams to deal with manually – where there’s no other feasible alternative but to automate. Patch management and event management (SIEM) systems are two good examples where manual processes are just not viable.

Creeping into the Network with Automation: It’s when we talk about automating security into change management, that trust issues come into play, big time. Nowhere else is security risk more predominant than when it comes to configuring security devices, because more often than not, misconfigurations lead to outages. When it comes to firewall policy or network security management, automation has to prove itself before security teams can hand it off. Baby steps are called for here.

How?

Consider starting with the processes that can be reviewed in detail first with a simulation of the automation before the “RUN” button is pressed – a dry run. This will certainly help determine what the system is going to do or not do. If teams have the opportunity to review what will be configured before it gets automated, you have started on the path to trusting security automation.

This is more about collaboration and breaking down silos just as much as it is about technology. Bringing security in to review the change processes as they are being automated, is a trust and team building exercise in and of itself, and ensures that security considerations are baked into process automation from the get-go.

Trust between the people determining which processes should be automated – and how – tends to occur over time, as everyone gets to know each other, and especially as security teams have the visibility and input they need to ensure that security is baked into process automation. As concerns are assuaged, security teams will become increasingly hands-off, but they need to be eased into it.

Phasing in More Network Automation: For security teams looking to take a gradual approach to full automation, Security Policy Orchestration (SPO) is the perfect vehicle. I like to think of this concept as a version of automation but a more intelligent way of doing automation. Automation in itself is like a robot repeating tasks in a dumb way. Orchestration is more like a conductor coordinating a talented group of musicians (network of complex systems) who are playing a symphony together.

Core to this approach is simulation. Once the organization has its network topology accurately mapped out, the system can automatically identify relevant policy enforcement points (firewalls, routers, load-balancers) that are associated with the flow of the business connectivity requirement. Based on this, a network simulation can be performed to accurately design the implementation.

In moving toward automatic network design, DevOps teams can greatly benefit from leveraging tried and true best practices already in place, in the security world. For example, automatic risk analysis against a pre-defined zone-to-zone security policy, across the entire network, allows the security team to review and decide whether certain business needs are aligned with corporate security guidelines or not.

As trust is gained in the automatic network design and the automatic changes set forth by pre-defined policies, there will be less reliance in reviewing every design change and more trust in automation. As soon as the system automates change provisioning, you have reached a mature automation model.

Orchestration handles changes as part of a process rather than through hidden API calls, this gives you the necessary transparency including an audit trail, separation of duties and check points that are critical for trusting the machine. The capability of providing an audit trail enables a post-mortem analysis in the event something fails to go as intended. Checks and balances, allow for human intervention where defined conditions occur. Lastly, the system has to be very accurate, reliable and secure.

The path to security automation may not be a quick one, but by taking a phased approach and automating the right processes, everyone will feel more confident with automation domination. This way, our favorite control freaks, the guys we trust to secure our network, can focus their attention on addressing the next generation of security challenges.

May 28, 2014
You have to crawl before you walk…

In previous posts I have explained what Security Policy Orchestration is, why DevOps folks should care, and how it can help facilitate the cultural change necessary for organizations to reap the long-term benefits of the cloud and virtualization. In this post, I’ll provide a few examples of how Security Policy Orchestration can create some “quick wins” – wins that create common ground between Security, Dev and Ops, and set the stage for higher profile wins with significant business impact.

As infrastructure becomes increasingly complex it gets harder to decipher. In software we call this Readability – it determines our ability to support the code over time and to ensure security. The same concept applies to networks – the more complex they are, the harder it is to use them, maintain them and ensure security.

The primary value of Security Policy Orchestration is providing a holistic view of the infrastructure. It “sees” the entire network and creates an index, which enables network administrators to perform quick queries across their environment providing “needle-in-a-haystack” search capabilities. While it might seem obvious, gaining visibility into the environment is a very powerful tool that can deliver huge benefits, especially when the environment is complex and subject to constant changes.

Let me give you a few use cases:

Audits: As my security brethren know, preparing for an audit can take weeks (if not months).   However, once the network infrastructure is indexed, meaning you have structured records of the network elements (devices, interfaces, routes, security rules…), the kind of documentation required for security audits for PCI, SOX, NERC, and other regulations is literally a few mouse clicks away. Imagine finding all applications flows involved in a PCI flow in one click, or, even better, through a REST API call.

Many of Tufin’s large enterprise customers report that their internal audit requirements are more stringent and time consuming than external compliance mandates, and require customized reporting, which they often have to script themselves. Policy Orchestration solutions enable them to be much more agile – to the point where security no longer becomes a “showstopper.“

A successful and effective audit (one that actually improves security) requires a collaborative approach across Dev and infrastructure – having the needed data at hand ensures agility, which is crucial for embedding this into the continuous process.

Server Migration: Many enterprises and service providers are currently tasked with server migrations to virtual data centers or cloud environments. The challenge they face is identifying the talkers, connections and protocols that are needed to maintain connectivity and business continuity. Firewall policies are often the best place to find this information because their important role in “attaching applications to the network”. However, it’s a labor intense task. Even if you have only 10 firewalls, with a few hundred rules and a thousand objects, identifying the flows is going to take a while. Now multiply that by 100 migrations per month – it’s virtually impossible.

However, once you automate the process, it becomes quite easy. With change automation in place, all you have to do is click “Find”, or, even better, call an API, and you will know the flows immediately.

Application Deployment: As most devopsy.kinsta.cloud readers are (likely) well aware, when an Application team needs a connection opened to say, an LDAP database, they open a ticket with a business request: “I need to connect server A to LDAP database B.”

It’s the network guy (or gals) job to translate that request into a set of IP addresses that plot a secure, efficient network path that connects the database to the application. A week or so later, the requisite changes are made, and voilà, the application can leverage the LDAP database. Or not, and then the network team needs to re-trace its steps and figure out what the problem is.

The above process, in many organizations, is a loose one, usually conducted via email, and between people who don’t often traverse the same orbit.

Add human error into the equation – a mistyped or incorrectly documented IP address, and you end up with re-dos, which everyone hates because they are time consuming and in complex network environments can be impossible to trouble shoot manually.

Ask any network manager – reducing (if not eliminating) re-do’s is an impressive feat that offers substantial efficiency gains.   It’s also worth noting that without automation it is really, really hard to properly risk assess proposed changes. In worst-case scenarios, an insecure network connection becomes apparent after something bad has happened. But why wait for something bad to happen before innovating? Bring security into your next network virtualization initiative at its onset – I promise you, you won’t be sorry.

DevOps folks are likely familiar with the idea of abstraction, but it is still a relatively new concept in network security circles.   When applied to network change management, Abstraction separates business logic from the technology underpinnings of network infrastructure, and allows Operations teams to translate higher-level application language into terms they can use to enable underlying network services.   In scenarios like the above, the ability to automate this process can not only deliver huge efficiencies and create a much more collaborative dynamic between application, network and security teams.

Hopefully, these examples build the case for how Security Policy Orchestration can be leveraged to deliver some quick wins. Of course, organizations that commit to deeper and wider change reap greater benefits. However, as my Co-founder, Tufin CEO Ruvi Kitov likes to say, you have to crawl before you walk, and walk before you run. So in my next post, I’m going to explore why security people are so weary of automation, and how to get around that, in hopes that Security, Dev and Ops teams can all walk together.

April 29, 2014
Dev, Ops and Security Collaboration: Bring the body and the mind will follow
Complexity has a way of muddying even the clearest of waters, and this has certainly been the case with IT Operations. While Dev, Ops and Security teams share a common purpose, the silos between them are so deeply entrenched, it’s easy to see how they have forgotten that they are working towards the same end. Fortunately, the rise of DevOps has provided the opportunity to break down those silos and leverage automation to increase agility, efficiency AND security. But accomplishing that requires an alignment that is only possible through a sustained, collaborative effort. So, I wanted to dig into more detail into the stakes Security, Dev and Ops all share, as I am certain the benefits of working together will become startlingly apparent.

First, let’s take a look at where each group is coming from:
- Developers want fast, automated application deployments. They only care about infrastructure as it relates to their ability to roll out applications.
- Operations folks (network managers) live, eat and breathe infrastructure. When it comes to application deployment, they need clear technical requirements to reduce “redos.”
- Security folks need to make sure policy & process are enforced with auditable security controls that also can ensure compliance.
A big part of the problem is that until now, security has been introduced at the tail end of the development lifecycle, after risky practices have been introduced into application delivery architecture. Security comes in after the fact, implementing fixes and controls the best they can. Obviously this slows things down, and creates a dynamic in which security is perceived as an obstacle, and security teams are pigeonholed as “the folks who are always saying no.”

The good news is that automation can ensure that security is woven into Dev and Ops processes from the get go, and to great benefit. But technology alone won’t change the underlying problem. Breaking down silos requires cultural shifts that don’t happen overnight. The first step to changing these dynamics is to better understand the relationship that exists between applications and infrastructure. This does not mean security folks need to become domain experts in agile development, or that developers need to become network security experts. In fact, even a basic understanding of the collective impact of these trends should create the common ground needed to move towards a more collaborative model.

As the DevOps movement reaches critical mass, the increased agility, greater efficiency and substantial security improvements it can deliver are being talked, written and Tweeted about in depth, and with increasing frequency. And most importantly, these benefits are not just marketing buzzwords – they are tangible and multifaceted, as is illustrated in Steve Hall’s Jan 2014 blog post:

“Business people completely understand ‘release my product faster’, ‘time to market’ and ‘make more money’ which are some of the outcomes that DevOps pontificates on. This is why DevOps as a movement has a higher probability of succeeding than other investment choices because there is a direct correlation between it and top/bottom line revenue. So if you’re an InfoSec leader, a good bet is to align yourself to the DevOps initiative (or spearhead it yourself for that matter) and help the business understand the value of security in a way that doesn’t measure things by # of incidents, time lapse from vulnerability to patch, or compliance score.”

As for the shared stakes between Dev and Ops, check out John Allspaw and Paul Hammond’s Velocity 2009 talk on how cooperation between Dev and Ops enabled 10+ deploys per day at Flicker. If that doesn’t demonstrate what “increased agility” looks like, I don’t know what does.

These are just a sampling of the myriad ways Security, Dev and Ops can reap greater benefits by working together. But…. it takes time and effort to get the ball rolling. Perhaps the best way to get started is to make sure Dev, Ops and Security start talking, literally. Once a week. Every week. No matter what! Get the bodies there, and I promise you, their minds will follow.

Based on our customers’ experience, a few quick wins is all that is needed to get the ball rolling. Once Dev, Ops and Security folks experience the upside for themselves, those silos will start to crumble. In my next post, I’ll provide some examples where automation can enable Dev, Ops, and Security to gain those quick wins and build momentum for further cultural change.
April 4, 2014
What is Security Policy Orchestration and Why Should DevOps Teams Care?
After attending the DevOps day I wrote about in my last post, I wanted to take a minute to explain exactly what Security Policy Orchestration actually is so we can begin to understand how the two concepts overlap.

Simply put, Security Policy Orchestration is intelligent network change automation. It goes beyond simple task automation by truly understanding the network through simulation and analytics. It also gives security people an extra level of governance and control, which is so critical to assure security and business continuity.

Security Policy Orchestration reflects the growing relevance that managed security policies are having on IT systems. In network security lingo, security policies are firewall policies, but can also apply to Access Control Lists (ACLs), router and load balancer configurations. Managing security policies is an arduous task that requires automation, and Security Policy Orchestration has emerged in response to numerous factors happening in tandem – mainly, that:
- Business agility is gaining importance as a key competitive factor. This is creating an increased demand for automation across IT and, specifically at the network layer.
- IT organizations are pressured to match the speed, convenience and self-service provisioning the cloud environment offers for internal application deployment.
- Enterprise networks are growing larger and more complex with numerous traffic routing and shaping technologies to service multiple data centers.
- The widespread use of stateful and next-gen firewalls for external and internal network segmentation has significantly increased the size of the enterprise firewall estate, forcing security policies to morph into complex rule sets that map how business is conducted over the network.
The collective impact of these developments on network and security operations tasks is having a profound impact on people, processes and technology, including DevOps groups, who like their Security Operations (SecOps) brethren, are under pressure to roll out services faster.

Security Policy Orchestration helps to alleviate that pressure, enabling SecOps teams to keep up with the demands of the business while ensuring security and preventing an outage or a breach. Being able to automatically generate, provision and synchronize Security Policy changes across a growing set of systems – with security, compliance and governance concerns accounted for – means that security will not become a bottleneck that slows down DevOps teams.

While Security Policy Orchestration evolved out of network security operations, its benefits transcend security. Even though it will remain a SecOps function, its impact on DevOps is only going to increase, especially as Network and Security Operations continue to converge. However, the fundamental enabler of Security Policy Orchestration is automation.

As a result, there is significant common ground between DevOps and SecOps when it comes to automating application connectivity. This may not have been an issue for you yet, but my introduction to DevOps came from my experiences helping Tufin customers automate application connectivity. It’s a subject that is extremely germane to DevOps teams, and a topic that I plan to explore in depth in this blog because those of us leading the Orchestration charge within SecOps are kindred spirits with DevOps evangelists.

Contrary to popular belief, Security Policy Orchestration wasn’t created to restrict a company but to free it and help DevOps teams to accomplish its vision at the lowest IT risk level possible. After all, there is no such thing as bulletproof security where the Internet is concerned.

Instead, security folks focus on reducing risk, which puts us in a much better position to add value to the business. Plus, since so many of today’s biggest threats (i.e. APTs, for example) are network-based, introducing the tenets of Security Policy Orchestration – which literally automates best security practices into change processes – is in the mutual best interest of both SecOps and DevOps teams.

In my next post, I plan to dig into more detail on the stakes Security, Dev and Ops all share to make IT work faster, better and safer, and propose a few ways in which SecOps and DevOps teams can collaborate to gain some quick wins.
March 18, 2014
Why Baking Security Into the DevOps Framework is Vital to Its Success

I attended my first official DevOps conference in Israel in September 2013. But I was initially introduced to DevOps by my customers – firewall and network managers who, through their efforts to automate security policy management, found themselves dealing with a new and like-minded set of IT stakeholders – DevOps teams.

They said DevOps was also all about automation – automation that allowed IT to increase their business contributions by enabling new applications and other IT services to be rolled out faster with greater control and efficiency.

Conceptually, it was clear to me that the DevOps value proposition is just as relevant to Security as it is to Operations and Development groups, especially since Security teams apply automation with the same fervor as DevOps.

In fact, using automation to accelerate IT has caught hold in the network security world to fuel a new movement and class of solutions, called Security Policy Orchestration. As the CTO of Tufin, a Security Policy Orchestration company, I could not help but recognize the synergies. I was (and still am) intrigued by DevOps, so when I heard about the conference, I decided to go.

And I am so glad I did.

The alignment between DevOps and Security Policy Orchestration is obvious. They both champion the use of automation to accelerate the pace of business. We Security Policy Orchestration folks are just coming at it from a different area within the organization. Kindred spirits, right?

In theory, absolutely, but in reality…not so much.

The conference as a whole was great. It completely validated my theories regarding the synergies between Security Policy Orchestration and DevOp. But what surprised me was that security was a topic that was not being discussed at all at the conference.   Maybe a hat-tip to application security, but network security was not even on the DevOps radar, at least not at this particular conference.

It needs to be. For DevOps, to really be able to move the needle, we have to work together. If history has taught us anything, it’s that when new applications and services are not created or rolled out with security in mind, at some point we will have to go back and retrofit it back in.

Envision the typical experience:   the development team builds a new application, brings IT Operations in to implement it. Once that is done, IT Ops prides itself on all the corkscrewing it took to make the application work on the actual infrastructure. Except for once the app is in production, it doesn’t work. The troubleshooting begins. Invariably, issues of network connectivity come up. Solving them requires app, dev and security managers to break down the cultural and organizational barriers that exist within their organizations to work together to ensure the application is architected correctly into the network. This starts and the process of change requests between AppDev and IT to get the app up and running.

If that process is not automated, it can take months longer than it needs to resolve everything.   Not to mention any company subject to PCI has to consider terms like secure zoning, a concept that is not second nature to DevOps folks.   If PCI is not an issue, there are plenty of other internal and regulatory mandates and security –related considerations that, if for no other reason than common sense, can exponentially reduce the chances of a company ending up in the headlines due to an APT.

These are just a couple of many scenarios that can be avoided if development, IT and security work together from the get go.

The net/net is that Dev and ops have the same goal but they are siloed, which is slowing down the pace of business – inhibiting it. Collaboration is not some warm and fuzzy team-building thing – it is a fundamental business requirement.

And that’s why I am so excited to have the opportunity to do this blog. I think I can add a unique perspective on how DevOps and Security can work together to reach our common goals.

My next post will explain will dig into what Security Policy Orchestration actually is, so we can begin to tease out the similarities and determine how to build on that to true alignment and partnership.

Stay tuned as the drama unfolds! This should be fun….

March 11, 2014