devopsy.kinsta.cloud

Category: DevSecOps

The #1 trick DevOps shops use to reduce vulnerabilities

A new report out this week highlights how important it is to find ways to fold in vulnerability assessment information into the software development lifecycle. Released by WhiteHat Security, the Website Statistics Report 2015 offers a smorgasboard of interesting data around web application security. Amid all of the statistics, though, DevOps shops should perhaps home in on one major takeaway—statistical evidence that points to a single practice that could make a huge difference in reducing vulnerabilities in the long run. The trick in question? Tying vulnerability data into the bug tracking system.

“When it’s all said and done, there are three metrics that matter in application security. It’s the number of vulnerabilities you have, the speed at which you fix them and the percentage of which are fixed,” says Jeremiah Grossman, founder of WhiteHat. “And what we’ve found is that when you wire into the apparatus whatever vulnerability assessment data that you get into the bug tracking system, which is very important for DevOps, that actually is what achieves the best results.”

As he explains, simply providing vulnerability assessment data in a raw format—for example, a PDF document listing flaws—this does nothing to fit within an automated pipeline and vulnerabilities will remain unfixed. In the report released this week, Grossman reports that organizations that made the connection between the vulnerability feed and the development process’ bug tracking system exhibited 40 percent fewer vulnerabilities, fixed issues a month faster on average and increased their remediation rate by 15 percent compared to the rest of the pack.

While Grossman admits that it is difficult to know exactly what percentage of customers ascribe to different development methodologies, his rough guess is that about half at this point are instituting some kinds of DevOps principles in order to release at least every two to three weeks. He recommends that security professionals hoping to get more involved in the spirit of cooperation that drives DevOps transformations should start with metrics.

“Create a metrics program, just like the DevOps guys do, to measure the security of that environment,” he says, explaining that the three metrics mentioned above are good ones to start with. “Then you can track the performance of the DevOps groups in terms of security and deliver that data back to the group. Then when you want to get strategic about what processes and improvements in the DevOps environment that need work, you can be very laser focused. App sec programs should be tailor-made to the internal processes of the organization around the metrics they want to improve.”

May 26, 2015
DevOps Leadership Series: Security at Velocity

If it does not fit, it does not get done. For many DevOps practices, application security falls into the “does not get done” bucket. That’s because for many DevOps-centric organizations, application security has historically been done somewhere else, by someone else, who is slow.

Go faster. Shift left. Remove complexity. Reduce rework. All mantras of DevOps practices. And while DevOps practices have changed dramatically in recent years, many experts will tell you that application security has not changed enough.

In this installment of the DevOps Leadership Series, you will hear Chris Corriere (DevOps Engineer, Autotrader) and Mitchell Ashley (VP Information Technology, CableLabs) share perspectives on the state of DevOps and security. It is about security at velocity.

But is going faster and automating resulting in better security? There are still security pros who are uncomfortable with that premise. Nevertheless, that is where this is headed. The security community has to come to the mountain, otherwise the mountain is coming to them.

First, listen to Chris’ Corriere of Autotrader tell us his perspective that security can move at DevOps speed, as long as you take a diversified approach:

https://youtu.be/8DdfHkAK1Fk

Then hear from Mitchell Ashley, VP of IT at CableLabs as he remarks on the evolution of security from a validation role (at the right) to an integrated, ingrained role that has shifted left:

https://www.youtube.com/watch?v=pg_m7dYqUPU

This is the third in our series of interviews from this event. Stay tuned as the actual sessions from DevOps Connect will be posted shortly.

NOTE: If you have missed any of the other videos from this series, you can find them here. (We’re up to 15 so far).

May 21, 2015
The DevOps and Security Manifesto

Early in Illumio’s history, I saw the future of DevOps security. The CISO of an electronics manufacturer brought us in to help secure a new support service that was built entirely in the cloud. After meeting with the security team, we were introduced to the application development team who cast a cold eye on our technology. About five minutes into the demo, the group’s manager asked to take control of the keyboard and start using our software. “I know how to use this. It’s like network-security Chef,” he said with a wry smile. He flew through the UI.

Fast forward to the recent DevOps Connect event at RSA, which underscored the critical importance of cybersecurity to the application development process. If business increasingly relies on agile software development, the lack of corresponding fast-moving security approaches effectively increases the risk of a breach or a cyberattack. You cannot build and mount applications in a distributed application and computing environment and then rely on a static, hierarchical security model built upon chokepoints, infrastructure control points and organizational silos.

Traditionally, one group created applications and another deployed them on the compute infrastructure. Then, a third group implemented security and ensured applications were trusted, particularly through instrumentation of the perimeter network security. Those days are disappearing.

It is time for a new relationship – and new, shared technologies — among DevOps, infrastructure and security, expressed through what I believe are are 10 principles for DevOps and security.

Principle I — No divine right of InfoSec primogeniture. Security must not be run in a silo. While security teams play the most critical role in assessing corporate risk and setting policy, there must be leadership and shared responsibility across various IT functions.

Principle II — Orchestrate but remediate. Security must be as responsive to continuous delivery as application creation and scaling. Application developers need cool, agile tools for security, too.

Principle III — Application developers must not put the organization in harm’s way. They must consider security at the beginning of the application development cycle and not simply hand off the responsibilities to others. Moreover, they must have security capabilities well instrumented into application development tools and infrastructure.

Principle IV — Speed should not kill. Lack of speed hurts business. The spread of malware kills business. The speed of application development should not provide a corresponding acceleration in security risk.

Principle V — Reduce the attack surface. Application developers must co-join with security teams to reduce the attack surface available to bad actors. By considering exposure and risk at the front end of the DevOps cycle, fewer issues should arise afterwards.

Principle VI — Security must be built-in, not bolted on. Application and security architectures that can contain the threats are the order of the enterprise. To Principles IV and V, this increases organizational speed and reduces risk.

Principle VII — Equality of the data center and cloud. Information security must be considered and deployed equally for the data center and the cloud. Separate but equal approaches are not the preferred path for IT professionals. The more widely distributed and deployed applications become, the more unified the security approach must be.

Principle VIII — Stream, not batch. Security must deploy as streaming technology, not batch processes. There is no “set it and forget it” approach for security in a DevOps world.

Principle IX — A minute lost finding a breach is unacceptable. Only a minimal amount of time (second, minutes) must pass in finding cyberattacks and breaches. Systems must be engineered for constant visibility and notification of policy violations.

Principle X — Common Contribution. The maintenance of a strong DevOps security approach requires a shared contribution to application and security approaches toward the common good. The day of IT silos is ending.

About the Author/Alan Cohen

Alan Cohen – As chief commercial officer, Alan leads Illumio’s go-to-market strategy and customer engagement life cycle organizations, including marketing, support, talent, and IT. He is a 25-year technology veteran known for company building and new-market-creation experience. Alan’s prior two companies, Airespace (acquired by Cisco for $450 million) and Nicira (acquired by VMware for $1.26 billion), were the market leaders in centralized WLANs and network virtualization, respectively.

Prior to Nicira, Alan led the enterprise marketing organization at Cisco, a team of 300 responsible for bringing to market an associated $25 billion product portfolio. He serves as an advisor to several technology companies including MobileIron, Highfive, and Netskope. Alan is a frequent industry speaker and commentator whose work has appeared in The New York Times, Fortune, Re/code, and Gigaom. He has a BA from SUNY Buffalo, an MA from the School of International Service at American University, and an MBA in Finance from the Stern School of Business at New York University.

May 20, 2015
DevOps Leadership Series: Software Supply Chains
We kicked off this series on Monday with Gene Kim (@RealGeneKim) sharing his views on the big theme for DevOps in 2015: proving that DevOps is applicable for large organizations.

Another theme that arose often during our recent “DevOps: Wine-ing, Not Whining” event was the importance of software supply chains. Every software development organization has a software supply chain, and DevOps leaders are now applying principles from lean manufacturing, Deming, and Toyota supply chain management to improve their operations.

They are applying software supply chain management principles help:
- choose higher quality parts (e.g., software binaries, containers, microservices)
- select higher quality suppliers (e.g., fewer defects, faster MTTR records), and
- improve traceability (e.g.., identifying defects or known vulnerabilities to expedite break/fix)
Today, we will hear from Jez Humble (Chef), Julie Tsai (Retail Giant) and Nick Galbreath (Signal Sciences) as they discuss on importance of software supply chains and DevOps. Then after watching the videos, you can learn more about the sea change in software supply chain automation in these two blogs from Mike Hansen (Software Supply Chain Automation) and Ericka Chickowski (Security, DevOps and the shift to a software supply chain).

Here are today’s short videos from our 2015 DevOps Leadership Series:

[youtube http://www.youtube.com/watch?v=4L4nrNpdTM4]

[youtube http://www.youtube.com/watch?v=UEkVYJ1T2NY]
[youtube http://www.youtube.com/watch?v=nTv8Yx0CpfY]

NOTE: If you have missed any of the other videos from this series, you can find them here. (We’re up to 15 so far).
May 19, 2015
It’s time security pros shake their DevOps fear, uncertainly, and doubt

There’s been considerable discussion recently about how to make certain good security practices remain integrated within DevOps-driven environments. To get the scoop from a security pro who is experienced working on delivering security programs in development environments, I turned to Andrew Storms for some insight. Storms has been leading IT, security and compliance teams for the past 20 years and his multidisciplinary background includes product management, quality assurance, and software engineering.

Storms is also a CISSP, a member of Infragard, and a graduate of the FBI Citizens’ Academy. Currently, Storms is vice president of security services at New Context. Previously, he was the senior director of DevOps for CloudPassage and the director of security operations for nCircle (acquired by Tripwire). At nCircle, he was responsible for the definition and enforcement of the security programs, delivering EAL3 certification and SOC2 audits, and managed the company’s PCI ASV program.

Storms recently gave a talk at the RSA Conference, How Security Can Be the Next Force Multiplier in DevOps, where he shared his thoughts on how security teams should integrate themselves as business enablers within fast-paced DevOps enterprises.

devopsy.kinsta.cloud: What’s your take on DevOps being a force multiplier and enhancing security?

Storms: Much of it [the concern around DevOps] really is rooted in fear. People see that the organization has brought together the developer and the operations team and they fear that everything will become the Wild West. However, we’ve shown over and over through the years that bringing these teams together actually has huge positive impact.

Unfortunately, too many people in security view it differently. They think that they’ve lost control of a situation that they barely had control over to begin with.

devopsy.kinsta.cloud: What are the best approaches for integrating security with DevOps?

Storms: There are two approaches to this. One of the best ways to bring DevOps and security together is to utilize the tools and the processes that DevOps really excels at and apply them to security – things like automation, orchestration, and instrumentation. Let’s use those tools to build these closed-loop security systems where everything’s automated and everything’s predictable. That’s a way we actually can fulfill the security requirements in an automated fashion with fewer resources.

Another is to treat security more as an enabler. Security has fallen into the pitfall that, instead of being an enabler to the organization, it is all about ‘No.’ There’s pushback on new initiatives for fear of falling out of regulatory compliance, or being breached. Instead, the organization needs to take a different approach. It should view security as a business enabler, just like IT and operations.

In this way, it can secure new initiatives and actually enable people to focus on their core competencies and allow the company to take educated risks. At the end of the day, that helps to provide more of what the business needs to do, whether that is increase revenue or decrease time to market, or whatever it may be that the company needs.

devopsy.kinsta.cloud: Done right, people say that security needs to be integrated throughout all of the processes, but that’s seldom reality. What do you say to security teams about this?

Storms: That’s the approach that I take here with the integration of DevOps and security. Really, if we think about it, the integration of Dev and Ops together in DevOps has actually opened the door for security to come in and do something very similar – to come in and work more collaboratively and integrate itself more closely with the rest of IT.

What security professionals need to recognize here, but often don’t, is that DevOps is an opening. Take it. That is to say: Don’t fear DevOps; embrace it. Address the fear, and then learn about DevOps, and see where you can integrate security people, processes, and technology.

devopsy.kinsta.cloud: When integrating the people, processes, and technologies that help forward security interests in DevOps, I imagine it takes time and effort to go back and review a lot of those controls, to make sure that they’re up to speed with the way people actually work today.

Storms: I think that’s one aspect. I think the other aspect is, if we look at the manifest of Agile, which is the ability to take and see change and adapt, all of those controls, to some degree, still make sense. They just need to be adapted to the modern way that code is developed.

One of the great examples I like to share is a health care company I worked with. It is deep into continuous deployment but has all kinds of compliance requirements. So, it can’t just centrally deploy code at any time it wants without all the compliance check-offs that have to happen.

It has to run application security and compliance checks. So, it automated them, to the point where the auditors are happy. When their code gets pulled in to master, it runs it through many types of integration and security tests.

devopsy.kinsta.cloud: What happens if there is an issue; how is it resolved?

Storms: The company has put in automated checks. Think of it as kind of a speed bump; for example, when an event happens, there’s a message that is sent to the chat room [See: ChatOps, Communicating at the Speed of DevOps], and there’s a Hubot that watches the messages. The Hubot has logic that says, right before they’re ready to go, you need to stop and alert the chat room that this event needs to happen, and it needs N number of people to approve it.

For example, it may require that there be a product manager, there must be a security manager, there must be an ops manager, and so on. And they’ve got logic in there that says if the right number of people in the chat room all respond with a keyword – let’s just call it “yes” – then let’s actually capture all of that, enter a log message to go ahead and approve the release, and push it out automatically.

If an issue is found in the tests or in the approval processes, then it has to be resolved, and is then moved forward along the pipeline.

With this continuous pipeline, the company actually has all of the steps, all the compliance requirements, and the audit log. Thus, it still can maintain compliance requirements, and security. That’s what DevOps and security should be all about.

May 18, 2015
Swim in the DevOps pool or drown in security problems

There has been a significant shift recently in security. Most security vendors and organizations recognize that the traditional model of keeping the bad guys out by detecting malicious exploits is flawed at best. The reality is that the bad guys are already inside the network using authorized credentials to bypass security controls and exfiltrate sensitive data. That sounds ominous but the silver lining is that DevOps changes the game and shifts the advantage back to the good guys.

There was a time when the traditional model made sense. The attack techniques used and the motivations behind the attacks were different. In recent years, however, the line between inside and outside attacks has been blurred beyond recognition. There have been some high-profile insider attacks like Bradley Manning and Edward Snowden, but the reality is that most of the “outside” attacks were perpetrated using stolen or compromised credentials. In other words there is no difference between an inside and an outside threat at the actual point of attack.

Organizations have to guard against both inside and outside attacks. In almost all cases, though, the root problem is credential abuse. Whether it’s an authorized employee accessing systems or data in an unusual way or an outside attacker moving laterally through the network and exfiltrating data using compromised credentials the crucial part for an organization is to have detection methodology in place capable of performing anomaly analysis to identify concerning behavior and activity.

Improving security through DevOps

That’s where DevOps comes in. Organizations need to have continuous monitoring in place. Anomalous activity isn’t something you can just conduct a daily or weekly scan for. If you don’t detect the activity in real-time and do something to stop it immediately the damage will already be done by the time you retroactively review log data and discover the breach.

Things are heading in that direction—but slowly. The good news is that DevOps seems to be picking up steam as more security vendors and organizations face the reality that the traditional security model can’t effectively protect against attacks.

“The DevOps movement is really gaining momentum but unfortunately still small,” agreed TK Keanini, CTO of Lancope. “It is a small but passionate community that can do nothing else but grow and help fill a void as we move from traditional enterprise IT to Internet IT. The tempo of DevOps is its major security feature and—with the principals of John Boyd’s OODA loop—has a winning formula to make it too expensive for adversaries to attack.”

DevOps isn’t just a trendy way of providing or delivering security. The security vendors themselves are also starting to embrace DevOps internally as a means of keeping up with attackers and working to develop new tools and techniques more effectively and efficiently.

Keanini explained that everyone will be moving to DevOps over the next 3 years—just not all at the same time. “Where IT was infrastructure and development was applications, suddenly infrastructure is the application or you could say that applications have become infrastructure but in either case, the business is forced to redraw the org charts and redo the processes.”

Jump in! The water is great

Attacks are relentless and attackers are nothing if not innovative. If you’re using yesterday’s security tools and techniques to defend against last week’s attacks and attackers you’ve already lost. Organizations need to move faster to stay ahead of attacks, and the most effective way to accomplish that goal is to incorporate DevOps tools and principles into the security model.

Keanini summed up with, “It is exciting and I encourage folks to start jumping off the diving board as they are filling up the pool.”

May 15, 2015
A True Story: DevOps(Sec) Manages Out Elective Risks
A True Story

There are over 2000 developers in Bill’s organization. He is a C-level executive for one of the largest insurance companies in North America. Bill boosted developer productivity by 15% last year after taking a closer look at the company’s software supply chain. And this approach isn’t unique to Bill’s organization. Many high performance IT and DevOps teams are adopting proven supply chain principles to accelerate software delivery.

Origins, Quality and Integrity

The concept was rather simple and straightforward. Bill’s team recognized that they were consuming hundreds of thousands of open source and third-party software components to build their applications. But at the same time, they did not apply much scrutiny to the origin, quality, age, or integrity of those components. Developers simply selected components at-will that met their functional requirements and helped them meet the next delivery deadline.

Secrets of the Free-For-All

Deeper analysis of their practice revealed that the free-for-all component sourcing methods were leading to quality, efficiency, and productivity issues. Here is what Bill found:
- The company had downloaded an average of 27 versions of over 100 different binary artifacts over a one year period. Bill knew this meant the quality of their applications was being impacted through the use of outdated software components.
- The company had also downloaded 13,203 software components with known security vulnerabilities — and 65% of those included alerts earlier than 2014. Without quality controls in place, Bill recognized they were sourcing in elective and avoidable risks that would impact the integrity of their application, grow their technical debt, and lead to more rework down the road to replace the flawed components.
Removing Complexity

Bill decided to take action. He knew that to accelerate innovation, he needed to remove complexity from the company’s software supply chain. He enacted three moves to eliminate waste and complexity within the open source and third-party components being used:
1. Automate the scrutiny of software suppliers to minimize risk and bloat.
2. Enable real-time traceability and visibility to components used across the development lifecycle, in order to improve response times when defects were discovered – cutting back from days/weeks to minutes!
3. Reducing complexity, rework and risk by standardizing on specific components types (e.g., web frameworks, logging frameworks, encryption modules), using fewer and more current artifact versions, and managing out elective security risks.
Want to hear the rest of the story?

The same principles that Bill applied to his business that boosted developer productivity by 15% are being discussed by Gene Kim (@RealGeneKim), author of The Phoenix Project, and Joshua Corman (@JoshCorman), CTO of Sonatype, on April 30th. Both Gene and Josh are huge fans of marrying DevOps practices with leading supply chain management principles that enable developers to maximize throughput of features from “code complete” to ‘in production,” without causing chaos.

If you’re looking to achieve similar goals, be sure to join Josh and Gene for this April 30th, 1pm ET, discussion.
April 23, 2015
Welcome to the ADC (After DevOps Connect) era of DevOps and Security

I was speaking with Britta Glade of RSA Conference after our DevOps Connect conference at RSA Conference Monday. She congratulated us on putting together a great day of tracks and sessions (kudos to Gene Kim and Josh Corman). But then she said something else that really struck me to my core. She said after today there is no longer any question about security working with DevOps.

Think about it. Three years after Josh and Gene first presented Rugged DevOps at RSA Conference. After so many papers, articles, presentations and tracks, it has finally sunk in. Security will embrace DevOps, Security will be better because of DevOps and as importantly, DevOps will be better because of Security.

It is very fulfilling to know that this event we produced (along with Mark Miller and theNexus Community) was what finally pushed this over the finish line. For me personally it really represents one of the fundamental reasons I was so attracted to DevOps.

It was 4 or so years ago that I first met Gene Kim and over a bottle or two of wine he explained to me what DevOps was about and what he was working on with the Phoenix Project. While DevOps may seem primarily about Dev and Ops to some, for me it was about making security work better. It seemed like such a no brainer that of course security should embrace DevOps.

But it was not that easy. The besieged security industry faced with a never ending barrage of breaches and a continually escalating threat environment could not accept that automation, acceleration, velocity would also allow us to shift security left, leaving us more secure and more compliant. Many security people dug in their heels and said no, this wouldn’t work and they didn’t take the time to really explore it.

Monday all of that changed. Security folk came out in droves to see people like Jez Humble (many people in the audience weren’t really familiar with Jez when he first took the stage, but they were enthralled with him by the time he stepped off. Jez finished to a standing room only crowd of close to 700. They heard Damon Edwards and Alex Honor talk. Though they were also unfamiliar to the crowd, their message resonated to the core.

When Julie Tsai of Walmart took the stage the audience heard how a “unicorn” like Wal-Mart used Agile and DevOps to be more secure. Terri Pots of Raytheon and Jessica Davita of Microsoft re-enforced the message that security needs to embrace this approach. I loved Jessica’s security org chart. Chris Corriere who writes here on devopsy.kinsta.cloud, Dr. Aaron Cois of Carnegie-Mellon also had great sessions.

Then names familiar to the crowd, Gene Kim and Josh Corman kicked it off with a great talk on Software Supply Chains. Nick Galbreath had a great session. David Mortman delivered a terrific talk, Dan Cornell on web app security and more. For me the perfect ending was when my friend Rich Mogull took the stage and demonstrated his Squirrel Monkey toolset. Rich’s scripts that he wrote himself showed how we could use automation to make our security better and easier. People were stunned.

Throughout the day, the buzz that was coming down the big hallway was that there were some great things happening over at the DevOps tracks. People from the Cloud Security Alliance meeting next door were hopping over to check it out. Other people joined in. By the end of the day we had given out all of the materials we had prepared.

By now I know you are wishing you were there. The good news is the entire event was videotaped. We will have videos and slides of all of the presentations shortly so stay tuned.

We are already starting to plan next year’s event. Also, the call for speakers is now open for DevOps Connect: Rugged DevOps event @InfoSecurity Europe June 4^th. But after Monday DevOps and security will never be the same again. No longer can security deny that they must be part of DevOps. We are and should be. We are now in the ADC (After DevOps Connect) era.

April 22, 2015
Illumio Hopes To Cater Security To Continuous Delivery And DevOps Shops

Since coming out of two years of stealth development back in October, the data center security gurus at Illumio have been building momentum in the drive to break out of the confines of the traditional network-centric security model to make it easier to secure IT in the face of distributed computing, continuous delivery and DevOps patterns. This week, the firm got a big boost in the way of $100 million in Series C financing and continued forward movement with some new tweaks to its Adaptive Security Platform (ASP).

Designed with DevOps patterns in mind. Illumio’s ASP is meant to be a continuous delivery system for dynamic policy enforcement within application traffic across everything from bare metal servers to VMs and containers within the data center and public cloud infrastructure. According to Alan Cohen, chief commercial officer for Illumio, the idea is to build security attuned to the infrastructure as code movement in IT.

“Security has been tied to the infrastructure for the last twenty years–not just client-server, but also right back to the mainframe. And as we’ve created software infrastructure–software models for applications and compute processes, whether its storage or network or servers that can be spun up or spun down or moved very rapidly–the security industry hasn’t adapted well because its still very hardware, choke-point centric,” he says. “The industry hasn’t done anything to adapt to distributed compauting and continuous delivery. (When vendors do make adjustments) they’re just running something on software but it operates as it did before.

The announcement this week adds further capabilities for the platform to segment the continuous delivery of that enforcement down to the process level within workloads, so that as any part of the application changes, the platform automatically adapts security policies on all impacted workloads.

“So let’s say you have an application that’s a credit card processing application and it uses a certain database. You may have two instances of that database running on a single host. One is for PCI and one is not,” says Cohen. “Through our virtual enforcement node we can segment and separate those two workloads and effectively provide a layer three layer for firewall at the process level on an individual host.”

The added segmentation of dynamic security enforcement gives DevOps shops a greater degree of flexibility in securing microservices environments.

“When you talk about microservices there’s two things to think about,” says PJ Kirner, chief technology officer and founder of Illumio. “One is that they’re small and process-oriented, and the other one is they exist for a short amount of time. Both those aspects are important to consider for security. The fact that it needs to adapt quickly to all of these ephemeral workloads that are spinning up, spinning down to accomplish your goals.”

April 21, 2015
Live Blogging DevOps Connect @RSAC

We will be live blogging the DevOps Connect: SecDevOps event from RSA Conference on Monday, April 20th. Stay tuned for updates

April 18, 2015