devopsy.kinsta.cloud

Category: DevSecOps

  • What Biden’s Cybersecurity EO Means for DevOps Teams

    What Biden’s Cybersecurity EO Means for DevOps Teams

    On May 12, 2021 President Biden issued Executive Order 14028, also known as the Executive Order on Improving the Nation’s Cybersecurity. This EO covers a lot of ground, and like all executive orders, it instructs agencies of the U.S. Federal Government to perform specific actions. What it doesn’t do is appropriate funding or create industry mandates. That’ll come later, once some of the plenary stages of the EO are complete.

    Despite this, I’m confident that EO 14028 could be as impactful to cybersecurity as GDPR was to data privacy. I’m also confident that the EO has the attention of business leaders who are trying to figure out what it means to them and their teams—DevOps professionals. While the EO has no underlying mandate to private businesses or those located outside the U.S., it does outline a reasonable set of cybersecurity best practices that can be reasonably interpreted as a call to action for greater transparency.

    In this article I’m going to use the terms supplier and buyer, but I don’t mean to imply that money needs to change hands; only that there are producers of software (suppliers) and consumers of software (buyers). If there is one supplier and one buyer, the software supply chain has one link. If that supplier is also a buyer, the chain has two links. For most software, there are many suppliers; in these cases, the supply chain isn’t so much a length of links but more of a mesh of interconnected links.

    The 2021 Synopsys OSSRA report showed that the average audited application had over 500 dependencies, so that mesh is quite complex—to the point where the security practices used anywhere in the mesh are often obscure, at best. It’s this obscurity that attackers can exploit, which is why one of the key tenants of the EO is transparency.

    As anyone charged with operating a piece of software knows, the security testing performed by a supplier isn’t really obvious, but, as a buyer, you assume the supplier has done the right things when it comes to security. But how do you know? Do you know where the code originated from? Do you know what security targets were used by the supplier? If an attacker is actively compromising the application, how would you know?

    These are all very good questions that are reflected in the EO, to varying degrees. Those questions also belie the reality that suppliers test software. Suppliers do, of course, test their software, but what testing techniques did they use and how effective are/were their tools? And, more importantly, how can you confirm that the mitigations and fixes applied to address identified defects were effective for your deployment scenario? After all, it’s not like every supplier is going to provide you with full access to their source code.

    This is where things become quite actionable for DevOps teams that have embraced the principles of operational feedback mechanisms. For those needing a bit of a refresher, that’s the part of the DevOps mobius where lessons learned operating software are fed back into the next iteration of the development flow. Let’s look at two pieces of operational feedback that are mentioned in the EO: least privilege and log management.

    Log management features prominently in the EO in part because logs enable an understanding of how an application is operating. If a supplier provides detailed information about the log structure and flows, it then becomes possible to build alerts in log aggregation services and SIEMs as indications of unexpected operation. That’s the proactive model the EO embraces—as opposed to forensic postmortem log analysis following a breach—but this requires the supplier to provide insights into the log data indicating that the system became compromised. Of course, to provide that type of data, the supplier must have tested their system software in a manner that allows such a log template to be created.

    We’ve heard that software should be developed using principles of “least privilege” for years, but if you’ve ever been asked to run an application as root or had a supplier not detail the precise network ports required by the application, you know there is still work to be done there. The EO specifically calls out cybersecurity techniques like multifactor authentication and zero-trust network access as ways to achieve granular control over how applications operate.

    It is, however, the coverage of Software Bill of Materials (SBOM) that could have the greatest impact for DevOps teams. In effect, the EO stated that suppliers need to tell buyers the origin of all code within an application. While the EO doesn’t specify the elements of an appropriate SBOM, it does instruct NIST to publish the minimum elements of an acceptable SBOM within 45 days of the EO. When combined with the sections on how build environments are created and managed and the security transparency elements elsewhere in the EO, it’s clear that the EO intends to raise greater awareness of how software is created, patched and maintained.

    In the end, Executive Order 14028 sets a high bar for cybersecurity in general—one where everyone involved in the creation and operation of software will need to up their game. I’ve said for years that it’s impossible to patch something you don’t know you’re running, and that remains true. What must be added is that you also can’t reliably secure something when you don’t know how well each component was tested. The EO is prompting us all to take a step toward solving that problem.

  • Fixing Risk Sharing With Observability

    Fixing Risk Sharing With Observability

    Incentives are mismatched among SREs, SecOps, and application developers. These mismatches create challenges around how and what information is shared across siloed teams. This asymmetrical information creates a moral hazard where one team can shift deployment risk to another team, with no accountability back to the originating team. Risk shifting results in unstable applications, inefficient infrastructure, security issues and poor customer experience. All of that impacts your company’s bottom line.

    Closing the Information Gap

    Observability is positioned as a way to fix the information gap between these disparate groups. Observable systems allow users to ask questions about their behavior in an open-ended way, unlike more rigid monitoring systems. SREs and SecOps teams are already reaping benefits from observability, but mismatched incentives limit its effectiveness. Meeting the observability expectations of IT leaders requires pervasive instrumentation across applications, infrastructure and third-party software. Delivering that level of instrumentation has remained out of reach due to incentive mismatches, as well as human and infrastructure costs.

    The main incentive for developers is shipping code quickly at a reasonable level of quality, meaning a low bug count. This contrasts with SREs, who are incentivized around uptime, performance and efficiency, while the incentives for SecOps teams are around risk reduction and breach mitigation.

    risk sharing observability

    Since SREs and SecOps teams don’t have insight into what developers have changed, it’s easy for performance or security problems to get thrown over the fence to operations. A fresh code push might only include minor changes and be harmless to existing operations. Or, it could replace large chunks of logic across the entire codebase, including adding calls to external and third-party applications. To the intrepid SRE and SecOps teams, one change looks like any other. They don’t get a peek behind the curtain because of the mismatched incentives. DevOps wants to deploy quickly. Waiting on approval from other teams slows down deployments, so comprehensive reviews don’t happen. This doesn’t mean DevOps teams are intentionally trying to sabotage partner teams. They’re simply acting in their own interests based on their incentives.

    The challenge is that one party, the developers, has more information than other parties. That information asymmetry is what creates unbalanced risk sharing. Coping with information asymmetry has led to all kinds of new collaborative models, starting with DevOps and evolving into DevSecOps and other permutations like BizDevSecOps.

    True collaboration has been hard to come by. Early DevOps efforts are often successful, but scaling beyond five to seven teams is difficult because teams lack the breadth of experience in IT operations or the SRE capacity to staff multiple product teams. The change velocity DevOps teams can achieve is often far greater than SREs and SecOps can absorb, making information asymmetry worse.

    If teams can’t maintain high levels of collaboration and communication, another option must be developed. Observability practices, like collecting all events, metrics, traces and logs, allow SREs and SecOps teams to interrogate applications about their behavior without knowing which questions they want to ask ahead of time. However, observability only works if applications, and the infrastructure they rely on, are instrumented. This creates another problem: who does the instrumentation?

    The expectation is DevOps teams embed instrumentation into their code as part of the development process. While that’s a nice idea, there are four reasons this falls short. First, the quality of instrumentation varies. Many log statements are terse and only understandable by the developer that wrote them. The message “In function xyx123!” isn’t helpful to an SRE digging into a performance problem that cropped up in the latest release.

    Second, instrumentation libraries vary by implementation, giving inconsistent results across language bindings. OpenTelemetry is trying to improve this, but its progress is slow and still requires developers to do more work that, if we’re honest, doesn’t benefit them; it benefits SREs and SecOps. Now we’re back to those pesky mismatched incentives and information asymmetry problems again.

    The third problem with instrumentation is the volume of data. Each instrumented application can produce terabytes of data each day. When you have robust instrumentation, the amount of data can be overwhelming, and extremely costly to analyze and store.

    Lastly, instrumentation is isolated to the code your team wrote. That represents a fraction of the code you rely on. Vendor-provided services and APIs remain a black box, limiting your observability into those components.

    Resolving information asymmetry across teams requires two things:

    1. Pervasive, pluggable instrumentation capabilities for all code that doesn’t require developer involvement, and
    2. An observability pipeline to filter, redact and enrich data, then route it to your analytics platform of choice.

    Pervasive Instrumentation

    Operations teams need instrumentation without having to go back to developers and beg them to add it to existing code. They need to turn it on and off as needed, and they need readily consumable data. They also need every piece of data they can get, including packet payloads, insight into encrypted data and so on. This goes well beyond what’s possible with today’s instrumentation options.

    AppScope, a newly released open source project, is a new take on instrumentation. AppScope interposes itself between application threads and system libraries, tracking things like file system access, network and HTTP activity, as well as CPU and process activity. It also provides payload data, and because it sits between the application and encryption libraries, it also gives access to users’ cleartext data. Because it works with any Linux binary, SREs and operations teams can instrument anything, even code they didn’t write.

    Observability Pipeline

    The challenge with pervasive instrumentation is dealing with all the data generated by applications. Network, file system, and other system data can easily swamp destination APM and log analytics platforms, driving up licensing and infrastructure costs. While instrumentation data is vital to rebalancing risk sharing in organizations, you need a way to manage that data intelligently to get value out of it. This is where the observability pipeline comes in.

    An observability pipeline sits between the sources of observability data and its target destinations. By acting as a strategic control point, the pipeline gives users control over how data is formatted, filtered, enriched, and redacted before it is routed to one or more destinations.

    Observability pipelines help SREs and operations teams deal with the flood of instrumentation data by routing low value data to low-cost storage, like S3, while higher value information lands in APM and log analytics tools. Later, if you decide you need the data stored in S3 to add more context to your analysis, you can replay it back through the pipeline and enrich your data set.

    Shifting business priorities are accelerating the pace of technology deployment across organizations, and teams are pressured to deliver faster. This faster pace of delivery pushes deployment risk to operations and security teams that lack visibility into changes developers have made across complex distributed systems. This information asymmetry also increases risk to the business as applications become less predictable and reliable over time.

    Traditional methods of resolving the information mismatch haven’t worked because incentives across teams aren’t aligned. Adopting pervasive instrumentation and observability practices will give SREs and operations teams critical visibility into rapidly changing application and infrastructure environments, without disrupting the developer experience and process.

  • GitLab Acquires UnReview to Further AI Ambitions

    GitLab Acquires UnReview to Further AI Ambitions

    GitLab announced this week it has acquired UnReview, a provider of a tool that employs machine learning algorithms to identify which expert code reviewers to assign to a project based on both the quality of their previous efforts and current workloads.

    David DeSanto, senior director for product management at GitLab, said the acquisition of UnReview is the latest step in an AI strategy that, in addition to optimizing DevOps processes, will eventually unify machine learning operations (MLOps) and DevOps workflows.

    Accessed via the Dev section of the GitLab platform, UnReview will also be employed to manage the overall code review process.

    GitLab

    DeSanto said GitLab is committed to employing AI technologies to automate workflows and compressing cycle times across all stages of the DevSecOps life cycle. The goal is to not eliminate the need for DevOps teams, but rather eliminate low-level tasks that conspire to hamper productivity while at the same time improving application security, noted DeSanto.

    Providers of DevOps platforms are now locked into an AI arms race on two separate fronts. Each of them clearly sees the opportunity to leverage machine learning algorithms and other forms of AI to automate a wide range of processes. At the same time, it’s apparent providers of MLOps platforms are borrowing heavily from DevOps best practices to manage the development of AI models. However, from the perspective of a DevOps team, an AI model is just another type of software artifact. Rather than deploying two distinct platforms, the goal should be to bring the data scientists that build AI models into the larger DevOps community, said DeSanto.

    The current challenge, of course, is the pace at which DevOps and data science teams work. It may take six months or more for a data science team to construct and validate an AI model that needs to be embedded within an application. Aligning the timetable for an AI model with the application development life cycle is difficult to accomplish if neither team has any meaningful visibility into each other’s workflows. As the rate at which AI models can be developed steadily improves, that alignment issue will only become all the more pressing.

    Going forward, it’s hard to envision any application that won’t have some sort of AI model embedded within it. The issue is that, just like any other software artifact, an AI model needs to be updated. As new data sources become available, an AI model is likely to drift beyond the parameters intended. A rapid change in business conditions may also require the replacement of an entire AI model. The same processes that DevOps teams apply to update any software artifact can also be applied to an AI model.

    It’s unclear how long it will take before some type of convergence of DevOps and MLOps inevitably occurs, as organizations look for more efficient ways to manage what will become thousands of AI models. The one certain thing is the longer it takes to make the effort, the more difficult it will become to overcome the organizational inertia that has been allowed to set in.

  • Automating Security with Palo Alto

    Automating Security with Palo Alto

    Today, automation is more important than ever before. Integrating security and automation early into the SDLC decreases the risk of potential cyberattacks, provides early and relevant feedback, reduces time to market and improves product quality.

    In this TechStrong TV episode, Taylor Smith, senior product marketing manager with Prisma Cloud, Palo Alto Networks, discussed the need to take a software-centric and cloud-focused approach to security. He explained the importance of automating security to keep pace with rapidly changing applications and software infrastructure.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: Well, I’m very privileged to be joined by Taylor Smith. Taylor is Senior Product Marketing Manager with Prisma Cloud, of course, of Palo Alto Networks. Welcome, Taylor.

    Taylor Smith: Thanks so much.

    Ashley: Good to be joined by you today. Tell us a little bit about yourself. I know you fairly recently joined Palo Alto Networks and you have an interesting background in the industry, and of course, tell us a little bit about Prisma Cloud.

    Smith: Yeah, absolutely. So, I’m coming up on my six month mark and I’m really enjoying it. I came from both big and small companies, so previously at infrastructure companies and then I did a little stint in the DevOps space working on some DevOps tooling and chaos engineering. Really enjoyed that, but glad to be back into cyber security, which is really my passion, and covering cloud security and shift left security for Palo Alto Networks.

    Ashley: Excellent. Tell us a little bit about Prisma Cloud. I know it’s pretty comprehensive, a lot of things there, but how would you give a kind of brief overview of what it is?

    Smith: Yeah, absolutely. So, Prisma Cloud, we have the benefit of being best in class in multiple different categories in cloud security. So, we really cover the full life cycle of cloud security, from the infrastructure all the way up to the code level, the code that’s the applications that are running on top of it, and we do that across the full life cycle of your cloud application. So, we’re securing your infrastructure as code all the way to your container images, and we do that at the build time and the run time.

    Ashley: Excellent. Well, as a—I’ve used Palo Alto Network products myself. As a networking company, if you wanna think of it that way, maybe that’s an outdated way of thinking of Palo Alto Networks, which is the point of my kind of thought here is, we’re talking about application security and applications and really, everything is becoming software now. So, the network itself, of course, has been turned into software and now is in cloud and on prem. But the whole stack, the entire stack, you know, infrastructure as code.

    So, that brings some obvious benefits, maybe some challenges with it. What are your thoughts on that?

    Smith: Yeah, absolutely. So, it’s an interesting time to be in the application space right now. Before, you had to go in and physically provision servers, you had to connect cords to routers and network switches, and all of that and then maybe get into the CLI to make some changes.

    All of that’s moved to code, all the way from the infrastructure as code all the way up to the containers that you’re defining how they’re gonna operate, the operating systems are all contained in code. Which is great for developers, they’re able to move a lot faster, they can provision things on the fly, they can use things as they need them or scale them down if they don’t. It saves us costs and makes things move a lot faster.

    The double-edged sword to that is, now it’s out of the security and infrastructure, the IT team’s hands. Developers are just deploying things, it’s kinda the Wild West. And so, to kinda bring that in, rein that whole world in and make sure they’re secure and doing all the right things, it’s good to embed that security in their process. So, making sure you’re scanning in infrastructure as code templates, scanning container images before it gets to the repository and before it gets deployed into production. And that really gives us a chance to actually patch more often, because now developers are getting the direct feedback, and frankly, they’re more qualified to make those fixes than the security team. So, they can actually go into the code that they wrote and make the changes if they’re given that right feedback.

    Ashley: Yeah, I think that’s one of the things, as a security engineer when you think about software, it’s not—I can think of this now as, instead of a solid, it’s almost a fluid.

    Smith: Yeah. [Laughter]

    Ashley: It’s constantly changing, right? And that makes it really difficult to secure, it also sometimes can be difficult from a quality standpoint. But, really, it is almost that kind of a form of matter and the things are constantly changing. So, you can’t expect manual processes were stopping the line to be what’s going to bring security back into the fold. You’ve got to equip people, you’ve got to equip the toolchain, the pipeline that’s creating code and delivering it out to production, and then guiding that process through what you want secured, how you want it secured, what you deal with, those issues as they come up.

    Smith: Yeah, absolutely. Microservices has really revolutionized this where we have a couple customers who, the application that they have, one at the beginning of the week looks nothing like what it looks like at the end of the week. They’ve completely redeployed the full stack, and that’s great for innovation, it really opens up the world, but it does—and you don’t want, as a security engineer, to be going in there and slowing things down. You wanna be a part of that development and actually be a supporter, be someone who actually enables that speed to happen, but still make sure it’s done secure and you don’t get bitten by the misconfigurations, the vulnerabilities in your applications.

    Ashley: Mm-hmm. What is it about the cloud, too—I mean, we talked about microservices containers, those don’t have to be run on the cloud, but of course, we think of them as cloud native architecture. Is it from infrastructure management all the way through to the application itself? How do we have to rethink things than maybe what we did when everything was running in our own private data center? What’s fundamentally different that we might try to do it the old way in the cloud, but really, we’ve got to fundamentally change how you think about it?

    Smith: Yeah, I think a lot of it comes down to that everything is accessible. I mean, there are private links and there are VPNs to make sure that it’s locked down, but at the end of the day, you don’t have physical access to these data centers. So, a lot of it comes down to, there is somehow public access, and we lock that down as much as possible.

    But if you have an EC2 instance in AWS or a VM instance in Azure or GCP, it can be exposed to the world. And making sure that the security groups or the firewall rules are set so that they’re blocking, there’s no catch—I had a customer who explained it to me very well. Where you kinda get these layers of checks in the on prem world where, even if somebody goes in and provisions a virtual machine that is exposed and has all these misconfigurations, most likely, it’s not gonna be touching the world. No one’s gonna be able to access it, so it’s not a big deal. But cloud security, those misconfigurations can really bite you, because they are exposed. And so, getting that right is more critical, and making sure the misconfigurations are found and fixed, it’s much more important than it was in the on prem world.

    Ashley: That’s interesting. I’m wondering how you approach a platform like Prisma. Obviously, people who are already Palo Alto customers may come to it kinda naturally through their relationship with you. But how is it, is it a network engineering team that says, “I need to figure out how to secure the cloud?” Is it a software Dev team saying, “I need better tools to make sure I’m configuring and writing secure code?” Who is it that usually is engaging you for Prisma and then how do they bring it into the organization and kind of spread the love, if you will? [Laughter]

    Smith: Yeah, yeah. No, it’s fun. Part of what I really enjoy about being at Prisma versus some of my previous jobs is, I’m not working with a single person or archetype, I’m working with some cloud engineers, who they, it’s this new—frankly, new to me engineering type where they’re covering everything cloud. They’re doing full stack up to the application for everything provisioning infrastructure in the cloud. Where before, you might have a network specialist and a compute specialist, now you have a single engineer that covers the full cloud. And then we also have developers and DevOps teams that work with us, and so it’s not necessarily the security teams. And then, we also still work with the security teams, so the SOC, the security engineers and the cloud security specialist.

    And all of them have different us cases that they come into Prisma Cloud with, but all of them in the end are trying to use Prisma Cloud to make their lives a lot easier to automate that security and make sure the checks are happening. So, the developers are much more worried about the pre-deploy time where they can secure their applications, their infrastructure as code when they’re container images before they’re actually running containers.

    And then, the security teams are actually going into our system to see what’s the posture in the cloud, getting that view, the complete, comprehensive view of all the things that are running in their cloud environments, and then securing their containers, their Kubernetes environments, their serverless environments, all the things that are actually running. And running incident management through Prisma Cloud, running through the incidents and all the trace data that we give them if there’s an incident that happens.

    Ashley: It’s really fascinating how the security engineer/network engineer job has changed, that role and the skills that are involved. I wonder, I’m just curious what your experience is being more, dealing with software and configuration of software, maybe automating, scripting, things like that as a network engineer/security engineer. It seems like that would make that job more relatable to what a software developer might be doing. They’re not the same thing, but much different than someone who is rack and stacking and, as you mentioned, plugging things in and thinking of networks and physical devices like we used to, not that long ago.

    Smith: Yeah. Yeah, it’s funny, I agree. The role of the infrastructure engineer is becoming much more like a software developer. So, they’re actually learning Python or HCL so they can write Terraform scripts or they can write Ansible scripts and deploy infrastructure using code. And that can now go through pipelines. So, now, I’m hearing infrastructure people start talking about CI/CD, which you would never expect in the on prem world. They’re using a lot of the same language, like unit testing static analysis of their infrastructure as code.

    So, they’re kind of learning to be a developer, but specifically for infrastructure.

    Ashley: I remember the day, it was actually about, I guess, 11 years ago when I took over an IT organization and a network engineer asked me, “What should I be learning? What’s the next thing I should be thinking about?” and I said, “Learn Python,” and he looked at me cross-eyed.

    Smith: [Laughter]

    Ashley: Ended up going to a conference, maybe it was yours, came back and said, “Now I know why you said that, now that we see what’s happening in”—yeah, that’s a huge transformation that’s happened in, really, less than a decade. Think about how massively the network stack has changed and really folded into infrastructure as code as its part of it.

    How has your job changed through your career, and have you progressed with how this has happened in our industry?

    Smith: Yeah, so, like I said, I came from a big infrastructure company and I covered traditional network security back then, and I moved to a DevOps company. And so, I kinda saw this complete transformation where things were automated, people were moving fast. Reliability is kind of—it’s a whole new definition in the cloud. And so, now, you can, even small companies can have redundancy where they didn’t before.

    That makes things very different for the security person. And so, if I’m—now that I’m covering security, again, but now from a more developer focus, I’m seeing how that automation is much more important than ever before, and how that network perimeter is no longer the only place where you can apply security. And it’s becoming less and less important, but it’s still critical to have that perimeter, but all the things happening inside, too. So, that zero trust networking inside your infrastructure, inside your applications, securing your Kubernetes deployments, making sure your containers aren’t over privileged or you’re not pulling in images from open source that are vulnerable.

    All that is now just as important or more important than the traditional security that I kinda started my career with.

    Ashley: Mm-hmm. Yeah, it’s a fascinating journey we’ve been on.

    Smith: Absolutely.

    Ashley: Well, we’re kinda running out of time, here. Where can folks find out more about Prisma Cloud and offerings from Palo Alto Networks?

    Smith: Yeah, absolutely, you can check out our website. So, you can go to PaloAltoNetworks.com/Prisma and you can find out a lot more there, or we have a lot of useful resources like blogs or white papers, and so, if you want to find out more about container security or cloud security, go there, and there’s a lot more information for you.

    Ashley: I was just gonna suggest, right, developers as well as security and network folks, if we wanna say whatever your role or title might be. Well, it’s been fascinating talking with you, Taylor, and good to be on this journey with you through the evolution of the technology that we use and innovation that’s happening.

    So, hope you are doing well and you’ll come back and talk to us again.

    Smith: Thanks so much, Mitch. It was a pleasure.

    Ashley: You bet.

  • 7 Step Transformation Blueprint for SecDevOps

    7 Step Transformation Blueprint for SecDevOps

    The struggle for continuous software security is obvious, but the solutions are not.

    As I indicated in my prior blog, SecDevOps is the Solution to Cybersecurity, a security-first mindset, coupled with SecDevOps-specific practices, provides an opportunity to achieve true continuous security. But, in reality, how can an organization accomplish SecDevOps?

    This blog explains how to apply my 7 Step Transformation Blueprint to SecDevOps.

    Leadership and engineering solutions require the persistent, methodical application of skills and practices toward leading and designing solutions that achieve business and team goals, including continuous security. While driven by visionary ideals, engineering requires practical, disciplined, progressively refined implementations using carefully chosen dimensions of people, process and technology solutions. At any point in the engineering life cycle, the goal is to achieve a balanced solution while evolving practices towards maturity.

    The seven step transformation engineering blueprint prescribes seven steps for achieving and continuously refining digital transformation methodically, no matter what the goals or current level of maturity are. The seven steps are visioning, alignment, assessment, solution, realize, operationalize and expansion. Each step considers the people, process and technology aspects of the transformation.

    Step One – Visioning

    Top leaders define a strategic vision for the digital transformation for the organization including a motivating vision statement, measurable goals, team values, and major implementation tactics. Identify senior sponsors that will own the transformation at the strategic level. Include key partner organizations that need to be strategically aligned to the transformation. In a SecDevOps transformation, a vision for a security-first mindset and associated SecDevOps practices are called out as the highest priority for implementation tactics supporting the vision.

    Step Two – Alignment

    Leaders and key team members who are most important to the implementation of the transformation align specific measurable goals and tactics for selected “model” applications. Specific measurable goals around continuous security are set in this step.

    Step Three – Assessment

    For the current state of selected applications, capabilities are discovered and assessed, deep-dive assessments are conducted for specific topics, and a current state value stream map is created relative to the organization’s goals. My earlier blog, DevSecOps Practices Gap Assessment, explains my recommended approach for conducting an assessment for security.

    Step Four – Solution

    An expert team performs analysis of assessment data and formulates a future state value stream roadmap including themes, epics and user stories and obtains alignment with key stakeholders. My earlier blog, 9 Pillars of Continuous Security Best Practices, outlines a comprehensive set of practices to consider when building a roadmap for any continuous security solution.

    Step Five – Realize

    Proof of concept (POC) trials are conducted to validate solution choices. Trials of security tools and integrations of those tools into the SecDevOps platforms also would be conducted during this step. The solution is validated with selected applications and use cases. Training is conducted as the solution is deployed to production. Governance practices for the new solution are activated.

    Step Six – Operationalize

    Deployed improvements are monitored and controlled with metrics. Retrospectives are conducted to create actionable prioritized lessons learned for continuous improvement. Chris Tozzi’s article 6 DevSecOps Metrics for DevOps and Security Teams to Share suggested metrics that can be developed and leveraged, both for this step and to drive ongoing improvements as the use of SecDevOps practices expand.

    Step Seven – Expansion

    Once continuous flow (the first way of DevOps) is realized for a select set of applications, the organization can safely expand the solution(s) to other applications across the organization. Further transformation cycles will lead to realization of continuous feedback (the second way of DevOps) and continuous improvement (the third way of DevOps) and apply it to SecDevOps.

    What This Means

    SecDevOps strategies and solutions are complex. The seven step transformation blueprint described in this blog can help organizations build a strategy and implement SecDevOps as an important part of their digital transformation. To learn more about how to apply the blueprint to continuous security (and other key elements of digital transformation) refer to my book Engineering DevOps.

  • Serverless Computing Brings New Security Risks

    Serverless Computing Brings New Security Risks

    Attracted by lower costs and less operational overhead, serverless computing is an unmistakable undercurrent in the world of DevOps. Developers are drawn to the innovation because it requires no architecture to manage while offering continuous scaling for anything from a few requests per day to hundreds of thousands of requests per second.

    More than 50% of the developers using serverless functions today started using the computing model in the last three years. By some estimates, serverless computing is the fastest-growing segment in the cloud services market.

    While businesses are drawn by the opportunity to extend and integrate cloud applications at greater scale, there’s a dark underbelly to this DevOps phenomenon: security. In fact, organizations cite security concerns as the leading reason why they haven’t yet adopted serverless computing. For good reason!

    Serverless functions are introducing a new breed of threats – misconfigurations, code-level risks, injections and weaknesses – many of which are the underlying risk factors for a software supply chain attack. With adoption of this technology accelerating, organizations need security that delivers more than just visibility into these dynamic environments. They also need the ability to detect and block code-level vulnerabilities and misconfigurations that occur during development and through to production. Most importantly, this needs to happen seamlessly and with little to no impact on DevOps.

    Dynamic Environments Create Complex Security Risks

    The nature of applications has evolved, creating a new paradigm of security risk. In serverless environments, code sprawl makes it harder to identify vulnerabilities and patch them before they become business-level risks.

    In a monolithic application environment, an identified vulnerability could be efficiently patched by the developer through upgrading a single package, pushing a deployment and then testing and deploying the app.

    In a serverless computing model, patching and resolving vulnerabilities becomes a lot more complicated, given that there may be several hundred (or more) environments across different providers and regions. So, instead of having to patch a single application, developers will need to patch, deploy and test hundreds of individual functions that rely on that library.

    With 39% of developers saying they’re now fully responsible for security, it means they’ll carry the burden of having to patch an identified vulnerability – often taking them away from development work.

    With Serverless, You Need to Rethink Your Approach to Security

    Given the distributed nature of serverless functions – essentially, the reason for its flexibility and scalability – many existing security tools will provide little to no visibility, nor the control capabilities, for these computing environments. Many of the security attacks that will occur in serverless functions will be a result of misconfigurations and mistakes that happen outside the purview of the security team and due to legacy solutions which don’t translate to serverless architectures. Further, because abstracted workloads create blind spots, attackers will have more room to maneuver undetected.

    Serverless functions will even render some traditional DevSecOps tools less useful. Scanning tools must monitor hundreds of individual repositories instead of a single monolithic repository, while application performance monitoring (APM) tools lack security proficiency and cannot protect from the OWASP Serverless Top 10 risks.

    The challenge for security teams will be implementing solutions in a way that does not modify workloads or require changing business logic — both of which slow development. When applying security to serverless functions, it’ll be critical that the solution can do the following:

    • Go beyond monitoring and actually detect and block vulnerabilities without elaborate or manual steps involved. The scale of serverless functions will make it nearly impossible for security analysts to manage and keep up with every alert they receive. Instead, they need a tool that delivers actual security and can help them block malicious activity without stopping development altogether.
    • Enable a positive security model that provides protection against malicious changes, like zero-day exploits, within the function. This approach gives security teams more control by automatically detecting and blocking anything that isn’t explicitly allowed in a given function.
    • Provide forensic detail from log-level information so security analysts can fully understand the context of every attack with virtually no impact on latency. Further, with the risk of supply chain attacks soaring, security teams need the ability to identify and map third-party dependencies used during runtime.

    For many organizations, serverless architecture is a very different and unique computing environment – unlike anything they’ve experienced or had to protect before now. That reality means that organizations need a fresh approach to securing these environments and will need to look beyond the traditional tools they have in their tech stack today.

    Above all, developers and security teams will need to work together to ensure security is built-in from the start. Otherwise, serverless functions will quickly become a playground for cybercriminals.

  • How to Move Fast Without Breaking Security

    How to Move Fast Without Breaking Security

    “Move fast and break things,” is a familiar motto. The phrase, attributed to Facebook CEO Mark Zuckerberg, helps to explain the company’s stellar growth over the past decade, driven by its product innovations. However, while it’s a useful philosophy for software development, moving faster than you’d planned is a risky approach in other areas, as organizations globally realized during the COVID-19 pandemic. While 2020 saw digital transformation program timelines advance by up to seven years, enterprises’ quick moves to the cloud also meant that some things got damaged along the way – including security.

    Our recent survey conducted with the Cloud Security Alliance showed that more than half of organizations are now running over 41% of their workloads in public clouds, compared to just one quarter in 2019, and this will increase further by the end of 2021. Enterprises are moving fast to the cloud, but they are also finding that things are getting broken during this process.

    The research showed 11% of organizations reported a cloud security incident in the past year, with the three most common causes being cloud provider issues (26%), security misconfigurations (22%) and attacks such as denial of service exploits (20%). In terms of the business impact of these disruptive cloud outages, 24% said it took up to three hours to restore operations, and for 26% it took over half a day.

    As a result, it’s no surprise that organizations have significant concerns about enforcing and managing security in the cloud. Their leading concerns were maintaining overall network security, a lack of cloud expertise, problems when migrating workloads to the cloud and insufficient staff to manage their expanded cloud environments. So, what are the root causes of these cloud security concerns and challenges, and how should enterprises address them?

    Confusion Over Cloud Control

    When asked about which aspects of security worried them most when running applications in public clouds, respondents overwhelmingly cited getting clear visibility of topologies and policies for the entire hybrid network estate, followed by the ability to detect risks and misconfigurations.

    A key reason for these concerns is that organizations are using a range of different controls to manage cloud security as part of their application orchestration; 52% use cloud-native tools and 50% reported using orchestration and configuration management tools such as Ansible, Chef and Puppet. However, nearly a third (29%) said they use manual processes to manage cloud security.

    In addition, there’s competition for overall control over cloud security: 35% of respondents said their security operations team managed cloud security, followed by the cloud team (18%) and IT operations (16%). Other teams, such as network operations, DevOps and application owners, all figured in, too. Having different teams using multiple different controls for security limits overall visibility across the hybrid cloud environment, and also adds significant complexity and management overhead to security processes. Any time you need to make a change, you need to duplicate the work across each of these different controls and teams. This results in security holes and the types of misconfiguration-based incidents and outages we mentioned earlier.

    How to Move Fast and Not Break Things

    So how can organizations address these security and management issues, and get consistent control over their cloud and on-premises environments, so they can take full advantage of cloud agility and scalability without compromising on security? Here are the four key steps:

    Get holistic visibility across your networks: a lack of visibility across cloud estates was the main security challenge for survey respondents. Organizations need to get holistic visibility across all of their different cloud accounts, including the security controls that are being used both in the cloud and to manage traffic to and from the cloud environments. This enables IT and security teams to quickly identify any potential security issues or risks that could lead to outages.

    One management console to rule them all: When using a mix of the cloud providers’ security controls as well as other controls, both in the cloud and on-premises, it’s a huge challenge for organizations to manage policies consistently. If all these diverse security controls are managed from a single console using a single set of commands and syntax, security policies can be applied consistently. This avoids duplication of effort and the error-prone manual processes that lead to misconfigurations and outages. It also provides a full audit trail of every change.

    Automate security and compliance processes for speed: Automation is essential to managing cloud environments efficiently while orchestrating changes across a complex array of security controls. It brings speed and accuracy to managing security changes across cloud environments. It also accelerates audit preparation and helps ensure continuous compliance. Automation also helps organizations overcome skills gaps and staffing limitations, which were cited as a challenge by over three-quarters of our survey respondents.

    Accelerate incident response, anywhere: Our survey also showed organizations are concerned about overall network security in the cloud. Security management solutions that integrate with SIEM or SOAR tools can help to address these concerns by accelerating incident response. As soon as a threat is detected by the SIEM or SOAR product, the management solution can identify all the applications and servers that the threat could impact. The solution can then mitigate the threat’s risk by automatically isolating any affected servers or devices from the network – whether in the cloud or on-premises – preventing lateral movement.

    With a network security automation solution handling these steps, organizations can get holistic, single-console security management across all of their public cloud accounts, as well as their private cloud and on-premises deployments. This helps them to solve the cloud complexity challenge and ensures faster, safer and more compliant cloud management – making it possible for organizations to move fast in response to changing business needs without breaking things.

  • Adapting Security with Menlo Security

    Adapting Security with Menlo Security

    Cloud data security is top of mind for CISOs. User data is moving across clouds, platforms and applications, and if we don’t shift to a data-centric approach, we won’t be able to protect that data.

    Jack Miller, head of professional services at Menlo Security and former CISO of over 16 years, addresses the balance of data, security and end-user convenience. Locating security close to users, close to data, and both are essential strategies for adapting security to the expanding presence of data.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: Well, I have the great pleasure of being joined by Jack Miller. Jack is Had of Professional Services, and also a former CISO at a different company, and he’s with Menlo Security. Jack, great to be joined by you today.

    Jack Miller: Thanks, Mitch, it’s great to be here. Yeah, before joining Menlo, I spent most of my career on the other side of the table as a practitioner. I was actually CISO at four different multi-billion dollar organizations over a span of 16 years. Been in the trenches for a long time, fought a lot of the battles—won some, lost a lot, and here I am today.

    Ashley: Such is the career in security, right?

    Miller: Yes, it is.

    Ashley: [Laughter] No doubt, no doubt. It’s a great pleasure to be talking with you. So, there’s a lot of things, of course, we could talk about, Menlo’s a great company, got some fantastic offerings. 

    What I’d like to really spend some time on is data. You know, we think about—I like to say there’s three reasons any application development team outside of IT is gonna come to IT. Security is one, and access to data and integration with systems.

    Miller: Mm-hmm.

    Ashley: So, let’s talk about data in the cloud, or what are some things we should consider maybe that we didn’t have to worry about in terms of securing data, but also not getting in the way with security for end user usability in the cloud.

    Miller: Yeah. I mean, look, I think the last point you touched on I wanna kinda drill into a little bit, because I think it’s so significant, right? That, you know, no matter how vested in security an organization is, right, how high they prioritize it at the very top, how much money they’re willing to spend, right? At the end of the day, it comes down to a battle between security and potential impact to your users and your business operations and business processes, right? 

    And when you think about it, I mean, companies and organizations don’t exist to be secure, right? They exist to make money, they exist to provide a needed service, right? So, at the end of the day, that’s what’s really most important. And so, you know, as the user data starts to move everywhere and it becomes in different places and now, we have the users moving everywhere. 

    So, we went from a really nice little model in the past where we had our users and our data in the same place most of the time, and now we’re in a model where our users are everywhere and our data is everywhere. And if we don’t shift our approach from a data center approach that we used to have to a data-centric approach, then there’s no way that we’re gonna be able to properly protect the data and do it in a way that’s not gonna be extremely, a difficult situation for the users to access the data, right? We need to make it fast and streamlined and accessible and once there’s a hiccup in their way, they complain and then we have to start turning our controls off, right?

    So, it’s a fine dance that we have to walk, and unfortunately, the legacy tools that we have and fitting into that data-centric approach just don’t really seem to work well in this new model.

    Ashley: That’s always the struggle with security is the convenience factor. Don’t get in the way of my job, but make sure that it’s secure. Don’t get me in trouble for having a breach or something happening.

    So, what are some approaches? Why do we have to think about this where we really can secure the data, wherever it is, and we’re creating more and more of it all the time, right? It’s a huge asset, but growing faster and faster. How do we do that where we can provide end users a level of convenience and security?

    Miller: Yeah, I mean, fortunately, today, we have the cloud, and for all the, as a security professional, for all the heartache that the cloud’s brought me over the years and all the challenges that it does introduce, it dos create this new opportunity now where I can, instead of having to have physical controls located somewhere, I can have virtual controls that are logically located all over the world, right? So, now, I can make sure those controls are close to the users or close to the data or close to both, whatever happens to make sense for any given business us case that we’re dealing with.

    Ashley: Yeah, that makes a lot of sense, too. And also, I don’t know if automation is the right word, but the manual processes we’ve had in the past of providing access control, whether it be for applications or end users, it’s not realistic with how far and wide we’ve spread data and where it’s all located.

    Miller: Yeah, and everything’s so dynamic that even if we could get our hands around it today, by tomorrow, we’re gonna be playing catch up again, right? You know, the cloud has made it really easy for things like shadow IT to flourish, right? So, now, business units can go out and they can add new apps.

    So, we spent years in the enterprise putting in robust two-factor authentication solutions to protect our data. Now, suddenly, even if we go out and we do an analysis and we find all of our SaaS apps today and we protect them, what happens tomorrow when another business unit just happens to go out and start using a new SaaS app and they don’t tell you about it?

    So, it is a struggle, right? But like I said, there’s a silver lining to it in that the same thing that’s making it difficult for us does give us an advantage to be a little bit more ubiquitous, I think, maybe would be a good word and where and how we can apply security.

    Ashley: Mm-hmm. How do you think—and I know, you know, Menlo Security, obviously, being a security product company, but if you thought more generally, more broadly, has the market kept up with the uses and the needs of securing data in the cloud in particular, or we still have a pretty big gap to meet?

    Miller: So, I think there’s kinda two parts to that, right? I think from a capabilities perspective, I think that we’re closing the gap, and I think significantly more capabilities exist today than existed a couple years ago to be able to do that, and we’re making, I think, huge strides and great progress there.

    From an adoption perspective of getting these new capabilities and technologies out there and adopted with companies, right, that’s where we have the gap today. It’s always hard to get people to embrace new technologies and accept new technologies. And we’ve been talking about digital transformation for years and years, and it’s been very, very slow, the transition. And a lot of what we were seeing pre-COVID was being done to really save costs associated with remote locations, right, and eliminate those expensive MPLS circuits, eliminate redundant hardware and that type of stuff.

    You know, now you throw COVID on top where you have, suddenly, all these remote users being remote that were never remote before. So, now, looking at using the cloud and direct access and things like that to kinda solve that same problem but coming at it from a different direction, right?

    So, I think what I find interesting right now is that you hear a lot of people talk that this is the new normal, that we’ve proved that remote work can work and everybody’s gonna keep working remotely. And I don’t think this, today, is the new normal. I think that many companies are gonna have their employees come back to the office, for a number of reasons. But I think it’s gonna be very different than it was in the past.

    I think now, those areas in the organization that weren’t allowed to work from home a couple days a week are gonna be allowed to, that weren’t allowed to be able to travel and be able to work are gonna be allowed to.

    So, I think, you know, we kinda went from this everybody was in the office to now everybody’s at home, and I think now we’re gonna go back to, it’s almost a—who knows where everybody is, because they can do their work from wherever they wanna be.

    Ashley: You’ve sort of disproven whatever maybe-ness, maybe partial truths that, you know, “You can’t do this work remotely.” Well, we had to, we did.

    Miller: Right.

    Ashley: You know, what’s interesting, I think a lot of folks are thinking of a hybrid model, even people that have employees come back to the office is, “Yes, but maybe one or two days of work, they can very easily work remotely.” So, maybe that’s a closer step in the new normal that we’re, at least for the next couple of years gonna be experiencing, and who knows what happens after that? [Laughter] 

    Miller: Yeah, exactly, exactly. No, I think that’s where we’ll land. I think it makes perfect sense. I think, from a productivity perspective, that’s probably the best path forward. And I think it’s a path we can secure, as long as we’re adapting our approach, right, and we’re focusing on where the data is and securing the data as opposed to, again, not being focused in the traditional of where the office is or where the data center is.

    Ashley: Now, I’m curious your thoughts about this. I’m sure others are thinking about it, I just haven’t heard it a lot in the kinda open press or writings. With so many people mobile—of course, if you would’ve had time to plan it, you might have done some things differently, but thinking forward, if we are gonna live in this hybrid world, there’s probably information, some data that you want to, telemetry information you wanna track about how people are using your applications, where they’re using them from. Maybe the same, of course, applies to customers, if they’re more mobile, also, and yet more data that we’re generating and creating to target this, you know, broad ecosystem of data that we’re trying to collect. 

    And now, so much of that data is going back into how do we create a better experience for employees, how do we create a better experience, better services for our customers? So, all of that data is just as important as the credit card transaction that goes through—well, maybe that’s a little more important, but it’s all important that we can share that. So, the challenge, to me, seems like it’s getting bigger, not smaller.

    Miller: Yeah, I think it’s getting bigger, and I think we’re seeing—it’s been a while in progress, right, but I think we’re seeing a shift to more of the ransomware type of attacks. I think that it’s easier for the bad guys to quickly monetize their code with a ransomware attack with less chance of getting caught. And plus, one might argue that, you know, we talk about herd immunity for COVID and things like that. From a data privacy perspective, maybe we’ve reached herd immunity because so many people’s data’s already gotten stolen, right?

    So, at some point, there’s so much personal information out there that the data itself starts to become devalued, right? But hey, just like the battle we fight as security professionals that I can’t impact my users, I can’t impact my business operations, right, the bad guys are teeing in onto that, saying, “Look, that’s the most critical thing. If we can get in there and threaten to be able to impact those, then that’s a very viable path for them that they keep following right now.”

    Ashley: Yeah, it’s the whole business continuity, right? That’s what the ransom is about.

    Miller: Exactly, exactly.

    Ashley: Using your data for some other purposes, which can happen, also, as well. I’m really curious, just in your personal story as a former CISO, both organizations large corporations—what are some of the things that you were bringing into the role in Menlo Security that you’re hoping to help? I’m sure you have that list of things, “If I could’ve had a product that did this” or some of those problems that you’d love to help Menlo and the broader community solve.

    Miller: Yeah, I mean—so, look, let me kinda break that into two parts, right? Because there was a process that I went through when I decided to switch teams, if you will, right, and go from being a CISO to working the vendor space. And a lot of people ask me that question all the time, “Why did you leave being a CISO?” Right? That’s the role everyone wants to get to today—‘til they get there for a while, and then they wanna leave. But, you know, there are a number of—

    Ashley: ____ future that you wish for. [Laughter] 

    Miller: Exactly. But, you know, one of the big drivers was that, you know, what becomes very clear to me over the years was that most security companies, the vendors, they don’t have a lot of people there that have worked on the customer side before as the practitioners. And so, they don’t really understand the real world in which a company lives and operates, right?

    And so, you know, they don’t understand this balance between security and impacting your users. And what that means for you as a CISO, with how far back it can set you—I mean, a story. I’d implemented a large scanning, vulnerability scanning system, a huge vulnerability scanning system. And somebody did a scan one night and his server that was running an old, vulnerable version of WebSphere, and it knocked the server offline. You know, we didn’t get medals because we found these vulnerability before the bad guys did; instead, we had to stop all scanning and we had to redevelop our process so that we could ensure that if we ever took a system down scanning that it wouldn’t impact production. I’m not gonna say that we shouldn’t have done that in the first place and that’s not the right way to do it, but again, it just kinda highlights this one step forward, two steps back, right?

    Another example where you see this gap is, ask any security practitioner, right, about IDS/IPS, and they’ll say IDS. Ask any security vendor out there, and they’re all IPS, IPS, IPS, right? But the practitioners know I’m not gonna turn on prevention mode, because the collateral damage to me and my program and what I’m trying to accomplish is too great, right?

    So, that was really kind of one of the big things I wanted to bring here. And I feel like heading up professional services is an opportunity for me to do that, because that’s when we’re really working with the customers and we’re trying to get them deployed. And by having that empathy and showing the customers we understand what’s really important to you and we’re not just trying to force a level of security down you, we’re trying to help you be secure but be successful at the same time, I think there’s value there. And that was one of the big reasons why I ended up coming here and what I’m trying to do with Menlo.

    You know, the other part of the answer would be from a technology perspective. I think from a technology perspective—and I don’t wanna make this sound like a company pitch here, right, but isolation can provide such a level of value that really, at the end of the day, nothing else can. And I think I’ve always been a big fan of isolation, going back to the very beginning when there were some companies that maybe tried to bite off too much and isolate too many things and that didn’t really work well, right? So, by narrowing down the surface and focusing on what really matters, which is web traffic, isolation can, for whatever connections you can isolate, you can virtually remove your risk, which is unheard of in the security field. 

    But, you know, it’s hard, it’s not easy. There’s a hurdle to get there during deployment, and if we’re not understanding those challenges for the customers, then, you know, as a vendor, we can start pushing them too hard and we can push them into trouble without intentionally wanting to do that.

    Ashley: Yeah, your experience, you and I have that shared experience, because I’ve been both on the practitioner and the CTO profit company side of it as well. It seems like one of the things that I would guess you would bring to this is, you know what it’s like to try to use products maybe similar, maybe the same as what you’re working with, with Menlo. And you’re immediately gonna have a very good feel for what’s gonna work well for the customer, what’s gonna work and what’s not, because you can very quickly assess where they are and what they’re going through, maybe some similar things, maybe different, but you’ve been in very similar shoes. 

    So, you can help bring something to the table to them that is much more compatible. You know the balance of security versus user experiences versus available versus available versus whatever, you know, reporting to the board—all that stuff, all those issues. And you can’t teach that to someone, so that’s a unique skill—unique value, I should say, that you bring. 

    Miller: Yeah, well, and, you know, given the options, right? So, interacting with someone like you and being able to—you know, again, a lot of companies want to kinda sugarcoat things. And, you know, as a CTO, right, I’m sure you would agree any day of the week, you’d rather have someone tell it to you straight so you can properly prepare than have someone kinda just make things sound like they’re easier than they are, right? And—

    Ashley: The worst problem is the one you don’t know about.

    Miller: Right. And so, you know, being straightforward with people and being—you know, oversharing the information and giving them options, right? I mean, at the end of the day, we’re the experts in our technology. So, it wouldn’t make sense for us to ask you, “Well, how do you want this done?” but for us to be able to come to you and say, “Look, you know, here’s a few different options and here’s the pros and cons, right? We don’t totally know all the intricacies of your unique situation, but you know, maybe it kinda fits in with one of these, and what do you think makes sense for you to proceed forward?”

    Ashley: Can you tell us a little bit about some of the professional services that you do offer through Menlo?

    Miller: Well, I mean, largely, it’s deployment services, right? So, we have a deployment service process we call our QuickStart Deployment service. You know, one of the main tenets is really, we gotta help customers be successful, and a lot of customers, they get a little bit too excited, they try to run before they can walk. 

    And so, by putting together, really, a defined process that slowly walks you through it allows you to get your users onboarded, start getting some level of value and security from it so that when you start moving into the harder areas, where you might start impacting some people, you’ve got some good ammunition of value to be able to offset the conversation. Whereas, you know, a lot of customers just wanna run right out the gate and turn everything on full bore. And some of them won’t even do that and start with doing it with their executives, right? So, a lot of—[Laughter] I know how crazy that sounds.

    Ashley: Good luck with that. [Laughter] 

    Miller: Right, so, a lot of what it’s doing is trying to save the customers from themselves, right? They get excited, they see what it can do for them, pull the back into reality, let them know—look, here’s the pitfalls, here’s one you’re not gonna be able to avoid, but here’s how we can minimize it, right, the time to resolution, and here’s others that maybe you can avoid if we take some proper steps up front.

    Ashley: Good. I’m sure it’ll be very valuable to Menlo’s customers. And folks can find out more about professional services on MenloSecurity.com, I assume, correct?

    Miller: Yep, definitely.

    Ashley: Great. Well, it’s been a pleasure talking with you, Jack. I wish you the best, and I’m sure it’ll be a lot of fun working with customers in your role there at Menlo, so thanks for joining me today.

    Miller: Well, thank you very much, Mitch, I enjoyed it a lot.

    Ashley: You bet.

  • The Importance of NetSecOps in Network Management

    The Importance of NetSecOps in Network Management

    The IT landscape has changed beyond recognition in the past decade or so. The vast majority of businesses now operate largely in the cloud, which has had a notable impact on their agility and productivity. A recent survey of 1,900 IT and security professionals found that 41% of organizations are running more of their workloads in public clouds when compared to just one quarter of respondents in 2019. Even businesses that were not digitally mature enough to take full advantage of the cloud will have dramatically altered their strategies in order to support remote working at scale during the COVID-19 pandemic.

    However, with cloud innovation so high on the boardroom agenda, security is often left lagging behind, creating a vulnerability gap that businesses can little afford in the current heightened risk landscape. The same survey found the leading concern about cloud adoption was network security (58%).

    Managing organizations’ networks and their security should go hand-in-hand, but, as reflected in the survey, there’s no clear ownership of public cloud security. Responsibility is scattered across SecOps, NOCs and DevOps, and these teams don’t consistently collaborate in a way that aligns with business interests. We know from experience that this siloed approach hurts security, so what should businesses do about it? How can they bridge the gap between NetOps and SecOps to keep their network assets secure and prevent missteps?

    Building a Case for NetSecOps

    Today’s digital infrastructure demands the collaboration – perhaps even the convergence – of NetOps and SecOps to achieve maximum security and productivity. While the majority of businesses do have open communication channels between the two departments, there is still a large proportion of network and security teams working in isolation. This creates unnecessary friction, which can be problematic for services-based businesses that are trying to deliver the best possible end user experience.

    The reality is that NetOps and SecOps share several commonalities. They are both responsible for critical aspects of a business and have to navigate constantly evolving environments, often under extremely restrictive conditions. Agility is particularly important for security teams to keep pace with emerging technologies, yet deployments are often stalled or abandoned at the implementation phase due to misconfigurations or poor execution. As enterprises continue to deploy software-defined networks and public cloud architecture, security has become even more important to the network team, which is why this convergence needs to happen sooner rather than later.

    We somehow need to insert the network security element into the NetOps pipeline and seamlessly make it just another step in the process. If we could automatically check whether network connectivity is already enabled as part of the pre-delivery testing phase, that could, at least, save us the heartache of deploying something that won’t work. And then, if we do need to open new ports, an automated process that could also raise a change request would save even more time. Better yet, if it’s one of those “make this like that” changes, the change could be automatically implemented on the firewalls – in minutes, not weeks.

    With NetSecOps, application developers can get creative without worrying about security slowing them down, and network security teams can sleep soundly, knowing that risk and compliance checks are built in to the continuous delivery process with a full audit trail, so they can even get time back to do some fun stuff.

    If we could do all this with only the rare cases of new connectivity requiring approval by security, the NetSecOps dream could become a reality, allowing fast, high-quality delivery. With NetOps and SecOps working together, organizations can establish a collective approach to network management and security, offering equal opportunity for both teams to present and prioritize issues and agree on the fundamental rules from the start; a match made in heaven.

  • Prevent False Positives From Derailing Shift Left

    Prevent False Positives From Derailing Shift Left

    Static application security testing (SAST) tools are designed to balance false positives (incorrect warnings) with false negatives (missed vulnerabilities) primarily because deeper analysis requires more time and computing resources. Both of these are in short supply among developers that are tasked with meeting shorter and shorter product delivery milestones.

    So, while SAST vendors consider a true positive a correct detection of a real defect, and a false positive an alert on a bug that does not exist, this is rarely the lens used by developers. What really matters to them is whether the output of a SAST tool is useful and actionable. For example, there is a great deal of variation in how developers interpret SAST scan results, depending on the nature of the defect, the role of the user, the platform on which the application will run and the environment in which it is deployed.

    Take, for example, a true positive result of a buffer overrun, one of the most notorious classes of C/C++ defects from a security perspective. In the early stages of application development, it almost always makes sense to change the code to fix such a bug. Developers are actively changing the code, anyway, so fixing it involves little extra overhead. However, if the same defect is found after the application has been deployed, then it is much trickier to decide whether it is worth fixing.

    That’s because the cost of fixing a real bug may exceed the benefit of fixing it, and the benefit of “correcting” a false positive may exceed the cost of leaving it alone. Since SAST tools can only be relied on to give narrow technical answers, humans must interpret and determine which static analysis results should be acted upon.

    Consider a hypothetical comparison of different SAST tools in the graph below. Tool A has good recall (i.e., the ability to identify real defects) and precision (i.e., the ability to exclude false positives) which results in finding many of the real bugs with a reasonable amount of false positives. Tool B has high precision but poor recall, resulting in low false positives but a higher number of false negatives (undetected security vulnerabilities). Tool C has poor precision but high recall, resulting in detecting all the possible errors but also with a very high number of false positives.

    A comparison of recall and precision of hypothetical SAST tools.

    Developers hate false positives because they introduce unnecessary work and delays. This, in turn, has a disproportionate effect on the way tools are designed, configured and used. If given a choice between Tool A that reports 40 real defects and 10 false positives, and Tool B that reports 50 real bugs but with 50 false positives, users will almost always prefer the former, even though it is finding fewer real defects. This is perfectly understandable — users are asked to weigh an immediate concrete negative (time spent looking at false positives) against an intangible potential future positive (vulnerabilities that may not show up).

    However, if one offsets the time and risk saved in finding those 10 bugs earlier (i.e. by avoiding expensive and potentially dangerous bugs in finished products) against the time needed to assess the additional 40 warnings as false positives, then it quickly becomes apparent that configuration B is more economical. This is especially true with security vulnerabilities that slip through testing and can have expensive consequences.

    Since the number of warnings within a DevOps pipeline can be a deterrent to getting the most out of a SAST tool – especially early in the project, when it provides the greatest benefits – organizations should consider the following techniques to increase developer attention to alerts:

    • Filter and Focus: Parse the viewed data from the tool’s interface, focusing on what is most important for the project and assigning developers to fix critical issues in priority order.
    • Mark and Defer: Lower the priority, or change the state to “later,” for example, on all or a subset of the warnings based on some set of conditions that are less crucial to the project.
    • Stop the Bleeding: Use the above techniques to temporarily defer existing warnings with the emphasis on fixing new defects that are introduced as code is changed or new code is developed.

    Despite the fact that it will invariably produce some false positives, SAST helps shift security further to the left by ensuring security and programming vulnerabilities are eliminated when code is being written, rather than at the testing stage when they are much more expensive to fix. Filtering and prioritizing warnings are two effective techniques to ensure developers don’t throw out the baby with the bathwater.