Tag: static analysis

  • Modern DevOps is a Chance to Make Security Part of the Process

    Modern DevOps is a Chance to Make Security Part of the Process

    I’ve mentioned before, and many of you have lived through, the slowly changing beliefs around DevOps versus security. We are past the days of “Security slows us down” and into “How can we bake security into the development process?” which is essential. Indeed, I would say it’s more essential every day as the number of threat actors and their use of automation to perform more attacks increases.

    So where can we slip in a little security to protect those sweet, innocent apps from the Big Bad Wolf? Let’s make a list, then chat about it.

    • Source code
    • Inclusions–libraries, shared objects, etc.
    • Container/cloud images
    • Network
    • Running applications and APIs

    This is honestly not an exhaustive or detailed list, but we’re writing a blog here, so let’s stop there so we have time to discuss.

    Source code security is greatly enhanced by putting in any of a dozen development security tools, which include static and dynamic security scanners along with a variety of other features.

    Inclusions are often nested and confusing. A mod that developers include relies on other mods and libraries—sometimes so many that the developer can’t even list them all. So, this type of security starts with software composition analysis (SCA), then uses the list that generates to pursue vulnerabilities—normally by looking for the included libraries in a vulnerability database. Several prominent source code security solutions bundle at least the SCA part, and some also help nail down included vulnerabilities. Check with your vendor if you’re just getting started here.

    Container images are akin to the “golden images” of old. Build one that has what most applications need, add on the security that an organization wants most applications to support and then have every project start there and only add what this specific app needs. That allows for scanning of the base image so all applications start out at least somewhat secure. Cloud images can be done much the same, but beginning at a higher level. Look into registry security or image scanning tools to meet this need. It is worth asking your source scanning vendor if they offer this functionality; some do.

    The network on which an application will be exposed needs to be locked down. While the access points into the org probably are protected, portability of applications means that the application itself should be locked down wherever it uses that network. Tools are specific to the platform and networking in use, so look into it—I can’t offer much more generic advice.

    Security has long tried to lock down running applications—generally with pushback from developers and even ops, a bit. Between web application firewalls (WAFs) and API security, we have the ability to stop a whole host of attacks on their way in … But you have to use WAFs and API security best practices. If you’re starting from scratch, consider tools in the application and API security space that protect against both the web UI and API attacks with a breadth of technologies. For those with a WAF, look at API security tools that protect APIs. For those with API security covered, consider a next-generation WAF.

    My usual encouragement applies. You are rocking it, and all of these tools are increasingly integrated into the DevOps toolchain, so it’s worth the time investment to get them set up and let them run. If you’re just starting out, start with source code security. Honestly. It will make the overall app more secure by tightening its source. But don’t ignore the ops side, as that is part of the attack surface. Protect those apps you built and/or maintain that are keeping the org running.

  • Integrating Security in the Development Process With DevSecOps

    Integrating Security in the Development Process With DevSecOps

    Occasionally, it’s worthwhile to reflect on how we develop and deliver software during times of rapid change and significant disruption. In those moments of reflection, we learn from the exciting trends and changes that are taking place. Recently I had the opportunity to commiserate with two technical thought leaders at Perforce: Stuart Foster, product manager, and Steve Howard, static code analysis evangelist.

    The disruption caused by COVID-19 goes much deeper than just the rapid shift to work-from-home. Development teams are being asked to keep up developer velocity, while the organization as a whole accelerates digital transformation projects, shifting priorities and emphasizing investments needed to successfully navigate difficult and uncertain economic and market conditions.

    In our conversation, we note how investments already made in shifting to the cloud, remote and cloud-based development, automated testing and pipeline automation and security tools are helping organizations today. We also explore how current conditions influence product roadmaps, causing software teams to take a holistic look at their development processes and tools, and organizations to rethink how to take their software delivery capability to the next level.

    Stuart and Steve share their software best practices on collaboration, automation, security, standardization and more.

    Explore our conversation in more depth by viewing the video and transcript below.

    Transcript

    Mitch Ashley: Well, I’m happy to be joined by Stuart Foster, who’s product manager with Perforce, and Steve Howard, static code analysis evangelist, and I think a lot of other things, too—does a lot with Perforce. So, welcome, gentlemen—good to be talking to you today.

    Stuart Foster: Thanks, Mitch.

    Steve Howard: Hello.

    Ashley: You know, we’re talking about—you know, Perforce does a lot of things, and I think we’re just kind of focusing on what maybe some of the current conditions, current best practices are. There’s a lot happening right now in everybody’s world—a lot of change, a lot of disruption.

    But before we get to that, could I have you both introduce yourselves? Maybe Stuart, would you start first?

    Foster: Yeah. Hi, everybody, I’m Stuart Foster, I’m Perforce’s product manager for our static code analysis products.

    Ashley: Great—Steve?

    Howard: Yeah, and I’m—well, I have, as you say, many jobs and roles within the organization, but primarily focusing on the evangelism around using static code analysis for both of our products, Klocwork and QAC, here at Perforce, yeah.

    Ashley: If you’re working with developers, you gotta be doing evangelism, right?

    Howard: Yeah. [Laughter] 

    Ashley: They expect you to give them support, and be talking to them, in a different kind of way. Tell us—would one of you tell us a little bit about Perforce and what all you all do?

    Howard: Yeah, sure—well, I’ll take that. So, Perforce is an organization, obviously as, you know, originally comes out of the version control software. But really, it’s about developer tools and how we can help make the developer’s life easier in the day to day tasks. So, whether that be trying to prove compliance to standards or just trying to get the stuff out the door in the first place—anything that we can do to help make that process smooth and reliable is kind of where we are. And, you know, these days, that has a bit of a—a buzzword, if you will, around the DevOps tooling ideas, but essentially, it’s developer tools and making their life more comfortable.

    Ashley: That’s great, because you all fit into a number of places in the sort of tool chain, pipeline, workflow process across the development life cycle.

    Howard: That’s right, yeah.

    Ashley: So, that’s awesome. Well, let’s get into this. I’m really curious about what you’re finding in market, and one of the things we like to do when I’m talking with experts like yourself is, let me just try to share some of the things that we’ve experienced within the past, but also kinda recently that it’s happening, because we’re all dealing with economic change, disruption, COVID, work from home, all of those kind of things, but you know, we’ve done some research at my company and there’s a lot—of course, a great, great emphasis on acceleration of digital strategies and, of course, that puts a lot of pressure and emphasis and people are betting money on software being the answer to helping them implement their strategies.

    Are you seeing the same kind of thing, that acceleration of digital transformation and software—elevation of software projects?

    Foster: Yeah. We’re seeing that across the board, even in our own company here. You know, a lot of us have transitioned to work from home, and then you have to start thinking about the considerations for that, right? 

    And what we’re finding ourselves and working with our customers is, a lot of people are shifting more technological tools, right, to be able to better produce workflows and code reviews, how to—working from home, security, right? How secure are your remote development work stations? How is security being built into your software and your workflow? So, tools are being used across the board for that. Just being able to rely on a piece of software versus in-office workflow has become more and more important. And amongst all that, then there’s the goal to, you know, keep development velocity up, which could be a challenge when working from home, right? So, we’re seeing a lot more DevOps automation of, you know , a lot of workflow process and things like that.

    Ashley: Yeah, there’s no more popping over the edge of the cube or to the table next to you, right? Somebody sitting there at the café, in the office, or down the street. It’s a fully digital world, right? You’re not sitting with people very often.

    Foster: Yeah, we’re relying on Slack, we’re relying on Slack calls, Zoom videos, everything to just stay in touch and work and collaborate with one another, yeah.

    Ashley: Steve—yeah.

    Howard: But I think that’s where—yeah, so, I was gonna say, I think that’s where those centralized processes that we’ve kind of all been working towards, luckily, for the last few years, are really gonna pay off. Because how do we not have that? If you imagine how things were maybe five years ago, it would’ve been a much bigger transition to get to where we are now, whereas it was almost like most of our organization was kind of ready to go. We were already there, just—you could call foresightful, but really, it was a little bit of chance, I think. Yeah, those processes have held things together remarkably well, and we’re very lucky for it.

    Ashley: You know, nobody would wish for something like COVID or a pandemic on any of us, but if you look back, you know, I think back when we were virtualizing things and developers were starting to be able to just create a fully self-contained environment to develop on their laptop—I saw people working on planes, right? First time I saw somebody writing code on a plane—that’s awesome.

    But so many things, and then of course, adding the cloud and all the services and software and tool chains and pipelines that we can create now, it—I mean, I think we’re lucky, if you were gonna have something like this happen, now is the time where we can actually continue to be productive, and put more emphasis or pressure on ourselves to deliver software. Do you agree?

    Howard: Yeah.

    Foster: Yeah, absolutely.

    Howard: Yeah—sorry, Stuart.

    Foster: [Laughter] We agreed at the same time, that was great.

    Ashley: [Laughter] 

    Foster: But you’re right, I think, like, some of it—you know, it’s a little bit accidental, but as you mentioned, setting up the development workflow to be able to do it on a laptop, people are shifting their hardware, they’re removing their hardware and going to the cloud, right? So, it’s kind of been like an expansion and a reduction, all—just naturally, but also, you know, you think about DevOps and there’s also, you know, cost savings to be had by not having to have this footprints, not having to worry about these things.

    So, this could be, in some ways, pretty revolutionary or changing for development costs for a lot of companies.

    Ashley: Steve, you were gonna jump in there—anything to add?

    Howard: Yeah, no, exactly the same. I think the fact is that we—you know, a lot of that infrastructure is being built up and the idea of being able to run our development pipelines kind of in a cloud environment or whatever and having that all ready to go just meant that we didn’t have to pick up ginormous work stations. I remember back in my days of developing in aviation software, we used to literally have, you know, Unix boxes stacked up on the side somewhere that we had to go and log in on each one and run our testing for each single one. It’s just unthinkable. It would never have been possible.

    Ashley: That would be insane today. You would never. I mean—

    Howard: Yeah.

    Ashley: – if you have a choice, you would never do that, right? [Laughter] 

    Howard: Well, exactly, and the very fact that, you know, 10 years ago, we still had to do that, how could we have transitioned? It wouldn’t have been possible. And so, it really is testament to how far we’ve really come in 10, 15 years to see where we are with our testing processes, the fact that we can do, you know, 99 percent of it, if not all of it can be done completely with no connection to the physical space.

    Ashley: Mm-hmm.

    Howard: It’s all—and that can even mean, you know, in the cloud completely, so it’s not even within our organization. It’s securely being done off site somewhere in the ether. [Laughter] 

    Ashley: Yeah.

    Howard: And that process is still completely manageable, it’s something we can all see as developers, everyone can get to see where we currently are with that process, you know, what the latest test run gave us across the whole—

    Ashley: I’m really curious—yeah, I’m really curious what your experience in market is. Do you see more people who are kinda doubling down and getting even more serious about their DevOps implementation and taking it to the next level, or do you see more people coming to you that maybe are relatively new to DevOps and are looking for tooling and looking for ways of helping them, you know, not just get started and learn all the problems themselves, experience them themselves, but looking from the experience of a software provider like yourself, or is it a little bit of both?

    Howard: I’d say a little bit of both. I mean, obviously initially, there was such a lot to do already and everyone kind of had no time to really think—understandably, but probably within, you know, since the end of the summer, maybe, things have started to come back and people are actually going, “Well, hang on a minute. We’ve got a process, but now, how can we actually make this a lot easier for ourselves?” and looking to improve that.

    And I think, you know, for a lot of the organizations where, perhaps, doing things off site would never have even been considered. And I think this was to Stuart’s point—that kind of, that’s been forced upon people and it’s almost gonna change the way we develop forever now as a result of that, because we have now, we’ve done it, we’ve experienced that it’s possible, and it’s safe and it’s secure. 

    How can we now make that better and make our lives easier by not having that massive footprint that Stuart was mentioning? We’ve actually got something that’s very, very lean, it’s in the cloud, we’ve got machines when we need them, but not when we don’t need them, and how can we really make that process as efficient as possible from the results side, you know, making sure we’re doing what we’re supposed to be doing as quickly as possible, but also actually trying to reduce our footprint on process and power, for example? You know, the development time that’s involved in that, how can we make that as efficient as possible?

    Foster: Yeah, and I also think what we’re seeing, too, is, you know, our existing customers, they’re doubling down on their tools, you know, because they need to rely on something, right? And that’s also, what it’s done is also, I think, is changing some of the roadmap items that all products are looking at, right? There’s a whole new—not a whole new paradigm, but you know, there’s a little bit more seriousness being taken in terms of some of these features and functionalities of being able to work remote, or the requirements for more cloud usage, right? We’re talking ease of use, we’re talking about things that are, you know, related to workflows to make working in these conditions—not just in these conditions. Assuming you get back to the office, just more efficient as well.

    Ashley: Mm-hmm.

    Foster: So, we’re seeing that at the same time, too. And then, you know, from the perspective of new customers coming in, they’re almost expecting these things to already be there, right? Because they’ve run up to these problems as COVID has created and they’re trying to find the solutions that are gonna be kinda turnkey out of the box.

    And also, I’ve done, I’ve been talking with various analysts related to our product, but it seems as though the analyst community itself is, they’re seeing the requirement for, you know, ease of use in DevOps becoming more important as well at the same time.

    Ashley: Mm-hmm. I think that may be also, in part, the expansion of DevOps into the organization. They may have had a core group of teams that were really taking the challenge of DevOps and implementing it and now, how does the rest of the organization do the same thing, but accelerate their adoption of it?

    Foster: Yeah, and I think also, a lot of development groups within a single company—you know, you may have a security team, you may have a development team, you may have a LiveOps team—they’re starting to recognize that there are choke points in the development processes.

    Ashley: Mm-hmm.

    Foster: And so, how do you unblock that? How do you make it more easy? How do you offload some of that choke point onto a broader group, right? A lot of the idea of DevOps, DevSecOps, whatever you wanna call it, right, is that developers are owning the code that they produce in more broad ways. That’s writing unit tests, that’s making sure they do the QA on it, right? That’s when they’re, you know, they have to implement the security requirements then and there, right? They need to ensure the traceability is done, right? They have to—they’re taking that into their hands. It’s becoming not so much that there’s a DevOps group that magically handles all this infrastructure, it’s becoming part of each developer’s responsibility as well, right?

    Ashley: Hmm. Interesting. You know, I know, as you scale things out and you adopt new tools, tools do a lot of things for you, but there’s oftentimes, there’s approaches that you need to adapt into your tools, things like what are our coding standards, what are our testing, what level of testing do we do? How do we break it into chunks? When do we do security testing of our code?

    And that’s part of that learning curve to is, how do you adapt a tool set to both what your requirements are, but also, you would like it to kinda take you to the next level. What are some of the things that really can help people kind of propel their expertise in the organization’s ability to deliver software more effectively?

    Howard: Yeah, that’s a really good point. So, one of the things, you mentioned coding standards there, but obviously, you know, those can come from multiple sources, they can be something that’s actually being required by a standard, for example, if it’s in a safety critical, security critical world. It could be something that’s actually demanded by the organization itself—they have their own rules or there’s certain things in any particular project that, if you do things in a certain way would make, you know, it would be dangerous or insecure or something. So, they have some controls, as well, of that.

    And what we do see is, the more that we can get that information out to our developers, for example, the more value that has as well. So, yes, we can help enforce coding standards, for example, with our Static Analysis tools. But that’s only half the story. Enforcement is one thing; actually getting the developers to understand what the problems are as they’re creating them and to have easy access to the help and the quick feedback of those problems makes it much more powerful, because it gives them, (a) you know, all the resources they need to deal with the problem quickly and to not perhaps find it a challenge to have to deal with, but also, they learn. 

    And so, the next piece of code they write doesn’t generally have the problem. So, there’s a developer learning element there as well, which we can control centrally, and then spread out to all of these remote workers working—you know, without that team collaboration that we used to have, we’re still able to do the same things. So, that’s become something that’s really focused people’s minds on the capability there in terms of what that means. And as I say, that applies to security standards that are well known or requirements for quality that we have to work to for a security or safety standard—

    Ashley: Mm-hmm.

    Howard: – but also the company’s own rules.

    Foster: Yeah, and I think, you know, outside of just simply the tools, development teams are, you know, taking a step back and re-evaluating or thinking more about what their software development process is, holistically, right? You know, Steve mentioned there some of the tools, best practices, but you know, you also have to think what kind of, you know, software development methodologies might you be using. You know, sometimes it doesn’t matter whether it’s agile, waterfall, whether you’re using Scrum or Kanban, you know, I think it’s giving people a moment to think about what’s the most important kind of Dev process that is gonna fit this team or how you have to work now, how it fits the businesses. 

    And then implementing processes to ensure the organization, communication, and what systems need to be put into place to help prevent any problems that arise during development and kinda identifying and understanding that. Then you’re picking the tools, you’re building your development workflow, whether it’s in environments, it’s setting up DevOps pipelines to ensure that everything’s followed correctly. It’s the tools, but it’s the process, and that’s changed, too.

    Ashley: That’s interesting. You know, we’ve done some research about the influence that developers have had on the organization, too. Even outside of development teams, you’re seeing DevOps and agile kind of principles, even though they aren’t calling it that—daily standup meetings, shorter intervals of deliverables, more kinda asynchronous communication tools and collaboration kind of tools, which really makes it easier, I think, for the development teams, the product teams—you’re a product manager yourself, Stuart.

    Foster: [Laughter] 

    Ashley: And beyond—operations, security—everyone is kind of forced to work a little bit differently, but also can benefit from what each other knows about how to do that more effectively. And we seem like we’re at this sort of serendipitous time of being able to leverage that, which fits right into the tooling approaches that we can take.

    Foster: Yeah, and you know, when you look at it, like, it always just ends up communication. Whatever the communication is, whether that’s the workflow, whether that’s in-person, whether that’s meeting, whether that’s documentation, it’s all about communication, and that’s true not for just development, but for business, right? Developers working, you know, with other groups—marketing, sales—everything, it’s all communication. It just keeps business running. And that’s kinda the funny kinda perspective that may not necessarily be outright visible by simply saying, “It’s communication.”

    Howard: What’s interesting is, that’s also kinda mirrored in the tooling, because a lot of the work we’ve been doing recently is about how to make sure the tools have all got open interfaces to communicate with the next thing in the pipeline or to feed that data back up to some dashboard that’s gonna be shared with the rest of the team.

    So, yeah, it’s kind of an interesting point, but that also applies with the tooling that we’re trying to build around that.

    Foster: Mm-hmm.

    Ashley: Mm-hmm. A really great point. So, to give you fair warning, I’m gonna ask you your three recommendations on best practices in the current climate, so be thinking about that.

    I was thinking about your comment, Steve, about being able to better integrate tools together and then taking the data from the tool sets, the pipeline of the process that we’re going through and the pipeline of work—there’s so much value in that data that sometimes ends up as exhausts or on the cutting room floor, right, to use two different metaphors. But that information could be extremely valuable, putting it into dashboards, feeding it into continuous improvement processes, shortening the window for getting feedback for, “Here’s a security vulnerability or a coding standard” or some kind of an error that you can fix in minutes instead of days and weeks and sometimes months, right? The security audit at the end of the release is much easier, because you’ve already gone through so much to it. Those can all be things that can help us.

    So, I gave you a little bit of time there to think about your three best practices. I did my best job of stalling that I could. So, who would like to go first? What would you recommend people consider today?

    Foster: Yeah, I’ll go. Just, I want to reinforce that message of,  you know, figuring out how you’re going to work as a development team, develop that communication, right? And that’s using your tools.

    I think in the world of remote work from home, having kind of a common language—again, back to communication—of how, you know, each developer’s system is set up. So, common environments, they should be using as close to the same setups as possible so that, you know, when one of you is running into an issue, it’s easier to get that information across, right? You’re not gonna have to install something obscure because somebody’s using something obscure. You should all be trying to talk and use the same tools so you’re all speaking the same language. That allows you to have that velocity that you could otherwise lose.

    And in this world, too, security hasn’t been more important than ever—security is more important than ever. So, think about securing your Dev processes, developing secure software, it’s gonna be very important. And those are kinda my three.

    Ashley: Okay, great. All good ones. Steve?

    Howard: Yeah, I’d answer—so, my three would be, first of all, to automate; take the process away from the day to day effort of the development teams and organizations around them. So, yeah, that’s the first one.

    Report—so, that’s the communication piece, I guess, but get the information to the right people at the right time and hopefully in the right sort of time window.

    And then shift left. So, then, we make it more efficient. Then we look to where we can try to remove the fat in the process and take away the time that was wasted and try and move it to the right position in that process. So, yeah, those three, in that order. [Laughter] 

    Ashley: Great. Excellent, excellent. I’m curious what you both think will be some of the things that will stay with us, the things that’ll last past a forced work at home, forced work remote. Maybe people are returning to the office, maybe they aren’t. What do you think are some of the things that will stick with us? Because it wasn’t like we were doing this for two weeks and then we’d just go back to the office—this is an extended period, we’re developing new habits, we’re changing how we’re working. Why go back, sometimes, right? What are your thoughts?

    Foster: I think you’re gonna see a lot of these workflows stay around. I think that the reliance on the numerous tools to keep velocity up and reduce and automate are just gonna stick around. You know, you don’t want to be wasting time on trying to get something to work. It’s easier to just simply automate it and get it out of your hands so that, you know, at least from a development perspective, you can just focus on what’s important, which is, you know, coding, the feature, getting the product out the door, right?

    Those are gonna stick around, and I think the tools, over the course of time, are just gonna keep reinforcing and strengthening that kind of automation pipeline.

    Ashley: Good.

    Howard: Yeah, no, and I think, from my perspective, just to add the one small piece there is, the cloudification, essentially, of the processes. So,  I think, you know, people will stop having the premises to host great, big server racks in, and that’s just all gonna move off site, off premise, outside of the organization, for sure, yeah.

    Ashley: Yeah. To kinda add what you both have said, which I think supports your thoughts on this is—you know, as we change to whatever we’re gonna go back to, I don’t think it’s gonna be back to what we were doing. But it’s gonna be a hybrid, right? I can’t imagine everyone shifting back to a different way. It’s—some people are still, maybe even more percentage of our teams are working remotely, never going back into an office, but we may have some that do.

    We’ve broadened the reach across the globe of people that we’re working with, the product and vendor, tool vendors that we’re working with. Those things aren’t gonna change, right? 

    Howard: No. No.

    Ashley: We’re gonna stay doing that. If we’ve found something that works better, you’re gonna stick with it, right? So, you’re gonna stick with—you’re gonna stay in the cloud, push more in the cloud, right? And leverage some of the automation, the processes that you talked about, Stuart.

    Well, this has been really fascinating. Would love to have you both back, have you guys back and we can talk some more about—I’m gonna delve into static code analysis evangelist, your role, Steve, get into some of that, which I know you all have some of your products do and to security and to some other areas within the software pipeline, but this has been really great. I’ve really enjoyed talking with you both.

    Howard: Yeah, it’s been a pleasure.

    Foster: Great. Thanks, Mitch.

    Howard: Bye bye.

    Ashley: You bet. We’ll see you soon.

  • What Developers Really Think About Pentesting

    What Developers Really Think About Pentesting

    A developer in their natural habitat is often spotted in a state of deep concentration, coding awesome features to tight deadlines. Feature-building is often our favorite part of the job, and really, it’s the fundamental outcome of the software development life cycle (SDLC). 

    However, as we have discussed before, many of us are still prioritizing features over security best practices. After all, in most organizations, it is set up to be someone else’s job and adequate security training for us is limited. Penetration testing and static analysis scanning tools (SAST) are just part of the overall process to mitigate security risks, operating rather independently from what we do — until the code bounces back to us for hotfixes, of course.

    It’s at that moment that many developers think, “Do the pentesters hate me?”

    These interactions often define a team and culture. The concerning thing is, the lack of communication, understanding and overall collaboration has created tension, at least on the side of the developer.

    Think about it: Imagine you’ve spent a few hundred hours sculpting a marvelous statue, and then someone comes along with a hammer and starts smashing bits off it after they’ve told you its foundations are not up to scratch. That’s the perceived dynamic between a tester and developer — the latter has their software darlings slaughtered by an outsider that hasn’t labored through the process with them; instead, they’ve extended the workload and delayed the satisfaction of shipping code. 

    Having moved into the security space long ago, I can see both sides of the story. And no, pentesters don’t hate developers. The pentester is in all likelihood, overworked and under a lot of pressure. As such, a constant stream of common security bugs that could be quite easily fixed at the code level take up time, resources and headspace away from the really serious issues. 

    I always saw pentesters as kind of like parents. They want you to do well, and when you don’t they’re not mad, just disappointed.

    Now that I’ve put that (perhaps slightly unfair) image in your mind, let’s explore this a little deeper. What has caused this world view among developers?

    Resentment for Being Told What to Do

    Nobody likes feeling as though they’ve done a bad job, or that someone doesn’t like their work. Sadly for developers, when static analysis and pentest results come back to them, it can feel like a report card. They’ve been given low grades, but at the end of the day, their bosses assess them on the features they’ve built and the time they’ve delivered them, not whether there were vulnerable elements in the software or not.

    For the poor pentester, this is a case of “don’t shoot the messenger.” It’s nothing personal — they are tasked with finding bugs, and they found them. Granted, at a person-to-person level, maybe some pentesters are grumpier than others, but they’re not (or shouldn’t be) out to crucify development teams. It would be far easier for both teams if they were on the same page with what constitutes security best practice. Developers are not expected to be perfect; realistically, the testing team is there to protect them from shipping vulnerable code.

    Minor Issues vs. Higher Priorities

    It’s true, a developer’s highest priority will always be the building of features, and in this crazy world of rapid digitization, it will have to be done at speed. While some coders have a personal interest in security and secure coding, the general sentiment is that security is “someone else’s problem,” which inevitably includes pentesters. 

    Most common vulnerabilities are indeed minor issues to remediate — once known, the fixes are simple to execute for things like cross-site scripting (XSS) and SQL injection. The problem is, many developers don’t realize they’re introducing them in the first place, and these seemingly minor issues are the small window of opportunity an attacker needs to cause devastating problems for a company. According to Akamai, between November 2017 and March 2019, SQL injection vulnerabilities accounted for 65% of all web-based attack vectors. For a vulnerability that has had a known fix for more than 20 years, that is a sobering statistic. 

    Some pentest teams do assist in the remediation of security bugs, but others will provide a report of the bad news and expect developers to work through hotfixes, even if they have moved onto a different project by the time this happens. And in some cases, the development team may be faced with a report that includes bugs they can’t (or shouldn’t be expected to) fix. It still has to be part of the findings, and again, not taken personally.

    The “happy medium” for this would be pentesters, security personnel and development managers acting in more of a mentor role to ensure the team has what they need in terms of effective training and tools, giving individual coders the best chance to succeed and code securely from the very beginning of the SDLC. Both teams really should be meeting half-way to ensure security is considered from the start, as part of a healthy DevSecOps practice. 

    Lack of Knowledge vs. False Positives

    Static analysis is an element of the security process in the SDLC, and static analysis scanning tools (SAST) play a fundamental role. And yes, false positives are an issue with these and other types (DAST/IAST) of scanners. It’s an annoyance in what is already a slow process, requiring manual code review and putting pressure on both developers and pentesters alike. Pentesting personnel have taken time to meticulously set up custom rules to avoid inaccurate readings and provided company-specific guidance, yet some false readings slip through and end up in front of a head-scratching developer. 

    This process isn’t perfect, but the other problem is that many developers lack enough knowledge to mitigate a lot of common vulnerabilities on a consistent basis. With security training rare in tertiary education, and on-the-job training varying in its effectiveness, it stands to reason that there may be some overconfidence at play as well (and it’s not their fault, we as an industry need to get better at equipping them with what they need).

    Remediation Tasks

    Sometimes, there is an assumption by overworked engineers that pentesters are just hanging in the wings, waiting for the moment to strike by testing an application and raining on the development team’s parade. They are over-testing, they’re nitpicking, they’re creating extra work. 

    The only problem with that is they, too, are overworked (more so, in fact, the cybersecurity skills shortage is at dire levels and getting worse) and simply don’t have the time to test without reason. They are not the sole decision-makers in prioritizing testing; it could have been requested by senior leadership, a customer, as part of a security audit or even determined as a result of a bug bounty program. 

    For a developer, being pulled off current feature-building sprints to work on security fixes is annoying — especially if it’s not their work. Perhaps a previous team did the last update, or another vendor. However, security is everyone’s problem. That doesn’t mean every developer has to take ownership of security bugs as though they have made them all themselves, but they do need to come to the party in terms of security being a shared responsibility. 

    Where to from Here?

    Sometimes, a mindset shift can be all it takes to make significant headway in solving a problem. We’ve talked about the rather frosty reaction a developer has to less than favorable pentest results, but what if they could turn it into a challenge? Perhaps they could think of the pentester as a friendly competitor; someone they can beat at their own game. After all, a security-aware developer that can eliminate common bugs as they write code is going to make their job much more difficult. By contrast, a developer with no focus on security is going to be comprehensively bested by their pentester counterparts when they can easily break their software. 

    Pentesters and developers may not be joined in harmony 100% of the time, but their relationship can be vastly improved when an organization addresses security as a key priority, and empowers teams with the right knowledge and tools to succeed — especially developers. It comes down to whether a company-wide, positive security culture is a priority, and if we are to fight the (currently) losing battle against common vulnerabilities, it absolutely should be.

  • Decoding the Modern Enterprise Software Spaghetti

    Decoding the Modern Enterprise Software Spaghetti

    Modern enterprise software architecture is growing increasingly complex. Organizations are jumping on the digital transformation bandwagon in growing numbers. They are striving to become more agile and continuously deliver new code, whether on-premises or to support cloud migration efforts. In the process, IT teams are amassing an ever-growing inventory of applications and databases, which generally includes a blend of custom code, open source and commercial applications.

    “Spaghetti code” is what some call a tangled mess of a program—one that is difficult to follow, poorly documented and almost impossible for anyone but the original developer to troubleshoot or upgrade. Even today, when managers and developers take a more structured and disciplined approach to implementing code, the enterprise remains a patchwork of technology, a kind of spaghetti that can be difficult to understand and detangle.

    Given these layers of complexities, it is imperative for organizations to be equipped with the right processes and tools to map out their IT assets. They need to understand how programs and databases interconnect and how a change in one can have implications across the network that imperil availability, performance or security and negatively impact employees and customers. Here are some best practices that enterprises looking for an easier digital transformation journey can follow.

    Gleaning the Impact of Enterprise Software Changes

    Implementing a software change or embracing a new application that has a puzzling code makeup has plenty of inherent challenges—not to mention potential long-term disruptions for multiple departments, functions and processes within an enterprise. Whether an organization is dealing with inherited applications from a merger or acquisition or simple system migrations, much depends on how it is coded. As a result, enterprises must familiarize themselves with the overall layout of the enterprise architecture, down to the code level. This comprehensive level of analysis aids organizations in identifying the full scope and impact of the desired changes or updates.

    This is especially important for software teams who are operating on tight budgets and strict deadlines. Some important factors to consider in a comprehensive analysis include:

    • Identifying the impact of a change request on the software development project or on a business’s IT infrastructure.
    • Determining which software resources must be modified to implement the requested change.
    • Calculating the cost, time and effort needed to perform the change request.
    • Ensuring that the requested change complies with internal and external regulations.
    • Eliminating or minimizing any security risks associated with the change.

    Decoding the Spaghetti With Automated Discovery and Code Mapping

    A study by research firm Enterprise Management Associates (Next-Generation ITAM) found that the average IT organization uses 11 or more different inventory and discovery tools—some even more than 40. However, none deliver the full picture and they can often contradict each other. As a result, the average IT organization spends 15 hours every week manually resolving discrepancies among its various discovery tools. These hours translate to hefty costs.

    Fortunately, new approaches to discovery blend static and dynamic analysis methods that can operate in real-time to decompose software dependencies down to the code level. These new tools provide significant improvements in discovery and mapping, eliminating the need for multiple, redundant tool use and wasteful manual deconflicting.

    According to a recent Forrester survey, 56% of IT managers don’t have a complete view of dependencies between applications or between applications and infrastructure. However, today’s evolving business landscape dictates that technologies must evolve to keep up with an organization’s shifting needs. Software changes, when implemented strategically, can help businesses be more productive, efficient and profitable.

    To ensure that software changes serve this purpose, IT project managers need to implement a solid software change assessment system. With the help of technology, checklists and processes, organizations can outline a path that yields efficiency. They can understand and decode that tangled mess of software called the modern enterprise tech stack.

    — Soumik Sarker

  • DevOps and Security is Like Smoking Meat

    DevOps and Security is Like Smoking Meat

    It isn’t everyone who thinks, “Doesn’t Ubuntu remind you of wild boar?” Or labors over his pit of slow-roasted pork shoulder while contemplating containers. Or dwells on e2e testing while mesquite smoke permeates spareribs.

    But, Apollo Clark (@apolloclark) does. Apollo is a foodie of smoked meats (is that a meatie or smokie?), working to master the craft: understanding different cuts, what each type of wood adds, and the subtleties of sauces. But, alas, it is a hobby; security and DevOps are his career.

    Being passionate about both, he naturally sees parallels between the two, so Apollo presented at the 2016 All Day DevOps conference with a session titled, “What Smoking Meat Taught Me About DevOps and Security.”

    Understandably, you are asking yourself, “What does smoking meat have to do with DevOps and security?” Apollo notes both have tremendous complexity and nuances, there are multiple ways of getting the job done, lots of ways to mess it up, a couple of ways of doing it right, and you are always learning.

    apollo 1.png

    For both, there are many tools and processes to get the job done. For smoking, your wood is a critical component, along with time. Let’s look at some parallels:

    • Oak = unit testing
    • Maple = coverage
    • Apple = dynamic analysis
    • Peach = static analysis
    • Mesquite = e2e testing. People love it, but it is difficult to handle
    • Wine barrel = browser support. This is when you are doing it really well
    • Bourbon barrel = device support. Pretty complicated
    • Smoking time = testing. You can test or smoke for five minutes or five hours, but there is a sweet spot. You can oversmoke meats and you can overtest.

    Getting hungry?

    apollo2.png

    Meat is critical to a meal of smoked meat, but so are many other components. Likewise, software applications take a suite of other tools and components to deliver the full package. Get your taste buds ready, because Apollo likens all of the goodness that goes with smoked meats to tools you use to keep applications running:

    • Sauces = auto-scaling. They both can get pretty complicated. Always a little different, but you have to make it work for you.
    • Bread = monitoring, which is the bread and butter of infrastructure. Make sure you have it up and down your stack.
    • Salad = system logs. Not the sexiest things, but you can rely on them.
    • Fruit = custom application logs. Takes a lot of time to pair them, but takes a really good thing and makes them better.
    • Beer = firewall. You should always have both.
    • Wine = Intrusion detection systems. These are your fine wines.
    • Whiskey = IR training. It takes time and there are so many ways of doing it. When things break, we have procedures on how to deal with them.

    When smoking meats and in DevOps and security, Apollo asks and answers, “Do we have to do everything? No, but the more we do, the better it will taste.

    “Does it cost money and take time? Yes, but you can do great things even without money and time. You can use cheap meats and you can use open source. Start simple, build up complexity, and always be learning.” Constantly ask yourself, “Am I better today than I was yesterday.” If not, be better.

    You can watch Apollo’s entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It’s all free and online Oct. 24.

    — Derek E. Weeks