Category: Blogs

  • DevOps Chats: 1-Click Vulnerability Scanning on GCP, With Qualys

    DevOps Chats: 1-Click Vulnerability Scanning on GCP, With Qualys

    Sometimes the best way to accomplish something is to choose a path requiring the least friction, or amount of change.

    Qualys customers now have that path available to them in bring vulnerability scanning into Google Cloud Platform. Qualys’ recent announcement means a one-click configuration change enables vulnerability scans in GCP with the results appearing in both Qualys Cloud Center and GCP Security Command Center.

    Join me on this DevOps Chats to explore this announcement with Sumedh Thakar, Qualys President and Chief Product Officer. We discuss Qualys’ and Google’s collaboration and how this benefits DevOps teams.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with DevOps.com and you’re listening to another DevOps Chat podcast. Today I’m joined by Sumedh Thakar who’s President and Chief Product Officer with Qualys. We’re talking about an important announcement that just happened with Qualys and with Google about security natively embedded within Google. Welcome to the podcast, Sumedh.

    Sumedh Thakar: Hey, Mitch. Thank you for having me.

    Ashley: Great to have you. Well, let’s start by just having you introduce yourself and, of course, as President and Chief Product Officer we can imagine what you might do at Qualys, but go ahead and tell us a little bit about what you do.

    Thakar: Yes, I’m not sure you can imagine because I’ve been here for 16 years, so I’ve done a lot.

    Ashley: Okay. [Laughs]

    Thakar: Today, as President and Chief Product Officer, I’m responsible for product strategy and implementation of Qualys product and where we have come from where we were many years, mainly focused on just vulnerability assessment. So today I have the engineering product management support operations team as well as the sales team and that kind of gives a pretty unique perspective because of the engineering and the DevOps teams within Qualys. Also, part of the organization for me helps me to really get a very good understanding of today what are the needs for DevOps and what evolution we’re going through with the DevOps team internally within Qualys, and being a massive SaaS platform ourselves I think that that’s something that I’d really like to focus on when we develop security solutions for DevOps teams.

    Ashley: Well, it does give you a very hands-on perspective, both _____ as well as living with DevOps in your own products internally. It’s great.

    Thakar: Exactly.

    Ashley: It’s good for you. Great role to have. Well, let’s dive into this announcement. So, security that’s natively embedded within Google – Google Cloud I think we’re talking about.

    Thakar: Right.

    Ashley: Tell us a little bit about what it is.

    Thakar: Yeah, we’ve had a fantastic partnership with the Google team and, you know, I think as digital transformation, moving things into cloud container and DevOps have been really getting traction for us to _____ differently than what we have done in the past, which has always been after the fact, adding security solutions. So, today with DevOps and DevSecOps the opportunity to embed security upfront into the infrastructure in an automated way so that you can ensure that you’re not missing – you know, that’s always the biggest issue with security is, oh yeah, I did all of that, but then I missed that one thing because I didn’t know that existed.

    So today, Google Cloud has a fantastic solution that really allows DevOps team, as they’re pushing code continuously into the cloud, to have the ability with this new Qualys integration to automatically have that capability of assessment of the security of that virtual machine embedded directly and done transparently so that the end user does not really have to go about installing agents and consoles and all of that. It’s done; completely embedded in the platform.

    Ashley: Excellent. So interesting, and we’d love to dive into this some more, because you’re talking about not bolting things on after the fact. You know, this is the whole shift left idea within DevOps.

    Thakar: Yes.

    Ashley: So, I imagine with the Qualys cloud agent for Google that’s something that you’re embedding within the software platforms that developers test production environments you’re using.

    Thakar: Yes.

    Ashley: Is it that kind of a lifecycle?

    Thakar: Yes, exactly. So, developers today use that solution to first of all ensure that in their – whether it’s Jenkins or whatever it is that they don’t even push solutions out to production that already have vulnerabilities, right? The unfortunate side effect of doing this bolting on after the fact is that the moment you spin up a new machine it already has so many vulnerabilities in the last two years, you spend all your time trying to patch them. So, this DevOps pipeline does give us the opportunity to ensure that first of all in your build process you already eliminate and patch your images so that you don’t go out with images that are already vulnerable, and _____ customers absolutely use the Qualys agent and scanning to really achieve that.

    But then once it gets pushed out to product you also want to have a monitoring to ensure that somebody doesn’t go to an S3 bucket or to a Google storage unit to pull down a vulnerable version of some sort of a software in their virtual image and create new vulnerabilities, so you also need to have that monitoring. And so that native embedding ensures that once you have done your DevOps process and cleaned the image, as the image is being pushed into the production environment in the cloud, in Google cloud, it is going to have that monitoring capability already embedded transparently without having to do additional effort.

    Ashley: Great. I imagine it’s not a heavy lift for developers to install this in the development test environment, right?

    Thakar: Yeah, and that’s the beauty of it is everything is automated, right? So, once you write the scripting it’s not a process that requires you to run commands every time and do all of that. Once you push that into the CD process then it’s really transparent, because in this case the developers don’t even need to run any command to get the agent onboarded. It’s an option that they check in the Google security center and then every single image that spins up will have that sort of one-click integration and automatically embedded in there.

    Ashley: Great. I imagine then you’re also tying back into integrating into the security consoles both within the Qualys environment and also within Google?

    Thakar: Yes, absolutely. That’s a unique ability here, because you have multiple business units for organizations. Different teams have different accounts in Google and they are, of course, only looking at their own application or their own account. And so the second part of this announcement really is also the fact that those vulnerabilities and those issues that are detected are then pushed back into the security center from Google so that the individual owners of the accounts can see all of the vulnerabilities on all of the issues that they need to fix right there within their console and they can create playbooks and use the automation that is available to quickly fix those issues so that they don’t have to know go to another tool and go to another console and try to do all of that.

    For the IT folks and the DevOps folks who are pushing these applications into the cloud, they just see their own individual view directly in Google, but then of course this information also is available in the Qualys console, which is a wider visibility into the overall risk posture because it will also include the security and vulnerability findings from other types of infrastructure that the customer also has, things like laptops and handheld devices and containers and other things as well. And so the security team will leverage the Qualys platform because then they get a holistic view of the entire outside vector. The individual team in Google will get their view within the Google console so that they can really focus on fixing what they need to fix.

    Ashley: Really kind of empowers that idea of SecOps, right?

    Thakar: Right.

    Ashley: Better right upfront.

    Thakar: Exactly.

    Ashley: Now I imagine that the security engineers – you know, usually there’s some kind of a corporate security team maybe involved in operations, maybe not, but also would be super pleased about having this implemented earlier into the dev cycle and also that it’s pre-integrated already into the Qualys platform, Qualys monitoring as well as Google’s cloud monitoring.

    Thakar: Right. And that’s really a very good point because that’s a big struggle for security teams, right, is that just getting – ensuring that the entire infrastructure is covered by the security solution is a challenge when you do it after the fact, because they may not even know that there’s a new instance that was pushed by the dev team overnight into the Google account, the security may not know. So, with this integration of course it ensures that no matter what gets pushed Qualys is always monitoring every single image that goes out. Now the security team is good for them because they don’t have to spend a lot of time as they have been doing so far trying to get the security solution implemented.

    So instead of focusing on the findings of the security solution a lot of security teams just spend time trying to get that actually installed and making sure it’s up and running and it’s connecting and all of that. So, having the DevOps team actually do all of that work is fantastic because now the security team can only focus on what is important, which is your policy. So, they can be the one that says, “Well, this is a failure if you have a _____,” and they don’t have to worry about how the thing is installed or whatever it is. Qualys is then the third party that ensures that the policy that the security team has defined is actually tested and a pass or a fail is given to both sides of the house and then they each can go ahead and do what they need to do in terms of fixing and reporting those issues.

    Ashley: Now, you described in the announcement sort of one-click integration. What does one-click mean? Can you visualize that for us, how that works?

    Thakar: Yeah. So we say one-click because if you have ever deployed security solutions they are absolutely painful, you know, trying to get the multiple commands –

    Ashley: Or like one month or one quarter. [Laughs]

    Thakar: Yeah, it’s a thousand clicks and multiple consoles and whatever it is. So the one-click is really because when you go and look at that integration, once you kind of go in Google security center and define that Qualys account that you connect to, then after that it’s really just checking one box that says that include the Qualys agent in every single image and then you don’t have to do anything after that, like you don’t even have to run a separate command every time you initiate a new image because that is already taken care of in a very easy manner with the automation that is provided by Google.

    Ashley: Wow. So it truly is a checkbox kind of process.

    Thakar: Right.

    Ashley: Just included.

    Thakar: Exactly.

    Ashley: Are there any other things that maybe you’re already doing or that are part of this announcement in terms of what you’re doing to monitor scan and report vulnerabilities in the Google cloud environment?

    Thakar: Yeah. I mean we have a very strong partnership with them. We’re listed, of course, in the marketplace as well, and with this sort of integration it really provides sort of a security reference architecture where, you know, Google can now provide along with Qualys customers the ability to make it significantly easy by defining the architecture of how you want to secure your workload that you’re putting in Google and then have – not only define the right solutions that should be used but then also work with vendors like Qualys to get those directly implemented with a single click so you also remove the resistance at integration and really simply that overall process. And so that will enable existing Qualys customers as well as new customers who already are going through the digital transformation, they may have Qualys already on their on-prem infrastructure; now it makes it – it removes another hurdle for them to really move into Google cloud because they can get the same kind of reports and the same kind of analysis as they have been used to because their auditors are very comfortable and used to having the Qualys reports so now they can get the same reports and don’t have to go through an entire process to recertify another solution, so there’s a lot of these kind of benefits that go across the board, and just having that embedded I think really helps.

    Ashley: Yeah. It certainly lowers the bar to entry –

    Thakar: Right, right.

    Ashley: – into the Google cloud if it’s already integrated into the Qualys and Google platforms. Let’s circle back to talking about what are some of the benefits to the DevOps teams getting the Qualys agent into their environment earlier? What kind of things do they learn in the dev cycle and the test cycle group normally they would’ve skipped and not found out till way later when they’re ready to go to production or in production?

    Thakar: Yeah. So because the way the platform is structured with APIs, the biggest advantage, there is automation of – and that’s what DevOps is all about, right? How can I script? You know, you talk about infrastructure as a code. I talk about security as a code, right? Can you just code in to say – you know, like you do your infrastructure, can you say if vulnerability is greater than this than failed _____ or, you know, can you write that as a thing?

    And with the way the Qualys platform is and being a cloud-based platform ourselves and with AP as available, that makes it extremely easy for the DevOps folks to really create directly these scripts that they can basically say, “In your pipeline I got a new image, bring that image up, Qualys agent gets already embedded, it does its analysis and the findings show back up in the console and now you can really code that thing with a human, never having to go in and verify and look at the scans or whatever it is, right?” The way the architecture of the Qualys platform and the agent is, is that agent is an extremely lightweight agent that is not focused on doing just vulnerability assessment.

    It does inventory assessment. It does vulnerability, also configuration assessment. And so what happens is that single agent can be used to look at a larger picture of the security of the device and not just if the software is not updated, right? You can do everything from is unauthorized software being put on this build, is end-of-life software being put on this build, is there configuration that is incorrect even though you may have patch or all your vulnerabilities, and so on and so forth. With that sort of an integration, what happens is that once you have done that one task of getting that agent on the virtual machine you can now suddenly with just APIs do a lot more to get a broader view and make sure that that overall workload is heavily locked down, because now you really are ensuring that all the configurations are as per the golden images and all of the CIS benchmarks are followed and all the software is updated and unauthorized software is already eliminated from that whole thing. DevOps is all about how do I eliminate the manual work and how do I use automation and reduce the amount of work that I have to do, and that’s exactly what that architecture enables us to do.

    Ashley: I think it’s an extremely good point; with all the automation, the increase in speed, the decrease in manual intervention or tasks that are being performed. It’s easy for something to slip in.

    Thakar: Yeah.

    Ashley: New opensource code we haven’t used before or something now has got a vulnerability that didn’t two hours ago and you’ll find out right away.

    Thakar: And, you know, one of the things that we have seen is the DevOps push to go quickly into production and sort of was that initial thing of, “Oh, I want to bypass my operations team, I want to bypass my security team,” and the flipside of that is like now your code that goes directly to production and has an issue, that’s on you as a DevOps – like you cannot blame the security person because, you know, they are not really involved anymore. And so that also incentivizes the developers up front to ensure that they’re doing all the right things from a security perspective also so it’s not their microservice that goes out that has their name on it that gets compromised, right? Because then you can basically say, “I’ve coded everything, done everything on my side up front so that what I am pushing out to production using automation is not something that is already vulnerable.”

    Ashley: Excellent. So, let’s talk about how do you get access to this with the release of this capability? Is it already native within Google cloud? You just connect to it and say, “Here – you know, I’m already a Qualys customer,” or do you have to go to Qualys to sign up for something or how do you get access to this?

    Thakar: Yeah. So, we – and as Qualys we provide directly – if you go to Qualys.com there’s – you can sign up for a free trial if you’re not a Qualys customer and, you know, we’re a SaaS service so you immediately get credentials that you need to access the platform, but then it’s completely embedded in the Google cloud console, so when you got there and there’s the configuration settings you will be able to now pick Qualys as a vulnerability management capability that you would like to add and, you know, there’s a one-time configuration where you provide your Qualys agent information and from that point on, once you have that done, every single – you know, do that one-click checkbox, after that that embedding will happen automatically and Google cloud will be the one doing that for you. So within a few minutes you can be up and running with this integration and starting to see the results in a couple hours.

    Ashley: Excellent. Is there any incremental cost for this or is it already included, just an added capability in the Qualys product?

    Thakar: Yeah. So, it’s a cost that the license of vulnerability management that customers already have. They can basically just take that license or use that or they can come and purchase additional licenses to run additional infrastructure in Qualys, but it’s the same license that they would otherwise use that they can use – that they get from Qualys and they can use in Google Cloud as well.

    Ashley: Yeah. Just the licenses they would need to manage a Google could environment anyway.

    Thakar: Exactly.

    Ashley: Okay. Excellent. Well, great. Fantastic news. Anything else we should know before we wrap up?

    Thakar: No, I’m really – you know, as heading the product, I’m always curious to get feedback, and so I encourage as many of your listeners to go and sign up and, you know, send us feedback on how they feel it helps simplify, and if there’s opportunities for us to simplify the DevOps process even more with better integration we’ll be happy to hear that.

    Ashley: Great. Well, let us know how it goes and we encourage our listeners, go check out Qualys. Qualys.com. And especially if you’re an existing customer, it sounds super easy to start to use this, and I know – I’ve been a Qualys customer and also worked with Qualys before as a partner and super easy to get set up and going, so I encourage folks to check that out. Well, thanks for joining us, Sumedh. Congratulations on the announcement.

    Thakar: Thank you very much. Thanks, Mitch. Good talking to you.

    Ashley: Nice to talk with you. I’d like to thank my guest, Sumedh Thakar, President and Chief Product Office of Qualys, and of course, thank you our listeners for joining us today. This is Mitch Ashley with DevOps.com. Have a great day and be careful out there.

    — Mitchell Ashley

  • Survey Sees Massive Adoption of Microservices

    Survey Sees Massive Adoption of Microservices

    Under increasing pressure to innovate faster, a majority (84%) of organizations taking part in a recent survey have already embraced microservices.

    The survey, conducted by Vanson Bourne on behalf of API management platform provider Kong, polled 200 senior IT leaders in organizations with more than 1,000 employees. Among the results, 89% of technology leaders agree that companies that are not able to effectively support microservices will be less able to compete in the future.

    The report also finds on average organizations are running 184 microservices, with 60% of respondents running 50 or more. The primary reasons cited for adopting microservices are improvements to security (56%), increased development speed (55%), increased speed of integrating new technologies (53%), improved infrastructure flexibility (53%) and improved collaboration across teams (46%).

    Kong CTO Marco Palladino said the report makes it clear that just about every organization is on a journey to making a transition to microservices as part of an effort to reduce reliance on inflexible monolithic applications. Initially, that transition involves lifting and shifting monolithic applications in the cloud before decoupling functions using microservices, he said.

    As part of that journey, Palladino noted, the survey also finds significant microservices challenges remain, including ensuring security (36%), integration with legacy applications (32%), the complexity of management (31%) and updating API documentation (31%).

    Despite these concerns, however, more than 80% of survey respondents who have adopted microservices report that their organization performs well against metrics for development efficiency, the ability to use new platforms, collaboration across teams and sharing of services across applications.

    Overall, a full 90% of respondents said securely connecting applications and ensuring performance at scale collectively represent one of the greatest technical challenges of the 21st century. Also, 71% said they believe that organizations would be out of business within six years if they fail to keep pace with innovation in their industry. For public companies, the urgency is even greater, with 39% reporting they believe organizations will be out of business in less than three years if they fail to keep pace with innovation. Overall, 57% of technology leaders feel organizations are concerned that they are at risk of competitive displacement.

    Pressure on IT leaders is high as a result. Nearly three-quarters of respondents (74%) agree that a failed digital transformation effort will result in a missed promotion, lost bonus or even termination. The report identifies the top three priorities for IT leaders as improving efficiency (47%), reducing cost (41%) and increasing security (40%).

    Technology leaders, however, continue to place the highest importance on the availability (75%), security (74%), performance (65%) and scalability (64%). At the same time, technology leaders are focused on faster development speeds (95%), increased collaboration (94%) and reduced deployment risks (93%) as key desired outcomes for adopting any new technologies.

    The challenges most cited in terms of achieving those agility goals are the complexity of using multiple technologies (35%), lack of automation (32%) and reliance on legacy technologies (29%).

    The survey also notes 83% of organizations are relying on open source software to become more agile. The most commonly used open source technologies are databases (64%), containers (48%), API gateways (41%), infrastructure automation (40%), container orchestration (37%) and continuous integration/continuous delivery (CI/CD) tools (36%).

    In terms of cloud adoption, Microsoft Azure (43%) narrowly beat Amazon Web Services (AWS) (42%), with Google Cloud Platform (GCP) coming in third at 27%.

    On the plus side, 78% of respondents said their organizations are increasing IT budgets in the coming year, with 27% of public companies expecting their budgets to grow 26% or more. Nearly 90% of companies will also be increasing headcount, with 56% hiring internally and 31% relying on more freelancers and/or outsourcing to meet hiring goals.

    Hopefully, with that increased funding more organizations will be able to strike that elusive balance between stability and innovation.

    — Mike Vizard

  • The Automated Data Center Emerges

    The Automated Data Center Emerges

    Hyperscale and automation seem to go hand-in-hand, at least when it comes to the data center. Synergy Research reported that, at the end of 2017, there were 390 web-scale data centers worldwide, up from 300 a year earlier. IDC identified that hyperscale data centers “have a minimum of 5,000 servers and are at least 10,000 sq ft in size but generally much larger.”

    Today, hyperscale data centers are driving a sense of urgency for automation, but many smaller corporate data centers are just as complex, even if they are less physically expansive. In either case, automation takes data from across the data center, applies analytics and machine learning to that data and helps to identify both inefficiencies and trouble spots in the data center–as well as their origins. 

    Some data center functions are well on their way to becoming automated, making new deployments and changes to existing deployments simpler, faster and more efficient than they are today. Among those functions, orchestration, change management, software-defined architectures and DevOps stand out. 

    Meanwhile, another data center function is lagging: problem resolution. The challenge revolves around the siloed nature of the data center, where the multiple, disparate tools and methods used to sleuth out inefficiencies and pinpoint trouble spots lack a unified view of the data center infrastructure. In addition, silo-centric infrastructure monitoring tools lack application context. Short of automation, today’s data centers are stuck in the past. Especially as a data center grows and must interact with cloud-based services, no amount of human intuition in using those tools and methods can overcome the barriers of siloed systems. 

    However, data center infrastructure planners are making strides in identifying the requirements for accelerating the move to the automated data center. 

    Three Keys to the Automated Data Center

    For years, the automated data center was just a dream. And as recently as late 2018, Forrester spoke of islands of automation.

    “For the enterprises I’ve talked to, it’s not like any of them are completely manual at this stage, but they’re all at varying degrees of trying to integrate all these pieces,” said Chris Gardner, Forrester senior analyst. He added that fragmentation was due to a lack of holistic solutions that could connect all aspects of the data center.

    But now, three forces are shaping up to help make the automated data center a reality. Let’s look further. 

    Real-Time Visibility 

    Real-time infrastructure visibility has become a critical element of automating the data center. Why? Because an issue can occur anywhere on the virtualized, hybrid infrastructure, and the issue can ripple to create problems elsewhere, making issue resolution a perennial challenge. 

    For that reason, your best defense is to build deep, real-time visibility into the compute/virtualization stack and the storage/storage network stack. This will help you quickly ensure the performance of mission-critical applications accessing the data from those stacks.

    For ITOps, it is about proactively pinpointing the source of the issue and resolving it before application users are impacted. That requires stitching data across the full stack of technologies supporting the business: application services and infrastructure services, both on-premises and in the cloud. 

    Real-Time Analytics

    Why is real-time analytics crucial in the data center? In short, it’s because real-time responsiveness is critical for supporting anomaly detection, root cause analysis, remediation, prevention and planning use cases, where processing and correlating massive amounts of information is critical.

    For ITOps, it means distributed real-time architectures must combine cloud-based and on-premises analytics that leverage AI and machine learning. It’s the analytics and correlation that helps you make sense of all that real-time data so it becomes actionable.

    Closed-Loop Systems

    IT organizations are acutely aware of the perils of introducing changes to critical production environments. In fact, they take great pains to establish a governance of change. Closing the loop is about humans delegating decision-making to technology.

    For ITOps, that means ensuring integrated control of change processes across the entire technology stack. Data and analytics should lead to optimal real-time decision-making that empowers the automation for closed-loop operation.

    How Automated Is Automated?

    In 2016, researchers at the University of Pisa argued that hyperscale data centers would have to rely completely on analytical, automated systems to have to any chance to manage the huge number of day-to-day issues their servers generate. They added that “large computer systems in general, and data centers in particular, will ultimately be managed using predictive computational and executable models obtained through data-science tools, and at that point, the intervention of humans will be limited to setting high-level goals and policies rather than performing low-level operations.”

    Now, more than three years later, critical technologies and methodologies are in place to make the automated data center a reality, even if humans are still required for guidance, if not for sleuthing out infrastructure issues.

    — Len Rosenthal

  • The Foundational Era of Open Source

    The Foundational Era of Open Source

    Contrary to popular belief, software is not eating the world. Open source software is eating the world. This was very apparent at the Linux Foundation’s KubeCon + CloudNativeCon event recently in San Diego. Driving this phenomenon is what I call the Foundational era of Open Source.

    This Foundational era of Open Source has replaced what I call the Big Brother era that came before it. The Big Brother era itself replaced what I call the Cathedral and Bazaar era of open source.

    The Cathedral and Bazaar Era

    I first became involved in open source software in the mid-90s. I might have dabbled and used open source software before that as I searched for free tools using WAIS and Gopher. But I really became aware when I founded TriStar Web, an early hosting company. Back then, web server software came in basically three flavors: 1. Netscape Web Server, from the people who brought us the Netscape browser (it was my choice); 2. Microsoft’s Internet Information Server (ISS), which was that era’s version of the Evil Empire; and 3. Apache (I don’t even think there was an Apache Foundation yet), the open source web server that was taking the web by storm. Besides being free, it was a lot more flexible and capable than the other two.

    During this time there was, of course, another open source software vying for dominance in the market. Something called Linux developed by a guy named Linus Torvalds was taking on all of the giants that had their own flavors of Unix, while Bill Gates’ Windows was fighting it out against IBM’s OS/2, which he also developed.

    Yes, those were heady days for open source software. I call those days the Cathedral and Bazaar days of open source. The Cathedral and Bazaar, of course, refers to the collected essays and then the book of the same name by Eric S Raymond. It, more than anything else, laid out for me the power of open source software and the tensions between the worlds of open source “free as in beer, free as in freedom” versus closed source software.

    Let’s be clear, the open source software world would not exist but for the lessons learned, battles fought and wins and losses of this era.

    Anarchy, Chaos or Freedom?

    In those days you had people such as Dr. Richard Stallman advocating for free, open software that you could use, distribute and modify as you wish. There were intellectual debates over the pros and cons of various open source licenses (Apache, GPL, etc). Most of all, it was very difficult to bring open source software into the enterprise. No support, no training, no SLAs … how was the enterprise to use open source?

    Nevertheless, open source prevailed—sometimes on sheer quality, other times on the love of developers who relished in the freedom of open source. It became a force in the tech world. Companies such as Red Hat, Suse and Canonical capitalized on Linux. Other companies tinkered and experimented with models to try and become commercially successful with other open source tools. By and large, it was a rough go, but some did succeed.

    In my own case in InfoSec, we had great open source tools including Snort, Nmap and Nessus. The folks at Tenable did stop developing an open source version of Nessus, but they still offered a free version of it. Yes, it was a great time in open source—or so I thought.

    The Big Brother Era of Open Source

    Sometime in the early 2000s, we passed into what I call the Big Brother era of open source. This was characterized by almost every open source project having a managing or controlling commercial entity dictating its direction and code. While this didn’t happen with Linux, it became the de facto standard.

    Companies were spun up by the founders of an open source project and while the project itself remained open, it was owned lock, stock and barrel by the commercial entity. The company would sell training and support and in some cases open core versions of the software with “freemium” modules available for a price. Eventually, SaaS-based, hosted versions of the open source software would be another business model companies used to monetize open source.

    Issues regarding who owned the code, who contributed to these projects and where the community fit in often caused conflicts. But by and large, the open source communities counted on the benevolence of the Big Brother to make sure the open source versions of the software remained viable.

    In retrospect, though, this era stifled contribution by other entities: Who would want to contribute code to an open source project for the benefit of the controlling entity? Many companies forked their project so they could control their own versions, resulting in the Balkanization of many of these projects, with the community being the big loser.

    The Foundational Era of Open Source

    Between 2012 and 2015, though, we passed into a new, modern era of open source software that I call the Foundational era of open source. Open source projects were given to community-owned, non-profit foundations to manage and nourish—organizations such as the Linux Foundation, the folks behind Cloud Native Computing Foundation and the Continuous Delivery Foundation; Cloud Foundry Foundation; The Apache Foundation; and the Eclipse Foundation (one of the early entrants, even if IBM did come to dominate it) took over the stewardship of many open source projects.

    This has ushered in a truly golden age of open source software that has seen open source become the dominant form of software in the world. The Foundational era of open source has flourished for several reasons. For one, now multiple commercial entities can contribute code without worrying that it is for the exclusive benefit of just one company. What’s more, these foundations have a mix of vendors, practitioners and large enterprise users that really do allow for the democratization of open source at levels not seen before. Also, there is a process of incubation to graduation that these projects are put through that help them evolve and grow.

    Not to be underestimated is the new generation of developers who have grown up with open source and only want to work in open software. They are putting their influence into practice. Instead of sneaking open source in the back door, they are knocking down the front door with it.

    There are other reasons, but the result is that open source today is now in use at well over 90% of enterprises and it is by far the dominant form of software. It is leading the way in the move to the cloud and in business transformation, and it is literally changing the world.

    Long live open source.

    — Alan Shimel

  • Raytheon Leans on Red Hat to Advance DevSecOps

    Raytheon Leans on Red Hat to Advance DevSecOps

    Raytheon Company is partnering with Red Hat to drive the adoption of DevSecOps workflows and processes it developed around the OpenShift application development and deployment platform.

    Jon Check, senior director for cyber protection solutions for Raytheon Intelligence, Information and Services, said Raytheon has developed a set of DevSecOps practices for organizations building applications deployed in highly secure environments, involving government contracts.

    Raytheon and these customers have been challenged by a chronic shortage of IT professionals with the appropriate level of clearance to work on these classified projects. To overcome that issue, Check said Raytheon developed what it describes as a “code low, deploy high” approach to DevSecOps. Developers who lack security clearances can still build applications; however, those applications can only be deployed by IT professionals having the appropriate security clearance.

    In addition, Check said Raytheon has developed integrations between its DevSecOps framework and various IT tools based on the ITIL framework, which so many IT operations teams depend on to foster collaboration across the application development and deployment process. For example, he said, whenever code gets checked into a repository, an alert can be sent to an IT service management application from ServiceNow.

    That approach enables organizations to build applications quickly without compromising the integrity of the deployment platform, Check noted. Now Raytheon is working with Red Hat to drive adoption of its approach to DevOps to organizations beyond the core vertical industry and government sectors it serves.

    Check said Raytheon chose to work with Red Hat because organizations are now building applications employing microservices based on containers that they want to be able to deploy on any cloud computing environment. Those environments include edge computing platforms to process data in near real-time closer to the point where data is collected.

    In addition, Check noted Red Hat OpenShift is built on Red Hat Enterprise Linux (RHEL), the first operating system to achieve Common Criteria certification with Linux Container Framework Support. Red Hat also assumes responsibility for curating the underlying open source operating system and application deployment platform.

    In general, Check noted that while there’s a lot of interest in deploying code faster these days, the deployment of insecure applications is counterproductive. Organizations too often find themselves rolling back application deployments to deal with one cybersecurity issue or another. By separating application development and deployment, Raytheon is making the case for a DevSecOps approach that enables more secure application code to be deployed faster, he said.

    Each organization will have to determine what rate of application deployment is fast enough for them. The more sensitive the application is, the more an organization is likely to want to be more deliberate when it comes time to deploy and update an application. However, the need to be deliberate doesn’t necessarily mean organizations should continue to rely on legacy approaches to application development and deployment based on waterfall methodologies.

    — Mike Vizard

  • Oracle v. Google and Its Impact on APIs and Code

    Oracle v. Google and Its Impact on APIs and Code

    What, if any, impact will the Supreme Court’s rulings on Oracle v. Google have on the industry at large?

    TL/DR: It is unlikely to impact the vast majority of us, vendor and enterprise developer alike.

    Disclaimer: I am a technologist, not a lawyer. I have done some research, and talked to lawyers, but all that I say here is opinion—even my direct quotes of lawyers are their opinion. The Supreme Court can and has caught people off guard and taken cases in different directions than expected. Though I believe this path is the one it will follow, nothing is assured until the Court rules.

    Throughout this blog, references to The Case are actually using Harvard’s case document.

     

    Background

    On Nov. 15, the U.S. Supreme Court agreed to hear a dispute between Oracle and Google. The Oracle v. Google case has been going on for nearly a decade. And its roots go further back than that.

    The origins of the case go back to 2005, when Google and Sun Microsystems discussed the licensing of Java for Android. This will likely be important because it shows that Google knew it was someone else’s intellectual property (IP). More on this later.

    The crux of the case falls into three parts:

    1. Eight files that were decompiled and used. In one court case, these were found to never have made it into distribution and only used for testing (which limits liability).
    2. Two source files from Java used directly. These were found to be infringing, but damages were set to zero with consent of both legal teams.
    3. The part at issue: the copying of the entirety of Java’s interface from 37 modules. Most tech writers imply this as 37 APIs. It is not—it is hundreds of classes and thousands of methods. The 37 is the number of Java packages.

    The case has been through four court hearings/rulings so far. To sum up the relevant parts of these cases:

    1. The first case (split between a jury and district court, running at the same time) included patent violations. Those were defeated and Oracle has not appealed that defeat. On copyright infringement, the court found that Google did violate copyright on the 37 packages and the directly copied source code, but not on the eight decompiled files. The jury could not settle the issue of “fair use” of the 37 packages.
    2. The district court decided that the eight decompiled files and the copied source code for one function were violations, but remanded the rest back to the jury trial to settle the issue of fair use.
    3. In the end, the district court determined that APIs are not protected by copyright. Their decision involved two parts:
      1. That there is only one way to write a given API.
      2. The interface code and hierarchy can be replicated by third parties without violating copyright because it is an interface and not content.
    4. On appeal, the finding was reversed, and APIs were deemed copyrightable, sending the case back again.
    5. At this point, Google asked the U.S. Supreme Court to intervene. The court chose not to.
    6. Back at jury trial, the court found that while APIs may be copyrightable, Google’s use was fair use in the legal sense. This is the one topic the jury could not originally agree upon.
    7. Oracle appealed this decision, and the District Court ruled that it indeed was not fair use to copy the structure and exact interfaces.
      1. Where the jury and the district court appear to disagree is in the idea of APIs. Calling a publicly documented API is obviously fair use, and the jury latched onto that idea. The District Court appears to acknowledge the difference between calling an API and copying it.
    8. And after some tumultuous hearings and years in the court, the Supreme Court has agreed to weigh in on the issue.

    The real problem here is that copyright law is not a great fit for software. It never has been. If I rewrite your Java application in Python, using the same exact structure and naming, is that protected? All of the “words” have changed. It is akin to rewriting your Hindi book in English without a direct translation  … except that in software there is the added issue of it behaving exactly the same.

    But we can’t fix the misapplication of copyright law to software—only Congress can. So what to expect?

    A fair number of people, from the open source community and from businesses that see opportunity in a ruling that favors their world  view, are advocating that affording copyright protection to APIs is a horrible idea because it would hurt independent developers. This is not quite true. Independent developers are not generally copying Java. While this case does have bearing on coding practices and what developers can do, it does not have a huge impact on innovation, just on copying others’ work and how that is achieved—much like the “reverse engineering” court cases of the ’80s.

    There are good arguments on both sides of this case, but it seems to me that fair use will lose. They are public APIs in the sense that anyone can call them; indeed, some of these packages are so core to the system that you could argue every app must call them. But calling a function is not the same as re-creating it. In that sense, they are not in the public domain. Anyone can read a book, but that does not make it public domain.

    But no matter how this turns out, it is all good for the industry. This uncertainty has hung over the IT sector for nearly a decade. Just like when a new boss comes along and there is uncertainty in the department until everyone adjusts to the new demands/style of the boss, a solid ruling will allow settling to start. The industry can adapt once it knows what the rules are and we can move along.

    The real long-term answer is to have an equivalent to copyright and patent that is designed around the unique parts of computer code. Clearly, software is not the same beast as a book or magazine. The considerations are half-patent and half-copyright, and trying to bend those two to meet the needs of software has created some serious issues. Oracle quoted a 1995 case to showcase this problem—the company is quoting it to win the appeal, but it should be quoted as the reason the protections need to be designed for this medium that is not a publication and yet consists of words whose sequence matters: “Applying copyright law to computer programs is like assembling a jigsaw puzzle whose pieces do not quite fit.” (attributed by Oracle to the concurring opinion of J. Boudin in Lotus Dev. Corp. v. Borland Int’l, Inc., 1995).

    One interesting bit that the courts thus far have ignored, and I would love to see the Supreme Court take up, is this: If Google entered into licensing negotiations, the company knew it was Sun/Oracle’s IP. To then go and copy parts of it without licensing appears wrong on its face. It would be interesting to understand the process that ended in the copying and hear whether that bears upon the legal standing of the case. I can think of both malicious and innocent routes to get there; it would be interesting to hear testimony about motivations and thought processes.

    But even without delving into the licensing negotiations, this case stands to redefine copyright with regards to software. J. Michael Keyes, a partner at the international law firm Dorsey & Whitney, had this (and more) to say:

    “One of the issues that the court may decide is the scope of copyright protection for software. Google had early success below in convincing the trial judge that the snippets of copied code were not copyrightable. Google may urge to have the court take a look at this issue anew. Another huge implication is the scope of fair use. The jury found that Google’s use was “fair” and that it was not liable for infringement. The appellate court overturned that jury conclusion and found that Google’s use was not fair as a matter of law. The Supreme Court will likely wade into this thicket and provide guidance on the scope of fair use in the context of software code and what role an appellate court plays in reviewing jury decisions in that regard.”

    What we (all of us) do not want to do is confuse “good for the industry” with “legal standard.” It is not the court’s job to do what we think is best for the industry; it is the court’s job to determine applicable law and what that implies needs to happen. Presumably, the Supreme Court will do just that. And it may not please any of us. But if we think the current iteration of the law is flawed, that is something to take up with elected representatives, not ask a court to “fix” on the fly.

    For the vast majority of us, though, it impacts our development not at all. Using APIs or forking open source projects are all still perfectly acceptable, and no ruling by the Supreme Court is going to change that—because copying someone’s APIs is the issue here, not using them. And open source licenses allow for copying and improving projects. So carry on. Get the job done. And read about the Supreme Court’s final decision with a smile, because either way, the industry will adapt to the new normal and continue forward.

    — Don Macvittie

  • The Road Ahead for Security, DevOps Transformation

    The Road Ahead for Security, DevOps Transformation

    Qualys hosted its Qualys Security Conference 19 this week in Las Vegas. Hundreds of attendees from around the world have gathered at the Bellagio hotel to learn more about the current state of cybersecurity and what the future has in store. As you might expect, given the technology landscape today, many of the sessions and discussions have revolved around DevOps and how cybersecurity can adapt to DevOps culture.

    Scott Crawford Qualys Security Conference
    Scott Crawford, Research Vice President at 451 Research presents “The Road Ahead for Security, IT, and DevOps” at the Qualys Security Conference

    The Shifting Technology Ecosystem

    Scott Crawford, research vice president at 451 Research, opened day two of the Qualys Security Conference with his keynote, “The Road Ahead for Security, IT, and DevOps.” Crawford talked about the dramatic changes over the last decade or so—from monolithic to microservices, from standalone applications to integrated software, from Waterfall to Agile, enterprise networks to IoT / OT, from standard networks to 5G wireless, and from traditional IT to DevOps. He discussed the challenges organizations face and the dilemma of doubling down on past success and missing out on innovation.

    With all of the changes from this shifting technology ecosystem, there are consequences. Organizations no longer have a single point of control and IT teams are faced with managing and protecting an increasingly complex web of interconnected platforms, applications and services.

    Security and DevOps

    Crawford also talked about the rise of zero trust. He described it as a sort of mashup of the concept of least privileged access, combined with behavioral analytics and an on-demand, just-in-time approach to managing access. The zero trust paradigm is based on the premise of never trust, always verify—and that means determining who is requesting access, under what conditions they’re requesting access and what actions they intend to perform with that access on a case0-by-case basis.

    He shared the Xebia Labs Periodic Table of DevOps tools to illustrate the overwhelming number of options available and stressed the importance of automation. CI/CD (continuous integration/continuous deployment), RPA (robotic process automation), and SOAR (security orchestration, automation and response) solutions have been developed to help address and manage some of the complexity of both DevOps deployments and cybersecurity.

    Xebia Labs Periodic Table of DevOps Tools
    Xebia Labs Periodic Table of DevOps Tools

    DevOps and Digital Transformation

    Crawford’s keynote was followed by a presentation titled, “The DevOps Transformation of the Qualys Platform: Lessons Learned.” Dilip Bachwani, senior vice president of engineering and cloud operations for Qualys, talked about how the company has addressed the “innovator’s dilemma” and steps it has taken to deal with growing pains as it goes through its own digital transformation.

    Bachwani explained that the engineering team at Qualys has expanded exponentially in a relatively short period. That comes with several benefits in terms of how quickly they can develop and implement new solutions, but the team also faced challenges with balancing that growth and trying to maintain velocity at the same time without sacrificing stability or reliability.

    Just like every other organization going through a DevOps transformation, Qualys has had to break down silos with people, processes and tools. Bachwani explained that they needed to do things differently to automate as many functions as possible. They decided to view infrastructure and operations as software problems and find ways to fully automate deployment and configuration.

    He also emphasized that security must be built in throughout the DevOps life cycle. Qualys adopted a hub-and-spoke DevOps model and developed a robust DevOps toolchain designed to combine and orchestrate an effective set of tools for developing, delivering and maintaining software. Having a standardized toolchain ensures that there is a common platform available to everyone.

    Accelerating Transformation With Citizen Developers

    At the end of Crawford’s keynote, he talked about the lack of people with the right knowledge and skills. Cyber Security Ventures predicts that there will be 3.5 million unfilled cybersecurity jobs by 2021. That’s just cybersecurity. What about developers and other key personnel? Crawford posed the question, “Where will we find the people to do it all?”

    The answer, he said, is citizen developers. What is a citizen developer? It’s a movement to empower employees who are not developers by trade with the tools and support to implement ideas. They are the ones closest to the problem and, arguably, best equipped to understand how to solve it. They don’t need to be professional developers or need experience with coding environments. Low-code and no-code platforms give them the power to turn ideas into solutions.

    A few years ago, George Hulme wrote an article on the growing citizen developer movement. Hulme noted, “The organizations that succeed with citizen development will be those that wholeheartedly embrace the movement and cultivate it, with IT’s help and guidance.”

    The DevOps revolution isn’t going away. Companies that don’t embrace digital transformation will likely fade into oblivion as they are left in the dust by more innovative competitors, and organizations that don’t figure out how to integrate security into the fabric of their culture will struggle against a rapidly expanding and evolving threat landscape.

    At this point, these things are like the ocean tide. It is coming whether you like it or not, and it will destroy those who are not prepared. Crawford wrapped up by asking the audience to consider what their role is going to be in building and maintaining the future of DevOps and cybersecurity.

    — Tony Bradley

  • 5 Best Practices for Feature Flagging

    5 Best Practices for Feature Flagging

    Feature flags, also known as the feature toggles, are important concepts in the field of DevOps. They have been around for a long time and have been used continually by developers for their convenience. As software development services providers are striving for higher agility, practices such as feature flagging are becoming increasingly relevant. If you are not familiar with the concept of a feature flag, let’s start with understanding what feature flagging is.

    What is Feature Flagging?

    Feature flags are used for the process of branching your code, which means to turn certain code paths on and off to separate the release of new features from the actual deployment of our code. Why do DevOps developers want to branch their code through feature flags? Well, feature flagging is a great software development practice for two reasons. Firstly, for features that are a work-in-progress, developers can use a feature flag to hold their release until they are ready. This ensures the release of other features that are ready is not delayed just because one feature is not. Secondly, features that are being worked on can be kept off using the feature flag and only once it is ready it can be turned on. This allows developers to work directly on the feature without changing the main code, also known as the trunk or master branch. When all is working and ready to be released, turning on the feature toggle releases the feature, just like a switch.

    To understand feature flags in the simplest term, feature flags are “if/else” statements to develop a conditional branch or pathway into your code. For example, you can use a feature flag to state that “if so and so feature is enabled, run new algorithm; or else, run old algorithm.” This allows developers to control the functionality—if the feature flag is enabled only then the new algorithm is executed; else skipped. This is the most basic form of feature flags, however, they could be more complex.

    Feature flags are a practical way for software teams to maintain the delivery and release of features, and there are many advantages of using them. However, too many feature flags can make your code more complex and difficult to manage.

    Here are the best practices of feature flagging that can save your project from becoming a feature flag hell.

    More Feature Flags Equals Complex Code

    Feature flagging is a good practice that can help you maintain the delivery and release of different features, however, using too many feature flags can quickly turn your code into a feature flag hell. The more feature flags you create, the more code paths are going to be created, this increases the complexity of your code and makes the testing process harder. A higher number of code paths can cause a mess and risk the functionality of your code. Therefore, it is best to only create feature flags when it makes sense and always check if a previously created feature flag can be re-used instead.

    Name Your Feature Flags

    For efficient management of feature flags, it is crucial to have a proper flag naming system. Without it, it is possible to confuse the flags, forget why they were created for, or create flags with the same name—all of which can cause disruptions and confusion. When working with a team of developers, it is important to ensure that everyone follows the same labeling method so they can get the information about the flag just by looking at the name. Otherwise, it is likely that a team member might mistakenly trigger the flags, upsetting your code.

    Log and Clean Up Old Flags

    Over time, it can become difficult to keep track of the changes that are made to a feature flag. Therefore, it is important to track changes through logging, especially when working in teams. Using logging, you can keep a track of who created the feature flag, who made a change and at what time. This can save you from wasting time figuring out who was responsible for what change. A log will set the record straight about every change introduced for every feature flag.

    This is also great to know which flag and code paths are obsolete and redundant now. When you know which flags are unused you can clean them from your code to avoid any conflict or confusion.

    Use a Feature Flag Management Tool

    For better management, control and visibility of your feature flags, you can use available feature flag management tools. Like every other management tool, feature flag management tools are enterprise-scale software to manage feature flags. Some of the most popular tools include LaunchDarkly, Rollout and Bullet Train. Tools like these can make the management of feature flags easier for you and increase your development speed.

    Use Feature Flags to Collect Data

    When it comes to scaling your product, feature flags can be of great help to collect data from users. For instance, if you want to introduce a new feature, you can roll out the change slowly to a few users first. Using feature flags, you can make the new feature or update available to a smaller set of users only and collect data about how well they respond to the release. You can then use their feedback to improve or change your features to deliver an improved user experience. Feature flags are used in this manner by companies such as Facebook and Netflix to test their new features before releasing them for the entire user base.

    — Emily Williamson

  • WhiteSource Acquires Renovate to Automate Dependency Updates

    WhiteSource Acquires Renovate to Automate Dependency Updates

    WhiteSource has acquired Renovate, a provider of an open source automated dependency update platform that the company plans to make available for free.

    Rhys Arkins, co-founder of Renovate who is now director of product for WhiteSource, said that as software projects become more complicated, there’s an increased need for a mechanism to identify and track dependencies and automatically update them as changes and updates are made.

    That requirement is becoming especially acute with the rise of microservices-based applications, which substantially increase the number of dependencies that exist within and between applications, added Arkins. Microservices make it possible for DevOps teams to build and deploy applications faster, but the dependencies between the microservices often result in an application environment that becomes too complex to manage. Most recently, Renovate added support for Helm Charts used to package applications in Kubernetes environments, on which many microservices are now being deployed.

    In addition, WhiteSource plans to add support for Cocoapods, a dependency manager for Swift and Objective-C Cocoa projects.

    Prior to the acquisition, WhiteSource was reselling the Renovate platform, which has more than 150 contributors. Now that WhiteSource owns the Renovate platform, including a hosted GitHub application and self-hosted GitHub and GitLab applications, the company has decided to make it available for free under the WhiteSource Renovate brand name.

    In the future, The WhiteSource Renovate app will add support for Bitbucket Cloud and Azure DevOps, thereby expanding the sources of dependency data that can be employed to make updates less risky and time-consuming, said Arkins. In effect, WhiteSource wants its automated dependency platform to become a natural extension of any continuous integration/continuous delivery (CI/CD) platform, he added.

    Arkins said that when it comes to managing dependencies, the biggest challenge is getting developers to identify them. In the absence of a platform for managing that process, developers will either not declare dependencies at all or, more commonly, not provide the right version number for a specific release of software on which their application is dependent. The more automated that process becomes, the less likely it is DevOps teams will find applications breaking because of a dependency that they were unaware of or lost track of at some point during the project.

    With the rise of DevOps, many organizations are now employing a more structured approach to dependencies. However, Arkins noted, many organizations are unaware that an open source platform is available that automatically updates all dependencies.

    WhiteSource is best known for helping organizations discover vulnerabilities in open source code. With the acquisition of Renovate, the company is furthering its ambitions to become a provider of software composition analysis tools.

    It remains to be seen just how many organizations will incorporate an automated dependency update platform into their DevOps processes. While many organizations have embraced DevOps, the level of maturity in terms of DevOps sophistication is uneven. The more software projects an organization launches, however, the more likely it becomes apparent there is a need for a better way to manage software dependencies.

    — Mike Vizard