Tag: open source

  • State of the Software Supply Chain: Secure Coding Takes Spotlight

    State of the Software Supply Chain: Secure Coding Takes Spotlight

    After almost a year of research that involved studying 36,000 open source software projects, 12,000 enterprise development teams and 3.7 million open source releases, we at Sonatype are excited to share the “2019 State of the Software Supply Chain” report.

    This year, we worked with research partners Gene Kim, founder of IT Revolution, and Dr. Stephen Magill, principal scientist at Galois and CEO of Muse, to examine and empirically document objectively for the first time the attributes of exemplary development practices, especially in relation to secure coding practices. But, as in years past, we’ve also analyzed the rapidly expanding supply and continued exponential growth in demand for open source components.

    Not All Open Source Projects Are Created Equal

    For the past four years, we’ve studied the ins and outs of the software supply chain—what it’s comprised of; how vulnerabilities are getting in and how often; the growing regulations; and, most recently, a new trend in which adversaries are purposely attacking the supply chain with malicious components.

    For our fifth anniversary of the report, we wanted to look deeper. We wanted to understand exactly how enterprise development teams—and potentially even more importantly, how OSS projects—were thinking about and addressing the software supply chain security issues. We wanted to understand and identify the very best practices so we could share them with others.

    As a result of our research, we identified five common behavior patterns across 36,000 open source development teams. This includes identifying attributes of Large Exemplars and Small Exemplars who rest within the top 3%, or 1,229, OSS project development behaviors.

    To arrive at this list, we examined a large number of variables, including:

    • Do differences exist in how effectively OSS projects update their dependencies and fix vulnerabilities?
    • Are there exemplary teams that do this better than others?
    • Are components from exemplary teams more widely used than “non-exemplary” components?
    • What factors correlate with exemplary components?
    • What advice can be offered to producers of OSS components and the developers that consume them?

    The answers were quite striking—and the resulting data even more illuminating. While the report identifies Small Exemplars and Large Exemplars, we’ve also identified three additional groups of OSS projects: Laggards, Features First and Cautious.

    Secure coding: 2019 State of the Software Supply Chain

    Exemplary Commercial DevSecOps Practices Create Superior Software

    There are clear, competitive advantages for teams with exemplary DevSecOps practices.

    We’ve known for years that innovation is critical, speed is king and open source is at center stage. This research further underscores these accelerating trends throughout the software supply chain. It also shows that taming the supply chain is possible. By making better supplier choices, component selection and using automation, dev teams are seeing impressive rewards. In fact, for those development teams actively managing their software supply chains, the use of known vulnerable component releases was reduced by 55%.

    The report details 11 other behaviors and attributes of leading enterprise development teams, including their frequency of software releases, their use of repository managers and their reliance on a software bill of materials.

    Gene and Stephen helped shed new light on exemplary development and DevOps practices that I believe will help developers around the world better understand what secure coding means and how to start addressing it.

    — Derek E. Weeks

  • 5 Ways DevSecOps Can Manage Software Supply Chains

    5 Ways DevSecOps Can Manage Software Supply Chains

    The unbridled use of open source components within the software supply chain is on a major uptick, according to new research. Even as this surge in open source dependencies fuels faster innovation, the study shows that it comes with high cybersecurity costs, as the number of breaches related to these components is similarly on the rise.

    However, the numbers also indicate there are some clear practices used by the best DevSecOps teams to minimize the risk while maximizing the speed and agility that third-party components offer.

    Sonatype and research partners from IT Revolution and Galois made a thorough analysis of download patterns, software component makeup and DevSecOps behaviors around open source components and third-party libraries for the “2019 State of the Software Supply Chain Report.” Their analysis found that the rate of open source component releases has picked up by 75% in the last two years and that download requests from teams using these components have risen 68% year over year.

    Most tellingly, the study showed that among modern applications today, 85% of the average code base is comprised of open source componentry. Not only are the majority of applications using open source components and libraries, but also they depend on them heavily.

    “The practice of assembling open source component releases into the form of an application screams of efficiency. Developers no longer need to code every line from scratch,” according to the report. “Developers can download component releases in seconds that deliver new capabilities, built by experts outside of their organizations who make their code freely available to others.”

    Perhaps not coincidentally, the study noted a parallel rise in breaches associated with open source component use. Last year 1 in 4 organizations suspected or verified they experienced a breach related to open source components in the last 12 months. That’s a slight improvement over last year’s stats, which had it pinned at 1 in 3 organizations. But since 2014, that rate of breaches has risen by 71%.

    The trouble is, many of the components developers download and integrate into their code are full of dangerous vulnerabilities. For example, just over half of all JavaScript package downloads contain known vulnerabilities, with 1 in 3 rated as high vulnerabilities. One in 10 of those flaws were rated as critical.

    The point the report hammered home is that not all components are created equally. There exists a certain class of well-run open source projects that release updates more frequently, update problematic vulnerabilities faster and remediate flaws more quickly. It is up to DevSecOps teams to systematically manage components, so they leverage more of these high-quality open source projects and try to avoid the riskier open source dependencies.

    “Management of software supply chains is not simply ensuring quality at velocity. Our supply chains are being attacked by adversaries in new and creative ways,” according to the report. It found that organizations that actively manage their supply chains are able to reduce the proportion of vulnerable components by 55% compared to those who do not.

    The report found several software supply chain practices contributed to this reduction of risk, with the following five standing out as the most impactful:

    Develop a governance program: Foundational to this is having some sort of open source governance or standards policies in place, according to the report, which noted that among the best DevSecOps organizations at managing risky components, nearly 3 in 4 have such policies in place.

    Have a process for evaluating and adding new components: As a part of that open source policy, the best teams have standardized processes for how developers add new open source components. This includes how these components are evaluated, approved, standardized and so on. The exemplary organizations are 11 times more likely to use this kind of process for new dependencies.

    Proactively remove problematic components: New vulnerabilities are discovered every day, so simply evaluating a dependency upon its first inclusion in an application can only take an organization so far. The best organizations also take the initiative to look for potential sources of open source vulnerabilities in existing code. That includes proactively seeking out and removing known vulnerable dependencies, as well as reducing the attack surface of the code by eliminating open source dependencies that aren’t being used. The best organizations are 9.3 times more likely to do so.

    Aim to always use the latest versions of components: It should go without saying that the latest version of any given component has the least vulnerabilities. The report showed that component releases that are 3 years or older have a 65% higher defect rate compared to newer components. Exemplary organizations strive to use the newest updates of all of their open source components and are 6.2 times more likely to do so than other organizations. That doesn’t happen accidentally—these teams are 10 times more likely to schedule update dependencies as part of the daily work of DevSecOps teams.

    Use Enforcement Automation: Just having a policy in place is not enough—organizations need to enforce the open source standards to reduce risk meaningfully. The study showed that DevOps practices combined with automated enforcement of policies increase developer adherence to standards by 150%. The exemplary teams with the lowest open source defect rates are 12 times more likely to have automated tools to track, manage and/or ensure policy compliance of open source dependencies.

    All of these best practices add up to less risk from components, as well as less pain for DevSecOps teams, who are three times less likely to consider the process of updating components as painful.

    — Ericka Chickowski

  • A Golden Age for Developers

    A Golden Age for Developers

    There’s probably never been a time since the dawn of programming when there has been more opportunity for developers—individuals and teams—to imagine, create and be successful. Developers are able to do a lot more (and do things a lot faster) with more tools, a better development ecosystem and a tighter connection with the rest of the enterprise than was possible even a few years ago.

    To me, we are in a golden age for developers. And it comes just as the need for developers has never been greater, in every enterprise.

    Three things have come together to create this moment: Culture, code and the cloud.

    Changing Culture

    First, developers today operate in a culture that is far more collaborative than in years past, both internally and externally. A big part of that is the growing adoption of the DevOps methodology, which breaks down silos between business units, IT and development through partnerships between developers and the enterprise. It’s changing how we work and think.

    DevOps leads to a process of continuous integration of new ideas and features that meet business needs, and continuous deployment life cycles that get those features operational rapidly. Rather than taking weeks or months to get a new version of an application up and running, it’s possible to make constant incremental changes and have things running in days or even hours. The enterprise is able to be exceptionally agile responding to business changes and challenges—and, if there’s a problem or failure, the mean time to recovery is just as fast.

    Democratized Code, Empowering Cloud

    A major factor in enabling this approach is the widespread use of open source code. Open source has democratized access to powerful tools and platforms that can greatly accelerate work for any developer. Open source projects are mature, stable and growing, which provides equal access for citizen and enterprise developers alike. The size of your IT operation no longer matters: Developers everywhere can call up on their laptops the same tools that once were available only to web-scale unicorns. It removes limits to a developer’s imagination and the ability to create something new.

    Finally, the ever-expanding realm of cloud-native computing is putting incredible resources and computing ability within reach of every developer. Tools such as Kubernetes, which let you create and orchestrate applications in containers that can then be deployed and run in the cloud; serverless technologies; and other new ways of using distributed computing power remove barriers. Where a developer once might have been reluctant to create apps that required high levels of computing power, that’s not the case now with the cloud. What once may have taken access to a Cray supercomputer is now at a developer’s fingertips.

    Retraining, Not Replacement

    If there’s any challenge in this golden age, it’s helping the enterprise to make the transition—in particular, facilitating the leap that existing development teams must take in using these new approaches and new tools. In fact, the top issue for cloud developers is to make this cultural change.

    That takes both strong top-down leadership and an openness by veteran developers to new ways of working for developers, especially those who’ve been around for a while.

    Too often, we see companies trying to make the switch by offering early retirement to their veteran developers and trying to recruit a new team that’s already used to DevOps, open source and the cloud. That’s the wrong way to do it. Instead, even as they bring in newer developers, companies should look to retrain their existing team.

    That takes more than just telling people, “Here are the new ways we’re going to operate,” and then throwing them into it. There are a lot of tools and gateways available that can help people move gradually from the tools and languages they have been using. For example, in the last 12 months alone, numerous open source Kubernetes operators, service brokers and frameworks have been introduced that allow enterprise teams to leverage cloud-native technology from a context they already know, including WebLogic, database and Java. Look for combinations of people and projects that are likely to result in some early wins, and then have those people help train the rest of the organization.

    Make sure to take advantage, too, of the external collaborative environment that exists. Have veterans go to conferences or meet-ups and learn from the community—which, thanks to open source, is huge and growing.

    This will do more than give you a development team that’s built for today’s market, which demands that you deploy faster and react to failures in a more graceful way. It also will help you recruit and retain. Developers are twice as likely to recommend their workplace to if they are using DevOps principles—they are happier, easier to recruit and easier to retain.

    In this golden age for developers, the potentials are limitless—and every enterprise can participate. Making that cultural change, taking advantage of open source code and exploiting the power in the cloud—those are the keys to building the team and the applications that will spell success.

    — Bob Quillin

  • RDX Eases Oracle Database Migration to PostgreSQL on AWS

    RDX Eases Oracle Database Migration to PostgreSQL on AWS

    RDX this week launched a set of tools designed to make it easier to convert applications running on an Oracle database to Amazon Aurora, an instance of an open source PostgreSQL database running on the Amazon Web Service (AWS) cloud that is compatible with Oracle databases.

    Announced at the AWS Summit event in Washington D.C., the clckwrk Refactoring Service for Oracle promises to reduce the amount of manual effort associated with refactoring applications for the Amazon Aurora platform by automating as much as 60% of the database schema conversion process.

    James Ball, chief strategy officer for RDX, said in addition to reducing costs, many organizations want to shift applications to open source databases in the cloud to further their DevOps ambitions. Each time an application needs to scale, an organization will incur additional Oracle licensing fees. In contrast, an open source database enables DevOps teams to scale database instances as required automatically, he said.

    AWS has been making a concerted effort to convince customers to give up proprietary databases in favor of open source databases running on its cloud platform. That competitive pressure is a driving force behind new alliances that previously would have been considered unimaginable. Oracle, for example, recently partnered with Microsoft to promote the integration of the Oracle Cloud platform with the Microsoft Azure cloud. Oracle would prefer its database run on its cloud, but through the alliance is signaling its willingness to work with Microsoft to deploy Oracle database on Azure, a cloud platform that is the leading industry alternative to AWS. Microsoft, of course, has its proprietary database to adopt in the form of Microsoft SQL Server.

    Ball said RDX not only provides tools to accelerate the conversion to Amazon Aurora, but it also provides managed services for customers that want to outsource the management of Amazon Aurora to RDX.

    It takes most organizations several months to migrate a legacy application on to a public cloud. AWS has a vested interested in helping accelerate that process whenever possible. But many organizations that have embraced best DevOps practices find their efforts are hampered by reliance on proprietary databases. In fact, one of the reasons so many developers tend to favor open source databases is to avoid all the headaches associated with getting approvals for new database licenses.

    It’s unclear to what degree the rise of DevOps in the enterprise will ultimately shift database allegiances. Oracle is trying to maintain as much control as possible by making its databases available as a managed service that eliminates some of the purchasing overhead associated with spinning up new instances. However, the cost of an Oracle database license is still buried in the cost of the managed service.

    Not every organization is looking to transition away from a proprietary database, but there is now enough momentum around the “open source first” movement to make it worthwhile to build tools that automate the refactoring of legacy databases not just to reduce costs, but also to make IT a lot more agile.

    — Mike Vizard

  • Pivotal Software Embraces OpenJDK

    Pivotal Software Embraces OpenJDK

    Pivotal Software has officially thrown its weight behind OpenJDK, the open source version of the standard edition of Java, by adding support for OpenJDK to the Pivotal Spring Runtime on its Spring framework.

    At the same time, Pivotal announced it is extending its commercial support for open source Apache Tomcat servers in addition to providing commercial support for all the major open source Spring projects, including Spring Framework, Spring Boot, Spring Cloud and Spring Cloud Dataflow.

    Since 2011, Oracle, IBM and SAP have been making a case for employing OpenJDK. Ryan Morgan, vice president of engineering for the application platform at Pivotal, said that while other languages have become increasingly popular, the primary workhorse for building back-end applications in the enterprise remains Java. Now that Oracle has formally relinquished control over Java source code to The Eclipse Foundation, research and development into the legacy programming environment should accelerate. Oracle retains licensing rights to the name Java, but versions 8 and higher of Java Enterprise Edition are being converted to an open source platform known as Jakarta.

    Morgan said much of the work going forward would be focused on modernizing monolithic Java applications to make it either to turn them into microservices or at least make it easier for microservices developed in other languages to make calls via application programming interfaces (APIs) to those back-end applications. As enterprise IT organizations make the shift, they increasingly will rely on DevOps processes to manage the transition. The effort should accelerate the extent to which many organizations will need to rely on modern best DevOps practices.

    As both OpenJDK and Jakarta continue to move forward, Morgan said Pivotal is committed to continuing investment in the Spring framework, which has done much to make developing Java applications a much more agile endeavor. The most immediate benefit, of course, is the reduction of licensing fees associated with employing Java platforms. Many IT organizations that have mandated open source code-first and adopted alternative programming languages now may want to reconsider their options, especially when the number of developers who know how to write applications in more modern languages is comparatively small.

    Of course, it will take many years for enterprise IT organizations to fully make this transition. In the meantime, many of them will continue developing applications in Java simply because they have large teams that have mastered a programming construct that has been in place for three decades or more. Many of those same organizations may be making greater use of rival programming constructs, but Java and its descendants will play a major role in the enterprise well into the next decade. The primary challenge Pivotal will face is maintaining dominance of the frameworks employed to build new applications based on Jakarta and OpenJDK, in addition to updated existing Java applications.

    — Mike Vizard

  • Adobe Aims to Build Developer Ecosystem

    Adobe Aims to Build Developer Ecosystem

    Adobe today outlined an approach to building a developer ecosystem that revolves around open source code, application programming interfaces (APIs) and DevOps processes.

    Matt Asay, head of developer ecosystem for Adobe, said in the last few years Adobe has increased its reliance on open source code to deliver offerings such as Adobe Experience Manager, a content management system that incorporates open source code for as much as 65% of its modules. In the course of that effort, the company has made reciprocal code contributions to a variety of open source projects being developed under the auspices of the Apache Software Foundation (ASF), including Apache Sling, a web development framework; Apache Oak, a content repository; Apache Felix, an implementation of a services gateway; and Apache Lucene, a text search engine.

    Adobe also recently launched Adobe Experience Cloud, which relies on a variety of open source technologies to create a big data repository on top of which Adobe is encouraging developers to invoke APIs to build applications. Rather than build their own big data repositories, Adobe is making a case for relying on a multitenant big data platform it manages on behalf of customers.

    Based on GitHub data, Adobe currently ranks as the 14th largest corporate open source contributor, up from No. 32 in 2017. One of the primary reasons Adobe was able to move up that list so quickly was its move to acquire Magento, an e-commerce platform based heavily on open source code. But now that the company has come to appreciate what can be accomplished by relying on open source code, organizations should expect to see Adobe accelerating the rate at which it will be making contributions to a wide variety of open source projects.

    At the same time, Asay said, Adobe is making major investments in developing microservices that it plans to make it possible for developers to consume via its APIs. The Adobe Experience Cloud, for example, is a suite of software-as-a-service (SaaS) applications that employ a common data model deployed on a set of Kubernetes clusters running in the Microsoft Azure cloud.

    Asay noted Adobe is trying to make it clear that it has a lot of empathy toward developers committed to open source code. Adobe is a provider of commercial software-as-a-service (SaaS) applications, but it wants to develop a developer ecosystem that consumes the data residing in those applications via its APIs. To foster the development of those applications, Adobe has already signaled it is willing to share best DevOps practices with organizations that commit to invoking those APIs.

    Adobe is one of many providers of commercial software that are increasingly dependent on open source code. It will be interesting to see where the line between the desire to use open source projects to build applications and actual open source applications lies in the years ahead. But the one thing that is clear is that without DevOps processes and a well-defined set of APIs to guide them, there’s not much chance any vendor would be able to build a meaningful developer ecosystem of any size.

    — Mike Vizard

  • Open Source Versioning: The Race to Stay Up-to-Date

    Open Source Versioning: The Race to Stay Up-to-Date

    Open source libraries, once shunned as risky and not ready for prime time, are now used extensively across major corporations, including insurers. The reason is simple: In time- and resource-constrained companies trying to stay technologically competitive, it doesn’t make sense anymore to try to reinvent a wheel that’s already been battle-tested. However, having made the commitment to open source code and solution sets, it’s imperative to keep up-to-date with open source library maintenance and updates.

    The Risk of Open Source

    Some active open source libraries are updated multiple times a year. Getting out of date means missing out on newer features. Also, if the gap between upgrades bridges several iterations, some future upgrades may not be backward-compatible, causing functional problems at best and risk exposure problems at worst. On occasion, an out-of-date open source library version can expose a system and a company to attack. In 2017, a major U.S. financial company was hit by a hack that exposed the information of millions of customers. The hackers had exploited a software vulnerability that had been public for a month, but the company had not patched its system in time. There are several other well-known examples of a similar vein.

    Keeping current is not easy. Upgrading may be costly, but falling behind can make it costlier to catch up. The question is, How frequently should one upgrade to the newer versions?

    The Effort and Cost

    One of the reasons companies choose not to upgrade every time there’s a new release is the cost involved. While the open source code itself is free, the effort required in time and resources can be substantial and can take valuable resources away from higher priority business-technology initiatives. An additional concern is that, even though the upgrade is supposed to be backward-compatible, there is no way to verify that without actually performing the upgrade. If there are compatibility problems, then a larger effort ensues involving upgrading, restoring or rebuilding a system. Any large system has to be tested thoroughly after upgrading, and without the help of automated tools, testing can be costly and time-consuming.

    One approach for mitigating these costs and risks is the process of continuous integration (CI) for open source libraries. A good suite of automated testing tools provides a high degree of confidence that nothing has been broken by the most recent upgrade, before it is implemented into any production environment. However, without automated tests that flush out potential issues from upgrading, the risks and costs are high enough that companies have a hard time keeping up and managers become more motivated to delay upgrades than to perform them. That’s why it’s important that companies employ CI and continuous delivery (CD) processes that include robust automated testing.

    Additionally, the use of microservices—an increasingly effective and popular way to construct systems—puts another twist into the efforts and costs of keeping open source libraries up to date. Individually, each microservice might be quick to test. However, if an application has been broken into 50, 100 or even more microservice components, even a small amount of manual work testing each component will add up quickly. Additionally, in an environment that has deployed microservices broadly, there may not be a business need to change some services for months—that is one of the benefits of using microservices.

    The Right Approach to Open Source

    There is a right way to deal with open source library updates that has the benefit of taking advantage of new functionality and code stability in a low-risk, low-effort and low-resource manner as follows:

    Planning: Enterprise and application architects should be vigilant regarding the road maps of any large frameworks that employ open source libraries as a key component to any project or initiative. Proactive identification of open source libraries that may need to be upgraded is key. Likewise, application project managers should make open source library upgrade part of a regular and ongoing project plan. For example, when microservices are involved, they should be classified proactively as services by the libraries they use, as a way to identify the few services that would normally be upgraded first. This has the benefit of providing a few sprints’ worth of feedback before rolling out the upgrades to other libraries.

    Monitoring: Open source version monitoring is essential. There are products that can help to identify and report library versions that have vulnerabilities. Such monitoring can and should be used as part of the CI/CD pipeline and also to monitor artifacts that already are deployed in production. Monitoring processes also can report when newer open source library versions are available, even if the incumbent library has no reported vulnerabilities.

    Automated compatibility triage: It is often too risky to set up production builds to always use the latest version of an open source library. However, separate automated processes can identify new versions, run them through a build and test process and report back the results. If a test fails, the development team gets an early warning that their code is not compatible with the upgrade version. This helps to plan and schedule fixes, and if a vulnerability is discovered in the current version, this kind of proactive planning enables a quicker resolution and turnaround.

    Processes: Strong processes, preferably automated ones, are the foundation of open source versioning. For critical security upgrades, a process must be in place to triage and prioritize identified changes. For other version updates, good processes can help make upgrades easier. For example, a standard and repeatable process may incorporate minor version upgrades into the regular flow of sprint work, thus saving time while mitigating risk.

    Open source versioning can be difficult, but it doesn’t have to be. With automated testing, good monitoring tools and some proactive planning, the costs and risks of staying up to date can be kept to a minimum, allowing companies to reap the benefits of new features and functions.

    — Darius Cooper

  • Logz.io Unveils Azure Open Source ELK Monitoring Solution

    Logz.io Unveils Azure Open Source ELK Monitoring Solution

    Needless to say, DevOps as a practice is largely fueled by open source. The growth and popularity of open source tools such as Elasticsearch, Kubernetes and others are driven largely by the increased flexibility, innovation and familiarity of options. While this isn’t news for developers (65% of which are active open source contributors), leading cloud providers are taking notice, as they try to increase their appeal to DevOps engineers to finally win the cloud wars.

    Currently, AWS is the leading cloud provider. However, with Azure gaining more and more momentum, all three major cloud providers are scrambling to offer their customers the best open source options and empower them to use the tools they know and love in a manner that is convenient and native to their cloud environments.

    Going Big on Open Source

    Today, Logz.io and Microsoft Azure announced that our ELK-as-a-service will now be offered on Microsoft Azure. Logz.io offers an intelligent machine data analytics platform built on ELK and Grafana, two of the most popular open source monitoring and troubleshooting tools. Our platform enables engineers to easily deploy, run and scale ELK without the hassle and pain of maintaining and managing the stack themselves.

    Now, with Logz.io for Azure, users can use open source ELK as a highly integrated experience in Azure. The new offering includes seamless integrations to various Azure services, out-of-the-box Azure dashboards, built-in alerting and crowdsourced machine learning capabilities.

    The integration between Logz.io and Azure is not only significant in the context of the cloud wars, but in the culture of Microsoft itself. While the company was previously associated with a more proprietary structure, they have made tremendous strides in recent years toward empowering open source by opening its .NET framework and Azure SDK, as well as their acquisition of GitHub.

    According to Julia Liuson, corporate vice president of Microsoft’s developer division: “At Microsoft, we see open source as a major driver of innovation within the engineering community. Since we strive to empower our customers and facilitate their success, it is important to us to meet them where they are by making it easy to access and use open source tools. Our partnership with Logz.io will strengthen Microsoft Azure ties with the developer community, enabling teams to easily build and monitor their products using ELK.”

    AWS vs. Google Cloud

    Microsoft’s investment in ELK  comes just in time to provide an answer to its competitors, AWS and Google Cloud. Just two months ago, AWS announced Open Distro for Elasticsearch, which in their own words “is a value-added distribution of Elasticsearch that is 100% open source (Apache 2.0 license) and supported by AWS.” The new offering includes managed Elasticsearch and Kibana along with features such as built-in alerting, security and more. This was AWS’ most recent appeal to the engineering community, offering one of the most popular monitoring tools as a managed service on AWS.

    Following this release, Google Cloud announced its own open source offering featuring integrations with leading companies such as Elastic, MongoDB, Confluent and more. According to Google, the purpose of  these integrations was to “provide users with a seamless user experience and the ability to easily leverage these open-source technologies in Google’s cloud.”

    Is Open Source the Future?

    If what’s happening between cloud vendors is any indication of future trends in the engineering space, it seems imperative that all organizations targeting DevOps teams must consider how they’ll fuel open source and empower its vast user community. At Logz.io, we are proud to be part of this initiative.

    Overall, it is great to see the strides large organizations are taking in furthering the interests of the developer community and enabling them to work using the tools, methodologies and strategies that best suit their workflows and ideals.

    — Daniel Berman

  • With YAML Support, Puppet Dives Deeper Into DevOps

    With YAML Support, Puppet Dives Deeper Into DevOps

    Puppet this week announced an update to Puppet Enterprise that adds additional DevOps capabilities, including integration with Bolt, an open source agentless task runner that Puppet employs to make IT automation more accessible to IT administrators.

    Matt Waxman, vice president of products at Puppet, said adding integration with Bolt will make it easier to integrate DevOps processes that rely on YAML to deploy applications.

    Available this month via a Puppet Enterprise 2019.1 update, Bolt provides an agentless approach to automation that enables IT teams to execute existing commands and scripts written in any language within the context of an IT automation framework that is designed to be accessible to the average IT administrator, Waxman said.

    At the same time, Puppet is providing the ability to set up pipelines for deploying automation code directly from within Puppet Enterprise. Rather than having to set up a separate continuous integration/continuous deployment (CI/CD) platform, Puppet is embedding the capability it acquired from its purchase of Distelli in 2017. In doing so, Puppet is making it possible to accelerate deployment of automation software in a way that doesn’t  require every change to be approved by a master Puppet administrator. Default pipelines guide organizations new to continuous delivery practices through setting up their first pipeline in less than 30 minutes.

    Puppet is also including an Impact Analysis capability that enables teams to assess the impact of a proposed Puppet code change before it is deployed.

    One of the issues that has held back adoption of DevOps practices has been the simple fact that most IT administrators have limited programming skills. By making available an IT automation framework that provides the ability to manipulate YAML files at a higher level of abstraction, it should become easier of organizations to incorporate IT administrators into any set of DevOps process they care to define, Waxman said.

    Puppet via this upgrade is also extending the reach of the agentless Bolt framework to include support for network devices running Cisco NX-OS and Cisco IOS, as well as firewalls from Palo Alto Networks. As network operations continues to be more integrated with DevOps processes, IT organizations increasingly need to be able to apply workflows across compute, storage and networking platforms, Waxman said.

    The challenge many IT organizations now face in terms of achieving that goal is each provider of a platform creates their own automation framework, which winds up creating islands of automation across the enterprise. Longer term, Waxman said that, in addition to invoking the application programming interfaces (APIs) that many platforms already expose, Puppet will be working toward integrating domain-specific automation frameworks with its cross-platform framework.

    Overall, Waxman said Puppet remains committed to providing a mix of agentless and agent-based approaches to IT automation based on the nature of the IT tasks that need to be automated. It will be up to each IT organization to determine when to employ one of those options. But regardless of the path chosen, Puppet is clearly trying to make its core automation platform a lot more accessible to everyone in IT.

    — Mike Vizard

  • DevOps Chat: F5’s NGINX Acquisition, with F5’s Lori MacVittie

    DevOps Chat: F5’s NGINX Acquisition, with F5’s Lori MacVittie

    F5 Networks has acquired NGINX, the leading open source web server player. The move clearly puts F5 in the DevOps space, as well as the open source market.

    In this DevOps Chat, we speak with Lori MacVittie of F5 along with Sidney Rabsatt of NGINX. Some great insights into why this acquisition makes sense for both parties.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey everyone, it’s Alan Shimel, devopsy.kinsta.cloud, DevOps Chat, and you’re listening to another chat. Today’s chat features two folks from an acquisition announced yesterday. In case you haven’t heard, F5 Networks has acquired NGINX. We’re going to hear more about it from two folks. Number one is our friend, Lori MacVittie from F5 Networks. Lori, welcome. Glad to have you here.

    Lori MacVittie: Thanks, happy to be here.

    Shimel: Fantastic. Joining Lori and I is Sidney Rabsatt. Sidney is VP of product at NGINX. Correct, Sidney?

    Sidney Rabsatt: That’s right, and I’m happy to be here too.

    Shimel: Well, everyone’s happy to be here. That’s not a bad thing. Are you happy to be here at the DevOps Chat or are we happy to be here talking about F5’s acquisition of NGINX? Let’s hope it’s both. Lori, give us a little background here. When I saw the deal yesterday. I said, “Wow, that’s a big deal.” I’m not sure – it’s certainly an expansion, up the stack, if you will, for F5.

    MacVittie: Up the stack, deeper into the app. It’s all about application delivery. One of the – Sidney mentioned earlier when we were chatting, building on NGINX’s vision, which as I read it, is flawless application delivery. That’s certainly something that F5 has always aspired to achieve as well, but there’s just pieces in the application stack that we don’t play in, or haven’t played in until yesterday with the acquisition. NGINX is the leading webserver. They’ve got an app server. They do some app delivery. They’re very prevalent in many, many places. So we’re very excited to have them on board to see how we can bridge the gap between those two worlds.

    Shimel: Absolutely. What about the perspective of NGINX? Money is always nice. Don’t get me wrong. What’s the attraction to F5 here?

    Rabsatt: I think there’s at least two sides of it. Working with the folks on this early on, we got to get a sense of the culture of the organization. It felt very compatible. The people we met were great. We had a great time talking, hanging out, sharing ideas. There’s some real optimism about what can be done together. The thing that really gets folks excited, especially on my side of the house, on the NGINX side of the house is the opportunity to do even more faster. We’re a startup. It’s grown to a good size. The set of resources that we have, but looking at F5 and talking to those folks, understanding that they have a shared vision. They see the world very similarly and bringing more resources to bear to execute on the vision that we’re always working towards. It’s something that’s very exciting for us.

    Shimel: I can’t disagree with you. I think we’re in violent agreement on it. It is a great opportunity. I guess the next question is how do we integrate, Lori or Sidney, how do we integrate the NGINX webserver, appserver, and the rest of the toolset into the F5 family? Or do we not?

    Rabsatt: Do you mind if I take a stab at it?

    MacVittie: Go ahead, Sidney, please.

    Rabsatt: Sure. Look, I think on the surface of things it might seem as if there’s a ton of integration that needs to happen, and a lot of overlap needs to be addressed, but quite frankly, I’ve been pleased to see how much complementary capabilities we actually have. Oftentimes, in the marketplace, folks will actually deploy both solutions side by side. It’s because we do slightly different things, both super important to folks. When we think about integration, there’s opportunity for F5 customers to be able to leverage NGINX solutions in places where they couldn’t necessarily before. That should be a net benefit to them.

    There’s the ability for folks that leverage NGINX solutions to take advantage of things that F5 does, that we don’t particularly do. I think that’s going to be another win for them as well. The integration is really focused on making sure we have the right extensions to the product line identified so we can really give customers the things they need to succeed in delivering those end to end applications, and yes, of course, we’re going to be looking to bring the best of both sets of technologies together in the places where it makes sense, but we’ve been pleased to see it’s not as many overlapping things as we might have found in some situations. We’re very optimistic about the path forward here.

    Shimel: So crystallize for me, where do you see things that overlap versus things that play well together? Can you give us some specifics?

    Rabsatt: Sure. Things that play well together fall into the category of “Hey, can I bring better scrutiny, better analytics, to the types of solutions that I might already have with NGINX?” On the NGINX side, I’m sorry – on the F5 side, “Hey, can I leverage solutions to give me a little bit more freedom and flexibility to build my apps wherever or however I might choose to?” So I think there’s some goodness there that folks will definitely get access to.

    On the integration side, you’ll find most of it is just around making sure that the products work seamlessly together. So to the extent of I have multiple devices in my environment. How do I have a single solution that gives me the ability to manage it the way that I need to? We still need to go through the work of understanding exactly how the different buyers are going to need to interact with the different solutions that we use going forward, but all that work is in progress and again, the overlap tends to be the little stuff. Oh, do you draw a report? Oh yeah, we draw a report. Do you guys do that too? Sure. So how do we bring the best of those technologies together? It’s that kind of thing.

    Shimel: Got it. Lori, interested to hear, you heard Sidney here. What do you add to that? What are your feelings on it?

    MacVittie: I think he encompassed everything well. We already – you already see both of us in F5 and NGINX together in a lot of environments. From our perspective, I think we’re looking at – one, continuing to invest and amplify the NGINX vision and what they’re trying to execute on, but also where can we add security? Security is a big deal that has to extend from end to end. Right now, that’s a very disjoint process sometimes. That’s why we keep telling security to shift let. I think this is an opportunity that we’re going to be looking for to be able to do those kinds of things and shift that security even lefter with NGINX.

    Shimel: Absolutely. It’s funny when you think about security being a driving force behind this acquisition. With NGINX being a webserver, that’s been an area and app server, that’s been an area where security has been required and important forever, even going back to IIS in the day, Internet Information Server and stuff. Here’s an area though I was thinking about. So many people in the public cloud use NGINX as their webserver of choice, whether we’re talking about Amazon or even Microsoft Google or any of them. Lori, does F5 maybe view this acquisition as a way to expand your footprint in public cloud? If so, how?

    MacVittie:  I don’t think we looked at exactly how that’s going to play out, and what we’re going to do with that. The webserver is something different from the proxy technology that most of our products are built on. So there’s some differences there in how that works. We’re still looking at roadmaps and trying to figure that out. For right now, we’re just going to keep going ahead with both sets of road maps and moving forward. As we continue to explore what we can do together, we’ll figure out where it makes sense.

    Shimel: Sidney, same – flip side of that coin – of that question to you around public cloud. Do you see an angle where F5 helps there?

    Rabsatt: Yeah, I think F5 already had strong relationships with quite a few of the cloud vendors as do we, and we expect that – we ultimately want to give folks a choice. This all boils down to organizations are trying to build the applications that make sense for them. They want to be able to leverage the best tools and capabilities that help them deliver those applications while giving them the freedom to build the application they want. So whether that is on one cloud, on multicloud, across clouds, and on prem locations, they need that flexibility to do what makes sense for them. A big part of what we’re trying to accomplish here is making sure that F5’s large customer base has the capabilities that NGINX large user base has, which is again, that freedom of flexibility. That means that whether it’s on Amazon Google, Microsoft, any cloud, or any on prem solution, they get to deliver their applications with the security, controls, the visibility that they desire.

    Shimel: Excellent. I’m just – in hybrid situation, what about – one of the hottest topics certainly in technology today is the whole Kubernetes container thing. I know both F5 and NGINX had Kubernetes-related solutions and playing in that market. What, if any, does this acquisition, what if anything does this acquisition bring to that? Either one?

    Rabsatt: I would say it should be a net benefit. NGINX, there was a recent report I saw. It’s in the public domain. NGINX is the most commonly used ingress controller for Kubernetes environments today. If you think about what Kubernetes means for organizations that are building these modern applications, it’s giving them an environment that allows them to build typically microservices-based applications. So getting those applications exposed to the outside world requires the first thing, the foundational element, the hierarchy of need. It’s getting traffic in and out of that Kubernetes environment. How do you do that? Through the ingress capability.

    Our ingress controller is pretty much the number one solution out there for doing that today. That’s the first place people have to start before they get into some of the more complex or exotic things they’d like to be able to do. So our customers, our users are already benefitting from that. It’s allowing them to do what they need to do when it comes to adopting or testing Kubernetes for their own uses.

    Shimel: Sure. Lori, even independent of NGINX, what effect is this whole move to Kubernetes and container-ized environment having on F5’s business?

    MacVittie: Wow. On the business? Don’t really know. No. There is – we see a lot of customers asking. One, they want guidance on how to do this at scale and safely. A lot of our customers are very concerned about security, about performance. How do they bridge those two worlds, coming out of Kubernetes and trying to get into an environment where there’s a lot more traditional apps and infrastructure still supported? It’s trying to bridge those two worlds.

    Our customers are looking at it and they’re bringing it in and they’re adopting it, but they’re also having challenges with the deployment side where NetOps traditionally sit that they don’t have the skills to do the automation. They have problems with integration of the toolsets to build those pipelines. So one of the things that this kind of an acquisition does is bridge that gap. Hopefully we’ll see better integration and easier in and out, both actually in the flow of traffic, but also just between the teams in the businesses, so we think this will help address some of the pains at the people level in organizations by having this end to end delivery chain with both F5 and NGINX.

    Shimel: Sure. Sidney, maybe you would know. What are the plans with some sort of common interface or more integration between the two lines?

    Rabsatt: I’d say it’s still very early on. We have some good ideas, but we’re not really in a position to preview too much. I will say there’s going to be a focus on again, making sure we’re bringing the best of both solutions together. Again, not nearly as much overlap as you might have expected, or we – I should say as we might have expected. Yeah, what we’re looking at is really the opportunity to bring more features to the things we already have, more so than trying to get rid of things. We’re looking forward to getting this innovation stuff underway. We’ve had some very early conversations, but over the next days and weeks, we’re going to be digging into it very deeply.

    Shimel: Makes sense. Lori, so this was as we said right from the outset, what we’re turning to is a move up the application stack. What – is F5 done with that? Will we see more F5 potential either via the acquisition or holistically – organically moving up this stack here in the application world?

    MacVittie: I can’t obviously comment on any kind of [laughter] –

    Shimel: Can’t tell _____ acquisition is. I get that one.

    MacVittie: That’s right. With any company, that’s always a possibility. We have been transforming ourselves in the past year really moving forward with a lot more innovation in terms of exploring things and looking at – we’ve got Aspen mesh going, which is service mesh trying to bring some sanity to Kubernetes inside for scaling. Cloud services, we’ve made a lot of our services as a service. You just hook them up. We’ve been moving deeper into the application stack for a while. This is just going to accelerate that transformation from ADC to application services company.

    Shimel: Understood. Understood. Well, guys it really is exciting. One of the things I sit here at devopsy.kinsta.cloud and Security Boulevard and we go to all of the conferences and we cover all the news, and you hear so much “What’s bleeding edge, bleeding edge, bleeding edge?” We tend to forget, or we overlook what are the nuts and bolts, the bread and butter that keeps the internet and these applications going? It is things like the NGINX web server that is the dominant web server in the world today. It is things like the F5 lineup of software and of _____ actually that have been delivering the internet to the public since 2002 or whatever it is. Lori, you probably know, but so it’s good to see innovation continuing to happen. It’s good to see new combinations taking place.

    I’m excited to see what comes out of – Sidney, as you said, it’s very early on. It’s a day after the acquisition was announced. Maybe we could check back in six months and see how things are progressing along these lines. In the meantime though, just the usual questions. Everyone from NGINX is staying on? There’s not going to be layoffs as part of the consolidations and stuff, or again too early to tell?

    Rabsatt: I mean what we know is F5 is interested in our organization and not just products. They’ve made it very clear they want the vast majority of folks to stay and we expect that to continue to be the case. We look forward to what the future holds together.

    Shimel: Yep, understood. Look, I wish you a lot of success, both you guys individually, as well as F5 and NGINX. It’s an exciting combination. Congratulations to both organizations and Lori, I think congratulations on especially to F5 for really making a real great addition to the family there.

    MacVittie: Thank you. I know there’s a lot of us. We’re just really excited about this and about the future and what we can do together.

    Shimel: Cool, cool, cool. All right, congratulations, guys. We’re going to call it a rap on this edition of DevOps Chat. Lori MacVittie for F5 networks. Sidney Rabsatt for NGINX, soon to be F5 Networks. Thanks for being our guest today on DevOps Chat. Thank you everyone for listening. You’ve just listened to another chat. Bye bye.

     

    — Alan Shimel