Category: DevOps and Open Technologies

Editorial Calendar for February 2016

  • Building a Real-Time App: 13 Open Source Projects You Should Be Tracking Right Now

    Building a Real-Time App: 13 Open Source Projects You Should Be Tracking Right Now

    Open source is the dominant model of software consumption in the modern era. Cutting-edge startups and entrenched incumbents alike find open source software development and community building a significant part of their overall business strategies.

    For many open source projects, independent steering committees ensure that projects remain stable and are not beholden to a single company or profit motive. Alternatively, for open core projects where a single company financially supports and contributes the vast majority of open source code, open sourcing makes a project publicly available and accessible for potential new users. Opening a project to public contribution widens the pool of potential users and contributors, therefore ensuring more rapid innovation than commercial software offerings.

    Choosing the Right Open Source Projects for Your Real-Time App

    There are seemingly limitless open source projects to evaluate. Further complicating matters, each project has its own domain competence and cultural nuances. Considering a single distributed application will have tens or (many) more open source dependencies, development teams must continuously test new open source offerings and hone new skills to use these solutions, while simultaneously architecting their applications.

    In order to help sort through the noise, I’ve looked at six categories of open source projects for this article: data storage, message systems, service meshes, REST frameworks and streaming frameworks. For each category, I’ll identify any dominant players as well as some other projects of note.

    Here are the top 13 open source projects by category.

    Data Storage: Cassandra, Hazelcast

    Apache Cassandra, originally developed at Facebook, is one of the most widely used open source projects for real-time applications. Functionally, it’s a distributed NoSQL database management system. But practically, it’s become the linchpin of most at-scale, real-time applications. This is because Cassandra was designed to handle large amounts of real-time data across many servers, and has robust clustering support.

    For applications that require extremely low latency or have exceptionally high data rates, Hazelcast may provide a useful alternative. Unlike Cassandra, Hazelcast is an in-memory data grid database, which enables development teams to benefit from the latency advantages of in-memory computing. 

    Message Systems: Kafka, Pulsar, NATS

    Apache Kafka is the dominant open source messaging system used by application development teams today. Originating at LinkedIn, Kafka has grown into the de facto standard for high performance open source messaging systems. For streaming use cases, Kafka users can also turn to Kafka Streams.

    Apache Pulsar, developed at Yahoo!, is another alternative messaging system which has been optimized for streaming data sources. Pulsar further differentiates itself with TTL message support and queuing via shared subscription.

    For applications where latency is a critical consideration, NATS.io (a CNCF project) is one of the fastest pub-sub messaging systems available.

    Service Meshes: Istio, Linkerd

    Trends such as microservices, serverless and hybrid cloud deployment have contributed to the increasing architectural complexity of distributed applications. With hundreds or thousands of distributed application services, service meshes provide a mechanism for pulling all the disparate services together into a manageable and cohesive whole.

    The most notable open source service mesh offering is Istio. One of the most popular open source service mesh projects, Istio is primarily used for applications built on top of Kubernetes. Another alternative is Linkerd, a flexible service mesh solution for Kubernetes or other frameworks.

    REST Frameworks: Node.js, Spring

    Node.js is a popular JavaScript framework for building scalable web applications. When comparing web development frameworks, Node.js typically ranks among the top three most (along with frontend frameworks React.js and Angular.JS). While Node.js can support large scale applications, it does not support real-time use cases well at scale.

    Spring is another application framework, optimized for distributed microservices-style architectures. Like Node.js, Spring “focuses on the ‘plumbing’ of applications so that teams can focus on application-level business logic, without unnecessary ties to specific deployment environments.”

    Streaming Frameworks: swimOS

    SwimOS is a software framework for distributed streaming applications. Unlike REST-based frameworks, swimOS utilizes a unique and lightweight Web Agent architecture, optimized for composing and transforming many data streams in real-time. Whereas DevOps oriented projects, such as Spinnaker, seek to provide an abstraction layer to converge distributed cloud services, swimOS provides an abstraction across distributed data streams, so teams can build applications by composing many distributed data streams in a single mesh environment.

    In order to ensure real-time consistency of distributed services, swimOS automatically ensures eventual consistency between all Web Agents. Complete with a UI toolkit and it’s own stateful messaging protocol called WARP, swimOS is the fastest way to build stateful streaming applications.

    DevOps: Kubernetes, Spinnaker, RancherOS

    Kubernetes is one of the most talked about open source projects in the world. Kubernetes describes itself as “an open-source system for automating deployment, scaling and management of containerized applications.”

    While it may simply be a platform for container orchestration, rightly or wrongly, Kubernetes has become a stand-in for DevOps on the whole. In spite of its dominance, there are alternatives to Kubernetes, for example there’s RancherOS to manage Docker containers.

    Furthermore, as application architectures continue to become more distributed and complex there is a need for platforms which can converge multiple distributed edge and cloud environments. Spinnaker provides an abstraction layer to connect the various environments and provides a simple mechanism for continuous integration/continuous delivery (CI/CD). 

    Final Thoughts

    Your own particular use case should determine the appropriate open source technologies for your team’s real-time application. Whether you’re building an ultra-high performance low latency application or simply trying to tame large volumes of real-time data, knowing your options and choosing the right framework or platform can save you significant time and effort.

    The open source software landscape is changing every day. If you have something to add about one of the technologies above or think I skipped over the next OSS breakthrough, I’d love to hear about it. Let me know your thoughts in the comments section below.

    — Chris Sachs

  • DataStax to Support Multiple Distributions of Cassandra

    DataStax to Support Multiple Distributions of Cassandra

    DataStax today announced it will provide support services for any distribution of the open source Cassandra database via a subscription offering dubbed DataStax Luna.

    The company also announced it has made available two additional free tools in the form of DataStax Apache Kafka Connector for integrating Cassandra with the open source real-time streaming engine, and Bulk Loader to make loading and unloading data into Cassandra faster and simpler.

    Based on a wide column store architecture, Cassandra has emerged as a popular NoSQL alternative to traditional distributed databases because the column store on which Cassandra is based can also be organized into rows made up of multiple columns.

    Previously, DataStax only supported its own distribution of Cassandra alongside an enterprise variant through which it makes available additional features. There’s also a managed service, available in beta, through which DataStax will run Cassandra on a public cloud on behalf of customers.

    Created at Facebook, Cassandra is currently maintained under the auspices of the Apache Software Foundation (ASF). DataStax was subsequently created to provide a curated distribution of Cassandra alongside the distribution made available by ASF. Cassandra also makes use of Cassandra Query Language (CQL) as an alternative to SQL for processing queries. DataStax is now the largest contributor of code to the upstream Cassandra project.

    DataStax CTO Jonathan Ellis said that as the adoption of Cassandra has increased, there is now enough critical mass of multiple distributions of Cassandra to support. DataStax is now offering support via community and enterprise editions of DataStax Luna.

    Adoption of DataStax largely has been driven by developers. However, as the number of instances of Cassandra deployed increases, IT teams are finding they don’t have the expertise required to manage yet another database platform on their own. DataStax Luna provides a subscription service through which IT teams can rely on support from DataStax regardless of what distribution or version number of Cassandra they are running, said Ellis.

    In many cases, Cassandra databases are being managed by DevOps teams rather than traditional database administrators (DBAs). However, it’s not clear whether more DBAs will be expanding their roles to include managing alternatives to traditional relational databases. Many organizations are now coming to terms with the need to support multiple types of databases. In fact, in many cases, the management of databases on which modern applications are built has been incorporated into a larger DevOps team.

    Regardless of who manages the database, Ellis said the most important thing for any organization embracing Cassandra to understand its data model. As organizations look to process analytics alongside transactions in real-time, Cassandra affords IT organizations a lower-cost alternative.

    When it comes to databases, there is no shortage of alternatives. The days when organizations standardized on a single database are, for all intents and purposes, over. The challenge now is defining the best DevOps practices that will be required to manage multiple database engines as instances of each type of database continue to proliferate.

    — Mike Vizard

  • Oracle v. Google and Its Impact on APIs and Code

    Oracle v. Google and Its Impact on APIs and Code

    What, if any, impact will the Supreme Court’s rulings on Oracle v. Google have on the industry at large?

    TL/DR: It is unlikely to impact the vast majority of us, vendor and enterprise developer alike.

    Disclaimer: I am a technologist, not a lawyer. I have done some research, and talked to lawyers, but all that I say here is opinion—even my direct quotes of lawyers are their opinion. The Supreme Court can and has caught people off guard and taken cases in different directions than expected. Though I believe this path is the one it will follow, nothing is assured until the Court rules.

    Throughout this blog, references to The Case are actually using Harvard’s case document.

     

    Background

    On Nov. 15, the U.S. Supreme Court agreed to hear a dispute between Oracle and Google. The Oracle v. Google case has been going on for nearly a decade. And its roots go further back than that.

    The origins of the case go back to 2005, when Google and Sun Microsystems discussed the licensing of Java for Android. This will likely be important because it shows that Google knew it was someone else’s intellectual property (IP). More on this later.

    The crux of the case falls into three parts:

    1. Eight files that were decompiled and used. In one court case, these were found to never have made it into distribution and only used for testing (which limits liability).
    2. Two source files from Java used directly. These were found to be infringing, but damages were set to zero with consent of both legal teams.
    3. The part at issue: the copying of the entirety of Java’s interface from 37 modules. Most tech writers imply this as 37 APIs. It is not—it is hundreds of classes and thousands of methods. The 37 is the number of Java packages.

    The case has been through four court hearings/rulings so far. To sum up the relevant parts of these cases:

    1. The first case (split between a jury and district court, running at the same time) included patent violations. Those were defeated and Oracle has not appealed that defeat. On copyright infringement, the court found that Google did violate copyright on the 37 packages and the directly copied source code, but not on the eight decompiled files. The jury could not settle the issue of “fair use” of the 37 packages.
    2. The district court decided that the eight decompiled files and the copied source code for one function were violations, but remanded the rest back to the jury trial to settle the issue of fair use.
    3. In the end, the district court determined that APIs are not protected by copyright. Their decision involved two parts:
      1. That there is only one way to write a given API.
      2. The interface code and hierarchy can be replicated by third parties without violating copyright because it is an interface and not content.
    4. On appeal, the finding was reversed, and APIs were deemed copyrightable, sending the case back again.
    5. At this point, Google asked the U.S. Supreme Court to intervene. The court chose not to.
    6. Back at jury trial, the court found that while APIs may be copyrightable, Google’s use was fair use in the legal sense. This is the one topic the jury could not originally agree upon.
    7. Oracle appealed this decision, and the District Court ruled that it indeed was not fair use to copy the structure and exact interfaces.
      1. Where the jury and the district court appear to disagree is in the idea of APIs. Calling a publicly documented API is obviously fair use, and the jury latched onto that idea. The District Court appears to acknowledge the difference between calling an API and copying it.
    8. And after some tumultuous hearings and years in the court, the Supreme Court has agreed to weigh in on the issue.

    The real problem here is that copyright law is not a great fit for software. It never has been. If I rewrite your Java application in Python, using the same exact structure and naming, is that protected? All of the “words” have changed. It is akin to rewriting your Hindi book in English without a direct translation  … except that in software there is the added issue of it behaving exactly the same.

    But we can’t fix the misapplication of copyright law to software—only Congress can. So what to expect?

    A fair number of people, from the open source community and from businesses that see opportunity in a ruling that favors their world  view, are advocating that affording copyright protection to APIs is a horrible idea because it would hurt independent developers. This is not quite true. Independent developers are not generally copying Java. While this case does have bearing on coding practices and what developers can do, it does not have a huge impact on innovation, just on copying others’ work and how that is achieved—much like the “reverse engineering” court cases of the ’80s.

    There are good arguments on both sides of this case, but it seems to me that fair use will lose. They are public APIs in the sense that anyone can call them; indeed, some of these packages are so core to the system that you could argue every app must call them. But calling a function is not the same as re-creating it. In that sense, they are not in the public domain. Anyone can read a book, but that does not make it public domain.

    But no matter how this turns out, it is all good for the industry. This uncertainty has hung over the IT sector for nearly a decade. Just like when a new boss comes along and there is uncertainty in the department until everyone adjusts to the new demands/style of the boss, a solid ruling will allow settling to start. The industry can adapt once it knows what the rules are and we can move along.

    The real long-term answer is to have an equivalent to copyright and patent that is designed around the unique parts of computer code. Clearly, software is not the same beast as a book or magazine. The considerations are half-patent and half-copyright, and trying to bend those two to meet the needs of software has created some serious issues. Oracle quoted a 1995 case to showcase this problem—the company is quoting it to win the appeal, but it should be quoted as the reason the protections need to be designed for this medium that is not a publication and yet consists of words whose sequence matters: “Applying copyright law to computer programs is like assembling a jigsaw puzzle whose pieces do not quite fit.” (attributed by Oracle to the concurring opinion of J. Boudin in Lotus Dev. Corp. v. Borland Int’l, Inc., 1995).

    One interesting bit that the courts thus far have ignored, and I would love to see the Supreme Court take up, is this: If Google entered into licensing negotiations, the company knew it was Sun/Oracle’s IP. To then go and copy parts of it without licensing appears wrong on its face. It would be interesting to understand the process that ended in the copying and hear whether that bears upon the legal standing of the case. I can think of both malicious and innocent routes to get there; it would be interesting to hear testimony about motivations and thought processes.

    But even without delving into the licensing negotiations, this case stands to redefine copyright with regards to software. J. Michael Keyes, a partner at the international law firm Dorsey & Whitney, had this (and more) to say:

    “One of the issues that the court may decide is the scope of copyright protection for software. Google had early success below in convincing the trial judge that the snippets of copied code were not copyrightable. Google may urge to have the court take a look at this issue anew. Another huge implication is the scope of fair use. The jury found that Google’s use was “fair” and that it was not liable for infringement. The appellate court overturned that jury conclusion and found that Google’s use was not fair as a matter of law. The Supreme Court will likely wade into this thicket and provide guidance on the scope of fair use in the context of software code and what role an appellate court plays in reviewing jury decisions in that regard.”

    What we (all of us) do not want to do is confuse “good for the industry” with “legal standard.” It is not the court’s job to do what we think is best for the industry; it is the court’s job to determine applicable law and what that implies needs to happen. Presumably, the Supreme Court will do just that. And it may not please any of us. But if we think the current iteration of the law is flawed, that is something to take up with elected representatives, not ask a court to “fix” on the fly.

    For the vast majority of us, though, it impacts our development not at all. Using APIs or forking open source projects are all still perfectly acceptable, and no ruling by the Supreme Court is going to change that—because copying someone’s APIs is the issue here, not using them. And open source licenses allow for copying and improving projects. So carry on. Get the job done. And read about the Supreme Court’s final decision with a smile, because either way, the industry will adapt to the new normal and continue forward.

    — Don Macvittie

  • WhiteSource Acquires Renovate to Automate Dependency Updates

    WhiteSource Acquires Renovate to Automate Dependency Updates

    WhiteSource has acquired Renovate, a provider of an open source automated dependency update platform that the company plans to make available for free.

    Rhys Arkins, co-founder of Renovate who is now director of product for WhiteSource, said that as software projects become more complicated, there’s an increased need for a mechanism to identify and track dependencies and automatically update them as changes and updates are made.

    That requirement is becoming especially acute with the rise of microservices-based applications, which substantially increase the number of dependencies that exist within and between applications, added Arkins. Microservices make it possible for DevOps teams to build and deploy applications faster, but the dependencies between the microservices often result in an application environment that becomes too complex to manage. Most recently, Renovate added support for Helm Charts used to package applications in Kubernetes environments, on which many microservices are now being deployed.

    In addition, WhiteSource plans to add support for Cocoapods, a dependency manager for Swift and Objective-C Cocoa projects.

    Prior to the acquisition, WhiteSource was reselling the Renovate platform, which has more than 150 contributors. Now that WhiteSource owns the Renovate platform, including a hosted GitHub application and self-hosted GitHub and GitLab applications, the company has decided to make it available for free under the WhiteSource Renovate brand name.

    In the future, The WhiteSource Renovate app will add support for Bitbucket Cloud and Azure DevOps, thereby expanding the sources of dependency data that can be employed to make updates less risky and time-consuming, said Arkins. In effect, WhiteSource wants its automated dependency platform to become a natural extension of any continuous integration/continuous delivery (CI/CD) platform, he added.

    Arkins said that when it comes to managing dependencies, the biggest challenge is getting developers to identify them. In the absence of a platform for managing that process, developers will either not declare dependencies at all or, more commonly, not provide the right version number for a specific release of software on which their application is dependent. The more automated that process becomes, the less likely it is DevOps teams will find applications breaking because of a dependency that they were unaware of or lost track of at some point during the project.

    With the rise of DevOps, many organizations are now employing a more structured approach to dependencies. However, Arkins noted, many organizations are unaware that an open source platform is available that automatically updates all dependencies.

    WhiteSource is best known for helping organizations discover vulnerabilities in open source code. With the acquisition of Renovate, the company is furthering its ambitions to become a provider of software composition analysis tools.

    It remains to be seen just how many organizations will incorporate an automated dependency update platform into their DevOps processes. While many organizations have embraced DevOps, the level of maturity in terms of DevOps sophistication is uneven. The more software projects an organization launches, however, the more likely it becomes apparent there is a need for a better way to manage software dependencies.

    — Mike Vizard

  • CNCF Advances Jaeger Distributed Tracing Project

    CNCF Advances Jaeger Distributed Tracing Project

    The open source Jaeger distributed tracing platform has officially graduated into the top tier of projects being stewarded by the Cloud Native Computing Foundation (CNCF).

    Designed to be employed within any DevOps environment, Jaeger is a tool for monitoring transactions across complex distributed computing environments. It consists of a backend for trace collection and analysis and multiple client libraries implementing the OpenTracing application programming interface (API) in various languages.

    Developed by Uber, Jaeger now integrates with tools such as OpenTracing, OpenTelemetry, Envoy and Prometheus to make it easier to discover the root cause of issues impacting transactions in complex IT environments. For example, DevOps teams can apply data mining techniques and machine learning algorithms to analyze data from a specific cluster.

    JaegerYuri Shkuro, a software engineer for Uber who works on the Jaeger project, said organizations may rely on tools such as Prometheus to monitor environments, but once an issue is identified, it typically takes a tool such as Jaeger to identify the root causes of an issue. The primary reason for that approach is the amount of data that distributed tracing tools would store if they were to run continuously, he said.

    Organizations that are already using Jaeger in production environments include GrafanaLabs, Northwestern Mutual, SeatGeek, Symantec, Ticketmaster, Uber, Weaveworks and Zenly.

    Jaeger is also starting to be bundled with other platforms, such as the Red Hat OpenShift Service Mesh. As that trend continues, more organizations are likely to become exposed to Jaeger as a tool for debugging a wide variety of issues.

    Although originally developed to address the inherent complexity associated with microservices, Jaeger also can be applied to monolithic applications, Shkuro said. While many of those monolithic applications eventually are likely to be deconstructed into a set of microservices, many of those applications are not going away anytime soon, he noted.

    Jaeger now has 15 active maintainers from four companies, with more than 1,200 contributors and 375 authors of commits and pull requests. It also has more than 9,000 GitHub stars and has been pulled from Docker Hub more than 10 million times.

    While distributed tracing as a concept has been around for a while, the rise of microservices is forcing IT teams to look for tools that make it possible to engage in deeper forensics. The dependencies between microservices coupled with the ephemeral nature of containers often make it challenging for IT organizations to discover the root cause of intermittent issues. Within IT environments that have adopted Kubernetes to run those containers, it’s also become easier to deploy Jaeger using tools as Kubernetes templates, Operator software and a Helm Chart.

    As developers are held more accountable for applications, interest in programmable forensics tools is rising. Performance issues that developers once viewed as a problem to be solved by faceless IT operations staffs are now being solved much faster by developers. As a result, application experiences have improved now that developers are seeking out tools such as Jaeger to identify issues that are preventing them from focusing their efforts on writing additional code. Given the long, difficult history of IT, there are more than a few IT leaders wondering what took them so long to figure that out.

    — Mike Vizard

  • How Open Source Software Is Defining the Future of Mobile Connectivity

    How Open Source Software Is Defining the Future of Mobile Connectivity

    Mobile operators face intense pressure to deliver more data, faster connectivity, better coverage and more functionality to end users who are (unfortunately) more demanding than ever. Increasingly, these end users will not be people as we know them but rather Internet of Things (IoT) devices. IDC has projected that by 2025, 60% of the world’s data will be generated by enterprises, double its level in 2017, largely due to the growth in connected devices, sensors, automation and related equipment.

    To address industrial automation, connected cars, smart cities, sensor networks, asset management, connected health and more, operators need to rethink how the connecting network is architected. They must support faster connections, greater density and dramatically reduced latency. And they must add this functionality and flexibility while driving down the costs of deploying, sustaining and managing the network infrastructure.

    For decades, the telecom industry has been dominated by proprietary business and operating models, but with market pressures to find innovative ways forward, operators must find a new approach. That means embracing the open-source principles and innovations that took the computer industry at large from supercomputing to smartwatches and wearables and applying them to wireless network infrastructure.

    Open source software is proving to be key in 5G and IoT development, powering the automation of the mission-critical functions required to support the high speeds and low latency of 5G and the huge number of endpoints in IoT. Stakeholders have recognized that commoditization and democratization of wireless network infrastructure is necessary to stay relevant.

    Several initiatives already are underway to break this proprietary stranglehold and deliver software-defined networking (SDN) into the wireless network. They include the major operators and wireless infrastructure vendors, while disruptive challengers and startups are making an impact, too.

    The operator community along with major enterprises are now actively engaged in collaborative alliances to help drive uptake of open source.

    The O-RAN Alliance includes members such as AT&T, Deutsche Telekom, Intel, Verizon and SK Telecom, and embraces open interfaces and intelligent technologies for the radio access networks (RAN) needed for next-generation mobile applications like augmented reality, IoT, connected cars, smart homes and more.

    The Open vRAN initiative is backed by Cisco and is primarily focused on specifications to enable “unbundling the components of the network into smaller, simpler components and virtualizing capability to reduce cost and increase agility.”

    The Telecom Infrastructure Project’s OpenRAN Group, created in 2016 by Facebook, Intel, Nokia, Deutsche Telekom and SK Telecom, is focusing on more simplified and efficient radio access network technologies based on vendor-neutral hardware and software-defined approaches.

    A key component and enabler of a programmable base station is providing for a software-defined radio, which can revolutionize the traditional RAN landscape and make it easier for operators to roll out tailored services to different customer segments, at a reduced cost, by using software rather than hardware to process radio signals.

    To that end, the MyriadRF open source initiative was founded by Lime Micro in 2012 with the goal of democratizing wireless innovation. It has grown to include contributors ranging from hobbyists and wireless enthusiasts to professional engineers and equipment manufacturers. As of late 2018, more than 8,000 high performance LimeSDR software-defined radio boards had been shipped to developers, with more than 4,000 commits made across 70 GitHub repositories.

    LimeNet’s CrowdCell network-in-a-box platform, which significantly lowers the barrier to entry for software-defined radio development, is opening up a portfolio of new use cases for operators and enables them to go much further than being just a connectivity provider. Furthermore, it has the added benefit of access to a developer community which is constantly creating new applications and modifying and testing existing ones for increased use case fit.

    For example, Vodafone in the UK wanted to extend coverage and add additional services to the network it offers to customers and use this as a means to generate new revenue. By using CrowdCell, Vodafone is able to localize communication to a greater extent and reduce reliance on major cloud service providers–meaning Vodafone gets greater reliability and security at a lower cost.

    They also have the opportunity to develop cloud-hosting revenues and additional services, from which new revenue can enable 5G development and deployment. All this is underpinned by safe access to the app development community that ensures continuous innovation.

    The future of mobile connectivity is software-defined. The software-defined approach and third-party app development will ensure that operators are no longer differentiated solely by coverage or subscription costs but by the value-added services they are able to offer.

    — Tom Canning

  • DevOps Deeper Dive: DevOps Accelerates Open Source Innovation Pace

    DevOps Deeper Dive: DevOps Accelerates Open Source Innovation Pace

    In the span of a few short years, open source software has evolved from being a means to create the free equivalent of an existing commercial platform to now becoming the primary means of driving innovation across the IT industry.

    That rate of innovation has increased dramatically in the last few years. However, much of that innovation would not have been possible if large swaths of the open source community hadn’t been able to employ best DevOps practices to collaborate, said CloudBees CEO Sacha Labourey.

    Early on, the adoption of open source software was driven by a desire to commoditize operating systems and databases by making available a free equivalent.

    “It was all about replacement software,” said Labourey.

    Open source adoption got a major boost in the enterprise during the last economic downturn. Many organizations implemented an “open source first” strategy to lower their IT acquisition costs. As part of that mandate, developers began making greater use of cloud infrastructure to build applications based on an open source stack of software using best DevOps practices.

    In some cases, that preference for open source software turned into religious fervor. In other cases, it simply became a means to bypass IT purchasing processes that developers considered obstacles to writing code. Of course, it was only a matter of time before the open source software used to build applications made its way into production environments managed by IT operations teams.

    Just as significantly, open source software was leveraged to build public clouds that today are transforming the way IT is consumed.

    None of this shift has been lost on IT vendors. As the demand for proprietary code slackened, many found it profitable to offer support services for open source software. The more there is to consume, the more the support services contracts grew. Now every vendor from IBM to small IT services providers such as Fairwinds has launched open source projects that help drive demand for IT services expertise.

    “There’s pain around integrating a lot of disparate open source projects,” said Robert Brennan, director of open source software for Fairwinds. “Organizations may be getting software for free, but there’s usually not a lot of help around.”

    Now almost every IT vendor in the world is making software engineers available to work on open source projects. All that talent focused on open source projects has led to the development of new platforms such as Jenkins, GitHub, Kubernetes and, more recently, a raft of smaller projects. With the rise of containers and cloud-native applications, open source software projects are entering another era that will see many of those same software engineers leveraging DevOps practices more broadly to drive even more innovative projects at increasingly faster rates.

    “All the innovation is now being driven by open source,” said Joe Fernandes, vice president, products for cloud platforms at Red Hat. “You no longer have to wait for a single vendor to move something forward.”

    The challenge many organizations now face is keeping pace with the rate of innovation. Most major open source projects, thanks to DevOps practices, are updated once a week. To shield themselves from having to work with raw bits of code, many IT organizations rely on curated distributions of open source code from vendors such as Canonical, SUSE and Red Hat.

    Today, the economics of software development have changed. Providers of proprietary software are at a distinct disadvantage when competing against open source projects that can marshal more developer resources. A vendor selling proprietary software needs to keep a smaller team of developers highly focused on adding compelling, innovative features faster than an open source project can replicate them. As many vendors continue to show, that goal isn’t impossible to achieve; however, it is a pace of innovation that is exceedingly difficult to maintain.

    It remains to be seen just how much of the software deployed in IT environments will be based on open source code going forward. But with each passing day, the balance on the scale continues to lean further away from proprietary code.

    — Mike Vizard

  • Red Hat Streamlines Operating System Update Cycle

    Red Hat Streamlines Operating System Update Cycle

    Red Hat is looking to make it easier for DevOps teams to absorb leading-edge innovations as they are slipstreamed into a new distribution of CentOS.

    CentOS is a distribution of Linux based on a fork of Red Hat Enterprise Linux (RHEL). The team that oversees CentOS operates independently of Red Hat. That team in collaboration with Red Hat is making available an additional distribution dubbed CentOS Stream, through which a continuous stream of content will be updated several times daily.

    Mike McGrath, senior director for Linux engineering at Red Hat, said those innovations eventually will find their way into RHEL, but until then developers who want to build applications using those features as they become available can use CentOS Stream.

    This latest distribution of Linux from Red Hat is intended to act as a bridge between Fedora, a distribution of Linux through which Red Hat makes available experimental technologies, and RHEL, he said.

    McGrath said developers have been asking for a more stable distribution of Linux that provides access to innovations that are likely to be incorporated in RHEL, which so many IT organizations today deploy in production environments. CentOS Stream will make it easier for developers to take advantage of those new capabilities a lot sooner than they can today, he noted.

    CentOS Stream will be maintained in parallel to CentOS, which means organizations that have decided to standardize on CentOS are not going to be impacted by updates to CentOS Stream.

    As more organizations subscribe to software, version numbers associated with any software platform are becoming less relevant. Many DevOps teams especially want to be able to take advantage of new capabilities within the context of a continuous integration process when and if they see fit, rather than waiting for a formal update of a piece of software to be released.

    It will be interesting to see how many IT organizations embrace CentOS Stream at a time when many of them are under increased pressure to build innovative applications faster. In many instances, those innovations are dependent on features and capabilities embedded in the operating system, so waiting for those capabilities to show up in a major RHEL upgrade isn’t especially helpful. Many developers want to be able to exploit those capabilities the minute they are certified to run on RHEL. In many cases, developers are gaining access to those features only a few short months ahead of time whenever a preview edition of the next version of RHEL is made available.

    Eventually, every software platform provider will need to come up with some similar approach to CentOS Stream to provide developers with early access to new capabilities. In many cases, developers already have access to next-generation features if they have a close relationship with the platform provider. However, the process for making new capabilities available to developers remains haphazard at best.

    — Mike Vizard

  • Continuous Delivery Foundation Momentum Picks Up

    Continuous Delivery Foundation Momentum Picks Up

    The Continuous Delivery Foundation (CDF) this week revealed that it now includes 33 member organizations, including now JPM Chase, eBay, Dell Boomi, Cycloid, White Source, the Association of DevOps Professionals and the DevOps Institute.

    Founded by CapitalOne, CircleCI, Cloudbees, Google, Huawei, IBM, jFrog, Netflix and Salesforce in March the CDF is committed to promoting the adoption of open source continuous integration/continuous deployment (CI/CD) tools and platforms. Dan Lopez, CDF program manager, said the CDF plans to launch technical committees soon that address areas of CI/CD interest, including cybersecurity, software supply chains, software delivery management and machine learning algorithms in IT operations (MLOps).

    The CDF is not focused on promoting one open source CI/CD platform over another as much as it is fostering cross-engineering whenever feasible. As such, Lopez said the CDF is trying to advance adoption of the open source Jenkins and Jenkins X CI/CD platforms as well as the Spinnaker platform from Netflix. As other open source technologies are presented to the CDF, a technical oversight committee will evaluate them for inclusion, he said.

    At present, the membership of the CDF falls into three broad categories: companies that build tools and platforms, companies that leverage those technologies to deliver products and services and end user organizations such as JP Morgan Chase, eBay and CapitalOne.

    Lopez said the next goal is to expand the reach of the CDF beyond the U.S. and Europe to create a global CI/CD community. It’s not clear how many CI/CD platform providers will join the CDF. However, as membership in the CDF increases, pressure to join continues to mount.

    In the meantime, DevOps teams should expect collaboration between vendors to accelerate the pace of innovation across the entire DevOps life cycle. Vendors will be able to more easily identify areas to collaborate on that don’t provide much in the way of differentiation and yet are critical to advancing DevOps adoption.

    While a lot of progress has been made in terms of CI use, CD has proven to be an especially thorny challenge for most organizations. That’s become a significant challenge as organizations build applications at a faster rate and in many cases are still being deployed manually in production environments. Via the CDF, vendors are hoping to promote adoption of best practices across the entire DevOps toolchain as they gear up to introduce additional products and services that assume a strong CI/CD foundation is in place.

    It’s not clear to what degree the CDF will be able to accomplish that mission, given that most of the CI/CD issue organizations encounter are more cultural than technical. At the very least, however, organizations struggling with these issues should find solace among other like-minded souls wrestling with the same issues.

    — Mike Vizard

  • ‘DevOps in the Multicloud World’ Virtual Summit Puts IBM Z in the Spotlight

    ‘DevOps in the Multicloud World’ Virtual Summit Puts IBM Z in the Spotlight

    Got mainframes on the mind? Wonder how you can unlock the value of the platform for mission critical workloads? Trying to figure out where IBM Z fits in an increasingly cloud native IT environment? Those questions and others will be discussed in an upcoming virtual summit that explores the role of IBM Z —within the context of delivering higher efficiency and greater productivity for hybrid multicloud environments.

    “DevOps in the Multicloud World” virtual summit is scheduled to take place Oct. 24, starting at 10 a.m. Eastern. The day-long event features a passel of IBM experts, partners and customers sharing their knowledge and offering a deep dive into how to build and manage hybrid applications for agility in a multicloud world of continuous delivery. Panels and presentations will be held on a multitude of topics ranging from open source and DevOps to provisioning and user experience—all with a focus on the IBM Z systems.

    In addition, IBM customers will discuss how they’ve successfully included IBM Z in their multicloud strategies to improve their operations and bring real value to their end-user customers.

    Featured speakers include:

      • Barry Baker, Vice President, IBM Z Software
      • Rosalind Radcliffe, Distinguished Engineer, Chief Architect for CLM and DevOps, Member of the Academy of Technology, IBM
      • Mike Fulton, IBM Distinguished Engineer, CTO Enterprise DevOps, IBM Systems
      • Keith Wilson, Principal Technical Architect, USAA
      • Patrick Wawrzoszek, Senior Programmer Analyst, Scotiabank
      • James Gill, Mainframe DevOps Engineer, Triton Consulting (Lloyds Banking Group)
      • Arun Sharma, Product Manager Core Banking, DXC Technology
      • and more….

    In between sessions, attendees can roam the virtual conference hall, where they can chat and socialize with other attendees including IBM staff, and visit the virtual expo floor, featuring virtual solution zone stands equipped with video, downloads and even prizes for attendees.

    Perhaps best of all? Because this is a virtual event, you don’t have to leave the comfort of your own environs. Feel free to log in wearing your pajamas, even—we don’t judge!

    This event is free, but space is limited, so register now to reserve your spot today!

    — Charlene O’Hanlon