devopsy.kinsta.cloud

Tag: open source

  • Z is for Zowe – the Open Path to Mainframe DevOps

    Z is for Zowe – the Open Path to Mainframe DevOps

    The energy and excitement in the IBM Z mainframe community during the two years since the announcement of Zowe have not been seen in a generation.  By enabling mainframers to work like technologists who work on the cloud, mobile and distributed platforms, Zowe unlocks the power of DevOps for mainframers, ensuring the vitality of the platform for the long-term.

    It accomplishes this by opening up the world of enterprise DevOps tools, both familiar and special-purpose ones, for mainframe applications, allowing users to easily create toolchains that include the mainframe. This freedom to leverage any enterprise DevOps tools across the entire deployment pipeline is achieved simply and elegantly.

    The Zowe Backstory

    The Zowe journey began when Broadcom, at the time CA Technologies, along with IBM and Rocket Software realized the challenges they were hearing from customers—IT isolation, need for increased delivery velocity and recruitment and retention problems—were bigger than any single vendor could solve on its own. The challenges of this scale required an ecosystem-wide response.

    So the vendors came together with the Linux Foundation’s Open Mainframe Project to create the open source framework known as Zowe (as in David Bowie, for the uninitiated). The Zowe community operates transparently and welcomes contributions from all in the form of code, enhancement requests, educational content and shared experiences. A virtual meetup held earlier in the year attracted participants from 47 companies from across the mainframe ecosystem.

    Traditionally, mainframe application development and operations have been separate from the rest of the IT organization due to specialized processes, roles and tools designed around the platform. While they have served the mainframe community extraordinarily well over the decades, the world outside has changed with demand for ever-shorter software delivery cycles resulting in the DevOps model.

    In short, the tipping point has arrived and the time to normalize is now.

    What Is Zowe?

    Zowe is a modern interface to the mainframe that extends z/OS-based services and tooling to the greater IT landscape, including DevOps. As a framework, it consists of four main components:

    • Zowe API Mediation Layer (APIML)
    • Zowe Command Line Interface (CLI)
    • Zowe Explorer (Visual Studio Code extension)
    • Zowe App Framework (UI)

    Broadcom is proud to have initially contributed the first three of these components. The first two were created as part of our CA Brightside offering, which won an award for the Most Innovative DevOps solution—yes, a mainframe offering won a DevOps innovation award!

    These components provide a modern way of working that supports core DevOps practices such as empowering developers with shift-left testing responsibilities and DevOps engineers with robust automation options.

    This article describes the framework’s ability to onboard the mainframe to enterprise DevOps, so developers, systems programmers and others who work with the mainframe can now do so the same way their peers do with other IT platforms (i.e., cloud, mobile, distributed). These shared experiences close the gap between mainframers and others while preserving the core advantages of the platform. Common tools fuel a common language that benefits all, especially when deploying hybrid applications (e.g., web front-end with mainframe back-end).

    Through automation and script libraries, Zowe helps reduce the range of mainframe-specific knowledge necessary to be productive, enabling organizations to more easily recruit, onboard and retain the next generation of talent. By using modern DevOps tools and techniques combined with access to “big iron” applications, the next-generation now sees mainframe as an exciting career choice.

    Recent milestones achieved by the Zowe community include the launch of the long-term support (LTS) release ensuring the stability and longevity of the framework and a Zowe Conformance program ensuring the interoperability of a growing ecosystem of conformant tools.

    The framework drives mainframe DevOps in several ways:

    Zowe API Mediation Layer – Discover and Securely Consume

    As RESTful APIs become table stakes, the greater the challenge becomes, making those services easily discoverable and consumable in a secure way. The Zowe API Mediation Layer (APIML) addresses this need by exposing mainframe services, ranging from z/OS system level to database/transaction services to tool-based services, through a centralized gateway and catalog.

    Conformance of these APIs with the Zowe LTS standard eliminates breaking changes while bringing additional value, including the usability and productivity benefits of single sign-on (SSO).

    At Broadcom, we’ve leveraged Zowe to achieve conformance across our product portfolio and have a healthy pipeline of Zowe-conformant products soon to be announced. By spanning the traditional Dev and Ops domains, we’re facilitating true mainframe DevOps adoption—teams can combine their preferred mainframe-native tools with off-platform ones, including ones used by their peers working on other platforms, to optimize speed, quality and productivity.

    Zowe CLI – Orchestrate and Automate

    For DevOps Engineers and DevOps-savvy self-service developers, CLIs are a “go-to” tool. They use CLIs for a wide range of platforms including AWS and Azure, Git, Kubernetes and so on … why not mainframe? The use of CLIs paves the way for robust, script-based automation, which is a cornerstone of DevOps. The Zowe community already offers some basic scripts, free for anyone to use.

    Using a CLI may be new for some, so community members often share tips and tricks to help these users navigate the productivity curve quickly.

    Broadcom currently offers a dozen or so CLI extensions to mainframe-native tools, once again spanning the traditional Dev and Ops domains to improve productivity across roles and processes.

    Zowe Explorer for VS Code – Visualize and Navigate

    Visual Studio Code is exciting for any mainframers editing code, especially developers and sysprogs. While ISPF and Eclipse will remain viable options for the foreseeable future, the extraordinary success of the lightweight VS Code, which is backed by Microsoft, and the rapid emergence of browser-based editors/IDEs expand the range of options. With 11 million users around the globe, VS Code provides an experience users love and a marketplace of 20,000 extensions enabling users to personalize their UX and optimize their workflows.

    Zowe Explorer, which streamlines interaction with mainframe data sets, USS files and jobs, is only one of the many VS Code extensions available in the VS Code marketplace for mainframe users (and a successful one with more than 12,000 downloads). Unlike Eclipse extensions, all of the mainframe extensions from Broadcom are free, and many are open-sourced.

    In the Queue: SSO/MFA, SDK, Mobile App

    The Zowe community has been working diligently on the implementation of SSO and multi-factor authentication (MFA), which eliminates the drudgery of constantly logging on to multiple systems.

    Also, software development kits (SDKs) are in the works, the goal of which is to make Zowe services (and by extension, the mainframe) easier to use by providing reusable code for common mainframe operations in popular languages and runtime libraries. The beginnings of a Zowe Node.js SDK and Zowe Python SDK already exist.

    Finally, a Zowe mobile app to help systems and operations support staff improve system availability is, like the SDK, being incubated by the community.

    Decision Time – Zowe Open Path or Closed, Proprietary

    Mainframe teams face a choice: Continue with a vendor-controlled model, in which vendors determine which tools you use and how you work or adopt the open path. While the prescriptive (aka opinionated) approach of the former may have been acceptable for a siloed mainframe, the open approach is the foundation of enterprise DevOps.

    As an indication of the early success of the open path, we’ve been delighted to learn about many mainframe shops adopting Zowe on their own, with CI/CD integration and code analysis a common entry point.

    For those with reservations about open source, consider the prevalence of Git, Visual Studio Code, Jenkins, SonarQube and so on in the DevOps landscape. Microsoft CEO Nadella recently stated, “We’re all-in on open source” and Zowe co-founder IBM recently acquired Red Hat—more shining examples that open source tools are a trusted part of the technology stack.

    Change is never easy, which is why no-fee Design Thinking workshops are available to help adopters prioritize the DevOps use cases enabled by Zowe. Because so many see “open” as the least costly, least risky and fastest path to mainframe modernization, familiarity with Zowe is quickly becoming an industry standard. Ask your vendors if they’re joining the Zowe community and ecosystem.

    With ongoing commitment from IBM, Rocket and Broadcom and the guidance and support of the Open Mainframe Project, the excitement around will Zowe continue so join the open mainframe movement!

    To learn more about Zowe, visit Zowe.org and the community blog site.

  • Gitpod Open Sources Automated Local Development Environment

    Gitpod Open Sources Automated Local Development Environment

    Gitpod announced today it is making available an instance of its Gitpod integrated development environment (IDE) available under an open source license.

    Company CEO Sven Efftinge said DevOps teams will now have the option of deploying Gitpod themselves as an alternative to the existing cloud service provided by the company.

    Designed to be used in conjunction with repositories such as GitLab, GitHub Enterprise and Bitbucket, Gitpod automates the building of development environments on-premises, Efftinge said. It continuously monitors updates to any Git repository.

    Gitpod will then configure the right tools for a specific git branch, automatically compile machine-executable code and download all the dependencies required to allow developers to build code on-premises. Rather than having to manually configure a desktop environment, developers can start coding in seconds, Efftinge said.

    Gitpod borrows concepts from continuous integration platforms and applies them to development environments, he said, noting developers today typically spend multiple hours per week setting up and maintaining local developer environments.

    By automating that process, he said, the local development environment can also better align with production environments. One of the reasons why code doesn’t work in a production environment when it is first deployed is because the local development environment has been manually configured in a way that doesn’t accurately reflect the production environment, Efftinge noted.

    That issue becomes even more problematic when developers are working on multiple projects involving, for example, separate microservices that are all going to run in different production environments, he added.

    Gitpod also prevents drift by making sure all versions of a local desktop environment among developer teams remain consistent. Collaboration also becomes easier when the members of the teams have standardized on the same instance of a local desktop environment.

    Efftinge said there are already hundreds of thousands of developers using the Gitpod cloud service. The open source edition provides an option that will enable DevOps teams to self-host their own instance, he noted.

    Gitpod, formerly TypeFox, collaborated with Google, Ericsson, Arm and Red Hat to develop the open source Eclipse Theia project, which provides a foundation for building IDEs in the cloud using TypeScript, CSS and HTML. Theia, now being advanced under the auspices of the Eclipse Foundation, consists of a front-end running in a browser or in the local desktop application and a back-end running on any host. Both the front-end and back-end communicate through JSON RPC over WebSockets. Gitpod leveraged that project to build its own cloud-based IDE.

    In the wake of the COVID-19 pandemic, there’s naturally a lot more focus on automation and developer productivity. Most developers will be working from home for the foreseeable future to help combat the spread of the virus. Of course, most developers have been writing code outside the office for years. The challenge now is enabling teams of developers to develop applications as efficiently as possible regardless of where they are located. Eliminating inefficient manual tasks such as maintaining local development environments could be a significant step in that direction.

    It’s not clear yet whether DevOps teams are more or less productive since the onslaught of the COVID-19 pandemic. However, it is apparent the need for a consistent experience among teams of highly distributed developers is acute.

  • Why We Need a Software Bill of Materials Industry Standard

    Why We Need a Software Bill of Materials Industry Standard

    The SBOM concept is part of is an industry-led, multi-stakeholder process to improve software component transparency

    Have you ever gotten a recall notice for the vehicle you drive? Perhaps your car or truck was part of the big recall on airbags a few years ago. It’s standard procedure for auto manufacturers to notify owners of their vehicles when a component is defective and needs to be repaired or replaced. This is true even if the component was made by a third party because it’s still part of the overall makeup of the vehicle.

    Such recalls are possible because manufacturers keep a bill of materials to know every single original thing, including physical parts and internal software, that goes into each vehicle. Title registration records kept by government agencies keep track of current owners of the vehicles. When a recall is necessary, it’s possible to identify which vehicles are affected and who owns them.

    Now the computer software industry is undertaking a similar effort to track individual components such as libraries, executables or source code that go into a software program. Initiated by the National Telecommunication and Information Administration (NTIA), the effort is an industry-led, multi-stakeholder process to improve software component transparency. The NTIA working group is establishing a shared vision of transparency and a “software bill of materials,” or SBOM.

    What Is a Software Bill of Materials?

    An SBOM is effectively a list of ingredients that makes up a software program or application. Like the auto industry, where components come from hundreds or thousands of suppliers, modern software is stitched together from bits of code and subroutines from numerous sources, including open source and commercial solutions. Reusing software code like this means developers don’t have to create everything from scratch themselves. It’s a great time-saver and a very efficient way to develop applications.

    Fig. 1: Software component relationships

     

     

    Source: www.ntia.gov

    A software bill of materials identifies and lists all software components, information about those components and the relationship between them. Figure 1 below illustrates this concept. The chart on the left shows the information that is tracked, while the diagram on the right illustrates the relationships of the software components.

    As the NTIA working groups go through their development phases, they are mindful of the inherent complexity of building and maintaining a verifiable software components list. Issues they are currently working on include how to automatically generate an SBOM and keep it up to date; what standard data format to use for the information; how to distribute an SBOM to someone who acquires a software program or application; and how to verify the authenticity of the data in the components list and their inherited dependencies. The intent is to define specifications that can be used worldwide by both developers and consumers of software. The adoption of SBOM by individuals and companies writing open source software will be key to the initiative’s success.

    The Need for a Software Bill of Materials

    There are several benefits to establishing greater transparency of what code components are in software. Most importantly, it allows companies to control risk by enabling earlier identification and mitigation of vulnerable systems or license infringing source code. Further, it incentivizes secure software development practices by enabling developers to better vet the code they are embedding into their own work. For software consumers, greater transparency supports informed purchasing decisions, especially where compliance with software licensing requirements is concerned.

    Perhaps the greatest value of having a verified SBOM is in the realm of cybersecurity and risk mitigation.

    According to industry studies, the average codebase contains hundreds of open source software components, and the majority of those codebases contain at least one vulnerability. Thus, having insight into code components and their vulnerabilities is critical.

    Once you know precisely what components are in your software, you can get a clear vision of the risk factor this specific bill of materials brings to your environment when it runs. What’s more, the risk factor can change whether or not something in the software bill of materials changes since new vulnerabilities are discovered on a daily basis. The only way to know if you are affected is by having this level of transparency.

    This means of risk assessment is especially relevant when changes are attributed to insider or outsider threats. An insider threat occurs when an internal employee changes something without understanding the full impact of that change on the overall risk; for example, installing a foreign application to SBOM considered to be shadow IT. An outsider threat occurs when an attacker injects malware into your server or a device and tries to make it look like a legitimate component of your system by replacing or introducing a new software component. Either way, expecting to have one component and actually having another introduces a lot of risk to your system.

    Cybersecurity companies will be able to develop solutions that can consume the SBOM in a standard way and cross-reference it with everything running on the system, including the reference to known or new vulnerabilities in the listed software components, and augmented by threat intelligence. Recommendations for mitigations such as software patching can then be put forward. This is commonly called a healing process. In some cases, auto-remediations steps can be taken, such as blocking the malicious software from running until it can be removed.

    Governance over software licenses can be improved through SBOM adoption. Every bit of software comes with a license that dictates the legal use and distribution of the software. The snippets of code in a supply chain that comprises a complete application can have many different licenses for the individual components. Any company that deploys that application has a legal obligation to comply with the licenses. Without a software bill of materials, there may be no way to know what the licenses require, or how to comply with them. 

    Beyond Software: Devices, Too

    There’s another concept that goes beyond just software known as a digital bill of materials, or DBOM. A DBOM extends to the hardware configuration and its embedded firmware and includes the interaction between the components, i.e., the behavioral context. A DBOM would collect the same sort of knowledge about hardware devices and network communication patterns to understand the context of the software materials running on the device. This would apply mostly to IoT devices that otherwise are challenging to monitor, update and protect.

    When you buy a device, you are essentially buying a black box because you don’t know exactly what’s going on inside. If you install it on your network, you don’t know what risks come with this black box. If you knew the risks, you could put mitigations in place. Thus, it’s important to motivate the device manufacturers to get behind the adoption of both SBOM and DBOM specifications and help them recognize how it can improve their operations and security posture. There is value in complying with security certification requirements such as ETSI, allowing their end users to understand the risks they are assuming, and empowering them with the ability to mitigate these risks.

    The medical technology industry has been at the forefront of creating a proof of concept initiative in 2019 to understand the role that SBOMs can play in managing the operational and cyber risks associated with medical devices. Working together, device manufacturers and healthcare providers demonstrated the feasibility of SBOMs by generating, sharing, and using data to improve security practices in pre-defined use cases. NTIA is applying the lessons from that PoC to develop industry-agnostic specifications for the applications of software bills of materials.

  • CNCF Provides Jaeger Distributed Tracing Update

    CNCF Provides Jaeger Distributed Tracing Update

    The Cloud Native Computing Foundation (CNCF) has made available a report detailing the progress being made around the open source Jaeger distributed tracing software. Among other things, the report notes there are now more than 300 organizations contributing to the project.

    Jaeger is designed to provide a framework for monitoring microservices-based applications in a way that captures every interaction as part of an effort to simplify troubleshooting.

    Originally developed by Uber, Jaeger has almost 2,000 contributors. Developers from Uber and Red Hat make the bulk of those contributions., while other developers work at Ryanair, IBM, Ticketmaster and CloudBees.

    Thus far, there have been nearly 40,000 contributions, 4,300 code commits and 3,200 pull requests for Jaeger.

    Yuri Shkuro, a software engineer at Uber who created and maintains the Jaeger project, said as open source agent software in the form of OpenTelemetry, a separate sandbox-level initiative also hosted by the CNCF, gains traction, the more feasible it will become for DevOps teams to embrace distributed tracing. Currently, most DevOps teams are dependent on sampling techniques typically employed via an application performance management (APM) platform. While that approach can indicate there might be an issue, distributed tracing makes it easier to pinpoint the precise problem.

    That’s critical because many DevOps teams can spend hours, if not days, trying to identify the source of an issue that often only takes a few minutes to fix. The trade-off is that distributed tracing collects a lot more data than sampling, so IT teams will need to decide how long they may want to store that data. Most organizations will only store tracing data that identifies anomalies based on a reasonable retention schedule rather than allocating storage resources to what could become massive amounts of tracing data, noted Shkuro.

    Providers of commercial observability platforms have been embracing OpenTelemetry to provide distributed tracing services, so it’s not clear to what degree IT organizations will opt to deploy Jaeger on their own or access a managed Jaeger-based service.

    Regardless of how they are invoked, next-generation observability platforms based on Jaeger should become more affordable at a time when microservices-based applications are being more widely deployed. Given the inherent complexity of those applications, observability platforms that are a core tenet of any best DevOps practice become a requirement. In comparison, the use of ALM platforms was typically limited to only the most mission-critical applications because of the costs associated with deploying proprietary tools.

    It may be a while before every application is instrumented by default. In the meantime, Shkuro said the Jaeger project is looking to recruit additional maintainers. As is often the case with any open source project, there are plenty of contributors. IT professionals willing to volunteer to maintain a project are often more scarce.

  • CDF Officially Graduates Jenkins CI/CD Platform

    CDF Officially Graduates Jenkins CI/CD Platform

    The Continuous Delivery Foundation (CDF) announced today that the open source Jenkins continuous integration/continuous delivery (CI/CD) has officially graduated.

    Jenkins was donated last year to the CDF, an arm of The Linux Foundation, along with Jenkins X, a CI/CD platform for Kubernetes; Spinnaker, a CD platform; and Tekton, a framework for building pipelines that will be employed as the foundation for next-generation CI/CD platforms.

    Tracy Miranda, CDF Governing Board Chair and director of open source community at CloudBees, said that although Jenkins traces its lineage back to 2004, the CDF wanted to make sure the same requirements that are being applied to graduate other projects are applied to Jenkins despite its obvious maturity. The CDF requirements for graduating a project are to demonstrate growing adoption, have an open governance process, feature maturity and a strong commitment to community, sustainability and inclusivity.

    The CDF also today published a public road map for Jenkins. Key features include improving the user/administrator experience and tighter integration with cloud platforms. Work has also begun on Operator tools for simplifying the deployment of Jenkins on a Kubernetes cluster.

    Longer-term, the CDF is focused on expanding the adoption of Tekton-based Trigger pipelines across both Jenkins and Spinnaker. Tekton pipelines are easier to create and can be reused across multiple platforms. Jenkins X already employs Tekton-based pipelines, which were originally developed by Google.

    The founding members of the CDF are CloudBees, Google, CapitalOne, CircleCI. Fujitsu, Huawei, IBM, JFrog, Netflix and Salesforce.

    In the nearer term, Miranda said the CDF is also attempting to standardize the glossary of terms applied to DevOps processes. The goal is to provide something akin to a “Rosetta Stone” that will help make DevOps more accessible to IT professionals just starting their journey.

    In theory, at least, platforms such as Jenkins X accelerate the development of microservices-based applications running on Kubernetes clusters. In practice, however, many IT teams continue to use Jenkins and other CI/CD platforms to build both monolithic and microservices-based applications.

    As Kubernetes gains momentum, however, the way applications are deployed may also evolve. Today most DevOps teams push application code from a CI/CD platform on to a target system. However, if Kubernetes is deployed on multiple platforms it may become easier to automate the delivery of code by enabling Kubernetes clusters to pull code from CI/CD platforms on demand.

    As organizations transition to building and deploying microservices-based applications, it’s also apparent there will be a much greater need to rely on best DevOps practices to manage all the dependencies that will exist between each microservice. Legacy application development processes will not be able to keep pace with the rate at which microservices based on containers are ripped and replaced.

    Of course, it’s not clear to what degree organizations may prefer to stay with Jenkins, which is built on a Java virtual machine, versus adopting Jenkins X or a CD platform such as Spinnaker. However, as the interoperability between these platforms continues to improve, it may not be surprising to one day find DevOps teams employing a mix of all of them across different application development projects.

  • Red Hat Updates Virtualization Platform

    Red Hat Updates Virtualization Platform

    Red Hat today announced an update to its virtual machine platform that now can run on the latest version of Red Hat Enterprise Linux (RHEL).

    Version 4.4 of Red Hat Virtualization can now run on version 8.2 of RHEL. In addition, Red Hat has added Grafana-based dashboards for its Data Warehouse (DWH) service that IT teams can employ to track performance and analyze workloads.

    Finally, Red Hat Virtualization 4.4 simplifies configuring network services using its NetworkManager console.

    Peter Lauterbach, a product manager in the Cloud Platforms group at Red Hat, said support for RHEL 8 is core to the company’s overall strategy to enable IT teams to run microservices-based applications simultaneously with existing monolithic applications.

    Red Hat envisions IT organizations will deploy RHEL on bare-metal servers running Red Hat OpenShift, an application development and deployment platform based on Kubernetes that can run applications built using containers. In addition, Red Hat OpenShift supports kubevirt, open source software that enables kernel-based virtual machines (KVMs) such as Red Hat Virtualization to run on top of a Kubernetes cluster. Made available as Red Hat OpenShift Virtualization, that software makes it possible to run legacy monolithic applications alongside microservices-based applications on the same Kubernetes cluster as needed, said Lauterbach.

    That approach not only is easier to manage from an infrastructure perspective, Lauterbach noted it also can reduce latency between monolithic applications and microservices. Many IT teams in the wake of the COVID-19 pandemic have accelerated the rate at which they are moving applications into the cloud by encapsulating entire monolithic applications into containers that then can be ported to a cloud platform without the need to refactor them. They then can begin to modernize those applications by breaking them up into smaller microservices.

    Lauterbach also noted it has become a lot easier to leverage Kubernetes to move applications between platforms because interoperability between storage and networking services has improved significantly, thanks to the Container Storage Interface (CSI) and Container Network Interface (CNI) that have been made available by the Kubernetes community. In fact, Lauterbach said standards in infrastructure are living up to their interoperability promise.

    Red Hat, of course, has been making the case for open source virtual machines for decades now. However, in the wake of the economic downturn brought on by the COVID-19 pandemic, there will be many more organizations looking to reduce costs by relying on commercial virtual machine software. In addition, Lauterbach noted that now that cloud service providers such as Amazon Web Services (AWS) support KVM, it is a lot easier to create a hybrid cloud computing environment by deploying Red Hat Virtualization in a local data center than it is to replicate a VMware environment in a public cloud.

    It’s too early to say how the battle for hybrid cloud computing will play out in the months and years ahead. The one thing that is certain is the more diverse the IT infrastructure environment is, the more difficult it is to manage.

  • Cassandra Update Boosts Recovery Speeds

    Cassandra Update Boosts Recovery Speeds

    Apache Cassandra is a row-oriented NoSQL database created by developers working at Facebook. With elasticity as one of its key features, open source contributors have added new streaming capabilities to Cassandra 4.0 for its beta launch. These improvements have been shown to enable the database to stream data between nodes at speeds up to five times faster than in the past.

    What this means is that if a node on a network crashes, for example, the data can be transferred that much more quickly to another replacement node. Conversely, during peak load data times, different nodes on the cloud or on-premises servers can more readily and quickly manage potential data overflows for improved operations elasticity, Cassandra’s contributors say.

    Elasticity—or lack of—was often seen as a weak point in previous Cassandra releases. Indeed, elasticity improvements, geared for DevOps taking advantage of distributed microservices environments are “really a key point,” said Josh McKenzie, a Cassandra committer and member of the project management committee. McKenzie is also the lead of open source strategy and vice president of software engineering for DataStax, which has historically funded the bulk of Cassandra’s contributions and support.

    The new streaming capability also could potentially serve as a welcome feature for database management, especially in highly distributed stateless environments of Kubernetes and microservices, said Clive Longbottom, an analyst for Longbottom Associates.

    “On the first one, this seems like an excellent update. It makes the ‘eventual consistency’ of Cassandra less of an issue, and the more guaranteed timescales of recovery across a range of scenarios is welcome,” Longbottom said. “I would hope that such a major change will get more to consider Cassandra for such data work—it can be, when implemented correctly and the right jobs put on it, an effective data platform.”

    The new streaming speed is made possible because the data is able to bypass the use of the servers’ CPUs, McKenzie said.  “Being able to get anywhere from four to five times the throughput when bootstrapping a new node on recovering from failure or scaling clusters up and down means the process is now a totally different ballgame.”

    An example of a six-node cluster setup.

    For DevOps teams, this means Cassandra’s modifications—including its now less complex and shorter-in-size code configurations—make for faster streaming speeds when bootstrapping a new node or during disaster data recovery. “Running repair and dealing with [server failures] means you don’t have to rebuild data so much and you can now run at line speeds as fast as your network,”   McKenzie said.

    The developers also sought to reduce the number of hours spent on performing what used to be an especially onerous task of transferring data to and from different database sources in a network. Multi-cloud and mixing and matching legacy and cloud environments further complicated the process.

    In other database platforms, the process of doubling database capacity can take five to seven months. Building in part on steady improvements past releases have offered, Cassandra 4.0 now can complete the process in just a few hours, McKenzie said. “I love hearing about how people can now run Cassandra with a script and it just takes maybe six hours on a Saturday morning with one person and one with one script instead of a whole weekend,” McKenzie said.”So it’s just really doubling down on those unique strengths that make Cassandra such a good fit for exactly where we are in the world today: in the state of always scrambling with massive influxes of data.”

    The committee members also sought to extend the timeline of the final beta release. Instead of adding more incremental improvements while attempting to accelerate the release cycle, Cassandra’s 4.0 contributors sought “to really take our time, making sure that we were hammering down and smoothing out all the dents,” McKenzie said. “We wanted to get things to where we have a really really stable platform to iterate on very rapidly.”

  • Xen Project Team Updates Open Source Hypervisor Project

    Xen Project Team Updates Open Source Hypervisor Project

    The team behind the open source Xen hypervisor announced today provided an update to the project that adds better nested performance, more robust live patching capabilities, improved security and support for Linux subdomains that make it possible to execute code within a specialized domain.

    In addition, Xen Project Hypervisor 4.14 adds the ability to run Xen as a guest under Hyper-V, the hypervisor Microsoft employs on the Azure cloud.

    George Dunlap, Xen Project Advisory Board Chair, said that latter capability will make it easier to create centrally managed hybrid cloud computing environments that span lightweight instances of Xen at the network edge and instances of Xen running in the Microsoft Azure Cloud. Xen is a core component of the Edge Virtualization Engine (EVE) being developed by LF Edge, which, like the Xen Project, is being developed under the auspices of The Linux Foundation.

    Dunlap also noted IT teams can employ Xen Project Hypervisor to run containers on top of virtual machines, which makes it possible to centrally manage the deployment of containerized applications from the edge to the cloud.

    Xen Project Hypervisor 4.14 also extends Raspberry Pi 4 support for devices with 4GB and 8GB of RAM, in addition to supporting the next-generation AMD EPYC processor, codenamed Milan.

    Finally, Xen Project Hypervisor 4.14 adds support for faster introspection of malware; the ability to live-patch security fixes in a way that ensures they are implemented in the right order; and control flow tools to combat Return-Oriented Programming (ROP) attacks.

    Dunlap noted the Xen team continues to work on Secret-free Xen, which prevents memory from being mapped as part of a side-channel attack.

    Golang bindings are also been expanded to make it easier to develop code on top of Xen using the Go programming language, and add the ability to migrate virtual machines with no drivers or broken drivers.

    While there is no shortage of options when it comes to virtual machines, Dunlap said the more they are viewed as a commodity the more likely organizations will opt to rely on an open source project, as many organizations are deciding there isn’t enough differentiated value to warrant paying for a commercial license.

    It’s too early to say how much organizations will continue to rely on virtual machines. Many IT teams do so now to provide a layer of isolation that is often deemed critical for security. Other IT teams are starting to rely more on containers deployed on bare-metal servers or a lighter-weight instance of a virtual machine that serves only to isolate workloads.

    Regardless of the path forward, the more platforms that run a common layer of virtual infrastructure, the easier it is to manage a hybrid IT environment. The real challenge is being able to enforce that level of IT discipline in an era where developers often spin up virtual machines wherever they find them.

  • Okta Offers PASETO as Alternative to JSON Tokens

    Okta Offers PASETO as Alternative to JSON Tokens

    Okta today launched an open source library for using Platform-Agnostic Security Tokens (PASETO) as an alternative to JSON Web Tokens (JWT) to authenticate end users.

    Randall Degges, head of evangelism for Okta, said PASETO is quickly emerging as an easier, more secure implementation of the JWT specification. PASETO is a draft specification created by Scott Arciszewski that reduces the scope of the Javascript Object Signing and Encryption (JOSE) family of specifications in a way that makes it easier for developers to embrace tokens to secure application access.

    Okta is trying to make it easy for developers to employ PASETO using a library written in Java, dubbed JPASETO, that has half the lines of code JWT token written in Java and is supported by a vendor, he said.

    While JWT tokens have been widely adopted, they are easy to misconfigure, which Degges noted has resulted in the recent discovery of many JWT vulnerabilities. Part of the fault for those vulnerabilities lies with the JWT specification itself, he added; JWTs support a wide range of cryptographic algorithms, including an option that employs no cryptography at all.

    In contrast, Degges said PASETOs are more cryptographically resilient and far easier to employ. The PASETO specification defines two types of tokens: local and public. Local tokens are always symmetrically encrypted with a shared secret key, which means no one can view the contents of a local PASETO unless they have the correct secret key. Public tokens are readable by anyone and are validated with a public key. There is no “none” option; there can’t be a security token that is not encrypted, he said.

    All PASETO formats are designed to be tamper-proof. The entire message is authenticated, so validation will fail if anything in the token changes, added Degges.

    That approach ensures higher levels of application security while at the same time aiding in the adoption of best DevSecOps practices using Okta’s JPASETO library, which can be incorporated easily into the application development process, he noted.

    In recent years software tokens such as JWT have gained traction as a way to implement two-factor authentication in place of creating a session in the server and returning a cookie. When a user successfully logs in using their credentials, a JSON Web Token is returned and must be saved locally.

    Software tokens, however, can still be vulnerable to attacks that either duplicate the underlying cryptographic software or phishing attacks that trick end users into giving up a password. There is no such thing as perfect security; however, software tokens provide a critical layer of security that should be employed much more widely.

    It’s not clear to what degree PASETO will further that goal. Many organizations may even mandate the use of either JWT or PASETO as part of their overall approach to DevSecOps. Regardless of approach, it’s clear that continuing to rely on sessions and cookies to authenticate end users is an antiquated approach to authentication that is not only more difficult to implement and manage but also ultimately less secure.

  • Snyk Tool Prioritizes Open Source Vulnerabilities

    Snyk Tool Prioritizes Open Source Vulnerabilities

    Snyk today announced it has enhanced the ability of its namesake vulnerability scanning tool by adding the ability to identify which open source vulnerabilities should be fixed first using a scoring tool that leverages data science and machine learning algorithms to analyze code.

    In addition, DevOps teams can now take advantage of automated pull requests to fix those vulnerabilities.

    Finally, DevOps teams also can define policies that require certain classes of vulnerabilities to be fixed whenever they are identified.

    Aner Mazur, chief product officer for Snyk, said given the number of open source modules employed and the dependencies that exist between them, staying current on patches and updates has become a major challenge. The only way to achieve that goal is to rely more on tools that help automate that process, he said.

    These days it’s easy for development teams to feel overwhelmed by patch requests to fix open source security vulnerabilities. The trouble is, not all those requests are of equal weight. Snyk provides tools that enable developers to understand which vulnerabilities are of the highest severity and how they might impact their code.

    Armed with that insight, Mazur said, it then becomes possible to prioritize remediation tasks within the context of a DevSecOps process. Otherwise, developers will simply fix vulnerabilities regardless of relevance as time allows. Developers may remediate dozens of bugs only to discover that a critical flaw that had been left unaddressed led to a major compromise.

    It’s also not uncommon for DevOps teams to fail to appreciate dependencies between open source code modules. When patches to code are applied in the wrong order, an application can break, resulting in DevOps teams having to start the whole process over. The prioritization tools provided by Snyk prevent that from occurring, Mazur said.

    In the absence of any ability to prioritize vulnerabilities, developers eventually become inured to patch requests because they don’t know whether their patching efforts are making any real difference, he added.

    Worse yet, all the time spent on patching low-level vulnerabilities reduces the amount of time developers have available to write new application code. In fact, one of the paradoxes of DevOps is as the number of applications deployed increases so too does the amount of time spent debugging them. Before too long, a DevOps team can find itself spending more time fixing existing code than developing new applications.

    As DevOps teams become more dependent on open source code, they benefit from the collective security efforts of all the contributors to that project. The result is more secure code, as patches to remediate specific issues are made available. The challenge now is finding a way to automatically apply those updates in a way that doesn’t conspire to slow down the overall application development process. Achieving that goal requires visibility into what is now a multitude of open source projects that no DevOps team is ever going to be able to consistently achieve on its own.