devopsy.kinsta.cloud

Tag: open source

  • Harness CD Platform Now Available via Source-Available License

    Harness CD Platform Now Available via Source-Available License

    Harness today announced a free community edition of its continuous delivery (CD) platform is now available under a source-available license rather than a more traditional open source license.

    Scott Sanchez, chief marketing officer for Harness, said the goal is to make CD as accessible to developers and DevOps teams as their current CI platform. Harness already makes an open source edition of its continuous integration (CI) platform available.

    The Harness CD platform is available under a Polyform Shield license that provides free access to the platform, including the right to use, modify and distribute it but not launch a competitive offering. A source-available license is an alternative to an open source license that allows an organization to potentially create a fork of the Harness CD that could, for example, be delivered as a cloud service. That specific issue has led to scenarios where vendors that create an open source project find themselves competing for revenue against a cloud service provider that is not necessarily as committed to contributing and maintaining the primary open source project. At this juncture, it’s not clear how many other vendors that make software available under an open source license might take a similar tack.

    While most organizations that embraced DevOps have been able to successfully implement a set of CI best practices, the number of organizations that have implemented CD is not nearly as high. One reason for that is automating deployments across platforms has never been especially easy. Each platform tends to be customized to the point where it is difficult to automate application deployments. In effect, each platform is its own unique type of “snowflake.”

    However, as the number of digital business transformation initiatives continues to steadily increase, the need to regularly update application experiences is going to require more organizations to implement a set of CD best practices. A source-available license should help increase the number of DevOps teams that can access and experiment with a CD platform, noted Sanchez. The challenge, of course, is that many of those applications are now based on microservices with components widely distributed across an IT environment.

    Historically, CI and CD have been synonymous when viewed within the context of a CI/CD platform. It’s not clear to what degree DevOps teams will look to better leverage the CI/CD platforms they already have in place rather than deploying a dedicated CD platform for IT operations teams and letting developers choose whatever CI platform they prefer. In general, a recent survey conducted by the Continuous Delivery Foundation found that 80% of respondents that work for companies with two or more employees are involved in DevOps to some degree.

    Regardless of approach, the coming year is shaping up to be one where many more organizations take DevOps initiatives to the next level. Arguably, as more organizations realize how dependent they are on software to differentiate themselves from rivals, the ability to deliver new application experiences has become critical. At the core of those efforts is a set of DevOps best practices that will ultimately determine just how competitive those organizations will be.

  • White House Meeting Puts Spotlight on OSS Sustainability

    White House Meeting Puts Spotlight on OSS Sustainability

    A recent meeting between IT industry leaders and White House officials highlighted open source software sustainability concerns as high-profile breaches and zero-day attacks have many organizations reviewing their software supply chains.

    The White House published a statement describing, among other things, how participants had a “substantive and constructive” discussion on how to make a difference in the security of open source software while continuing to effectively engage and support the open source community.

    Meeting participants included Deputy National Security Advisor for Cyber and Emerging Technology Anne Neuberger, National Cyber Director Chris Inglis, and officials from the Office of the National Cyber Director, Office of Science and Technology Policy, the Department of Defense, the Department of Commerce, the Department of Energy, the Department of Homeland Security, the Cybersecurity and Infrastructure Security Agency (CISA), the National Institute of Standards and Technology and the National Science Foundation. Akamai, Amazon, Apache Software Foundation, Apple, Cloudflare, Facebook/Meta, GitHub, Google, IBM, the Linux Foundation, the Open Source Security Foundation, Microsoft, Oracle, Red Hat and VMware all sent representatives.

    The White House reported that the discussion focused on preventing security defects and vulnerabilities in code and open source packages, improving the process for finding defects and fixing them and shortening the response time for distributing and implementing fixes.

    There were few specific recommendations, but the White House is clearly bringing more pressure to bear after the disclosure of the zero-day Log4j vulnerability in Java applications wreaked havoc in enterprise IT environments and government agencies. That vulnerability made it clear just how dependent organizations are on open source software projects and brought to attention the fact that many are created and maintained by just a handful of volunteer maintainers and contributors. The individuals that created those projects don’t always have a lot of cybersecurity expertise. In fact, many of them would argue that the onus for securing open source software is on the organizations that use what amounts to free software. It’s not the responsibility of the contributors and maintainers of open source software to drop everything and create a patch on-demand to address a zero-day vulnerability.

    The federal government, however, has made it clear via executive order that it expects IT vendors and large enterprises that depend on open source software to do more to secure it. In the meantime, IT teams will need to evaluate their dependence on open source software; especially if, from a security perspective, that software isn’t sustainable simply because there are not enough contributors with the necessary expertise.

    Naturally, this is a complex issue. In many cases, organizations are relying on open source components without even realizing it. Those components have been embedded within an application by an independent software vendor (ISV) that typically doesn’t disclose how that application was constructed. When a zero-day vulnerability is disclosed, cybersecurity teams can spend weeks looking for all the ways the instances of an application they either built themselves or licensed are impacted.

    “There’s no way to really know,” said Mitch Ashley, principal for Techstrong Research, an arm of Techstrong Group, the parent company that publishes devopsy.kinsta.cloud. “A vulnerability can be anywhere.”

    It’s not abundantly clear how critical an issue the sustainability of open source software security is just yet. Relative to the amount of open source software currently employed, the number of security issues that have been encountered is comparatively small, noted Ashley.

    “The innovation benefits enabled by open source software far outweigh the risks,” he said.

    It may be a while before the open source community comes to terms with rising software supply chain concerns. However, it’s clear that many more members of the IT community are about to be held accountable for it.

  • Using Open Source to Secure Software Supply Chains

    Using Open Source to Secure Software Supply Chains

    Recently, there’s been a lot of attention paid to software supply chain security. In particular, here’s a quote from the May 2021 presidential executive order on improving the nation’s cybersecurity: “The Federal government must … advance toward zero trust architecture; accelerate movement to secure cloud services, including … platform as a service (PaaS).”

    There are two parts necessary to create a truly trusted software supply chain; securing the non-technical areas and securing the technical areas.

    Non-technical aspects of any secure software supply chain involve having individuals or teams focused on security and audit compliance. Internal company policies that act as a regulatory system and set standards for developers are a must, as are efforts to enforce compliance with security best practices. While this can bode well for large organizations, small software engineering teams and startups do not have the bandwidth, budget or culture to make this a reality.

    Envisioning Robust Security Best Practices

    Tools that are open source, governed strictly and enable automation of secure build and deployment are the components that form the technical aspects of the solution. Engineering teams must find a way to envision robust security best practices and find a way to apply them without unduly affecting the developer workflow. This is a founding principle of the DevSecOps efforts within the larger community of software development professionals.

    Ergo, advocating a technology stack that:

    • Is composed of open components
    • Provides automated builds
    • Reduces the dependency on developers
    • Allows security operators to extend control
    • and is actively supported by the community is an ideal choice upon which to build a platform for use by development teams.

    The focus of this article is to describe a robust stack that will sit above compute resources and power applications. The stack will consist of fully customizable open source components that put reliability and security at the forefront.

    Simplify Operations

    Application source code is the single source of truth when working with this stack. The use of Git, a popular version control system that is free and open source will allow developers to work with a source code management system from which all the downstream steps will be triggered.

    Kubernetes is an open source container orchestration tool developed under the auspices of the Cloud Native Computing Foundation (CNCF). It serves as an abstraction over the infrastructure and compute layers to power the system from underneath. Kubernetes has managed to gather a large community following—second only to the Linux OS itself. Using Kubernetes will bring homogeneity above the infrastructure layer and will simplify further operations.

    Cloud Foundry is an open-source PaaS tool. The use of Kubernetes introduces complexity that introduces a bit of overhead for developers who work with the platform. Cloud Foundry provides a countermeasure that simplifies the developer experience and greatly eases the pain points commonly associated with Kubernetes adoption. The Cloud Foundry platform does the job of deploying all the applications to the Kubernetes infrastructure. The cf push command triggers the export of a container image from the application source code.

    When building these containers to deploy to the runtime within Kubernetes, Cloud Foundry uses Paketo buildpacks internally. Paketo buildpacks are an implementation of the cloud-native buildpacks specification which aims to provide a unified means to generate OCI-compatible container images for all languages and frameworks that are commonly used to build applications. Paketo buildpacks are also open source and fully customizable according to the needs of software development teams.

    A Trusted Architecture

    Together, this platform, built on open tools, supported actively by the community and facilitating the exact needs of software developers helps promote a trusted architecture upon which to stage applications. Using these tools would enable an organization to build a software development ecosystem that satisfies all the requirements implied and enumerated in the various recommendations for improving software security.

  • NPMs Sabotaged as OSS Sustainability Crisis Continues

    NPMs Sabotaged as OSS Sustainability Crisis Continues

    A long-simmering debate over the sustainability of smaller open source projects moved beyond the theoretical when two widely used open source node packaged modules (npm) were deliberately sabotaged this week, allegedly by a primary contributor.

    Colors.js is an npm that has been downloaded more than 3.3 billion times, with more than 19,000 projects depending on it. Faker, meanwhile, has been retrieved 272 million times, with over 2,500 projects depending on it. Colors.js enables organizations to print colorful text messages on the console, while faker is used to generate fake data for testing applications. Developers that pulled a recently published Colors.js version found their applications caught in an infinite loop, printing ‘LIBERTY’ ‘LIBERTY’ ‘LIBERTY’ followed by a sequence of gibberish non-ASCII characters. Functional code, meanwhile, was also removed from Faker.

    Ax Sharma, senior security researcher for Sonatype, a provider of an open source software security platform, noted these actions occurred in the wake of the series of zero-day vulnerabilities that impacted the widely used Log4j logging tool for Java applications. Sharma first reported the update issues with Color.js and Faker.

    The small team of contributors that work on the Log4j project found themselves creating multiple updates to the package to address vulnerabilities of varying severity on the common vulnerabilities and exposures (CVE) list. There is some debate, however, over how many of those vulnerabilities warranted a CVE listing given their severity, said Sharma.

    That issue has now emerged as a flashpoint for contributors to smaller open source projects. These contributors contend that larger organizations are taking advantage of their efforts without making any substantial contributions to a project in return, much less compensating any of the contributors for their time and effort. The recent updates made to the npms are, essentially, a protest statement, explained Sharma.

    It’s not clear whether other contributors to small open source projects might follow suit, but the debate over the sustainability of these projects has become heated. Many contributors to open source software assume that the organizations who use the free software they created should assume responsibility for securing it. That “user beware” approach to security is understandable for contributors that are not compensated for their efforts. However, when asked to patch open source projects used by billion-dollar organizations—and do so on an urgent, emergency basis—the resentment among those volunteer contributors rises sharply.

    Fortunately, some effort is being made to address these issues. The Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation, raised $10 million to help maintainers embrace best practices and better protect open source projects from malicious code. Google has pledged $1 million to help open source developers adhere to National Institute of Standards and Technology (NIST) guidelines in response to the Biden administration’s recent executive order on cybersecurity. Administered as a pilot program by the Linux Foundation, that effort is part of a larger $10 billion commitment that Google previously made to open source security.

    White House national security adviser Jake Sullivan also recently sent a letter to major software companies and developers inviting them to discuss initiatives to improve open-source software security. The first step is a one-day discussion this month hosted by Anne Neuberger, the deputy national security advisor for cyber and emerging technology. In the letter, Sullivan specifically noted that while open source software has accelerated the pace of innovation, much of it is maintained by volunteers. This is now a key national security concern, he noted.

    It’s not clear how this emerging open source sustainability crisis will play out. However, DevOps teams might want to consider how dependent they are on open source projects whose contributors might have issues with how their work is being exploited.

  • Open Source Observability Comes of Age in 2022

    Open Source Observability Comes of Age in 2022

    Observability is growing in importance to meet envisioned business objectives. Despite the benefits to performance data, organizations have faced challenges in realizing the full potential of observability based on implementation, distributed software tools, and other technological hurdles. But with new innovation, the era of true observability is upon us.

    Innovative changes

    The open-source observability landscape, in particular, is going through a phase of innovation. Powered by distributed tracing “coming of age” with OpenTelemetry, and eBPF taking center stage in terms of restructuring as the next frontier of observability and monitoring on Linux, innovation is at an all-time high. 

    Kernel technology

    Extended Berkeley Packet Filter (eBPF) has presented a fascinating technology native to the Linux kernel. Born with the intent of having a programmable way of dealing with networking, eBPF has evolved into a rather generic and performant facility to have safe, dynamic access to the Linux kernel, that enables, among many other things, to extract telemetry for observability and security purposes. eBPF is powerful and growing in popularity, and Microsoft has been working on a port to Windows.

    Versatility

    In terms of observability, eBPF proved to be remarkably versatile in 2021. For example, there are tools that use eBPF to perform distributed tracing, like the Pixie project that was open-sourced in mid-2021 by New Relic. But the most promising aspect of eBPF for 2022 is the fact that it is possible, with eBPF, to implement continuous profiling of applications in production. Examples of this are the aforementioned Pixie project and Parca. As proven by tools like Google’s Cloud Profiler and various commercial observability tools, having a low-overhead, always-on profiler in production is a powerful way to troubleshoot latency and memory issues in production environments, which are notoriously hard and time-consuming to reproduce in test labs.

    Clear differences

    While production profiling with eBPF is happening now, it is not yet a general capability for use with most applications. The way profiling works is surprisingly different across different runtimes and programming languages. While pretty much all of the languages behave similarly in terms of using the CPU and allocating memory (and even in such foundational computing matters, there are differences), the way process concurrency is notably different, pretty much across the board. 

    For example, Java is using threads inside the Java Virtual Machine that are mapped to threads in the Operating System (to be fair, there are also other options, like NIO and libraries like RxJava, Reactor, Vert.x, plus Project Loom will eventually ship). In Node.js, on the other hand, concurrency is mostly dealt with the Event Loop, and “real” threads at the Operating System are abstracted away from the developer. Differences like these matter for a profiler, because the data shown to the person troubleshooting an issue must be “translated” to the model for concurrency used by the programming language, and this requires explicit support in the profiler for specific runtimes.

    Open source observability comes of age in 2022

    Fortunately, 2022 is going to be the year that brings us full-featured, open-source, eBPF-based production profiling tools that understand the specifics of the many runtimes that are prevalent in today’s cloud-native applications, like Java, Node.js, Python and .NET, as well as compiled languages like Go and Rust. And that will be a game-changer for improving software and solving issues faster, for all DevOps and SRE and operators.

    As we look out to the year ahead, revolutionary technology like eBPF means the rate of innovation happening at the operating system level is only picking up and setting the stage for the year of true observability. 

  • The Future of OpenTelemetry for Observability

    The Future of OpenTelemetry for Observability

    The observability landscape is changing. Today’s digital transformation journey has highlighted the need for observability as organizations look to capture, analyze and correlate all data across every level of the application stack including logs, metrics, traces and events. With the rampant rise of data, collecting these signals is imperative to both business insights and system optimization.

    Over the last several years, open source data collection has given software and engineering teams the freedom to decide which technologies they want to standardize on for data collection and which vendors they would like data sent to. While open source data collection brought a lot of freedom and advantages, it also created some new problems. The lack of a single, comprehensive framework for all the critical signals observability tracked created a lot of friction for developers. As we move into the new year, businesses are taking the opportunity to reflect on how they can better use their data and apply intelligence to help customers make decisions and sift through all the noise. Enter OpenTelemetry. 

    What is OpenTelemetry?

    OpenTelemetry is a collection of tools, APIs and SDKs that instrument, generate, collect and export all machine data; it was born from the need to develop an industry-wide standard for software instrumentation. By leveraging a single cloud-native framework to complete observability, OpenTelemetry is becoming the gold standard for collecting machine data. 

    OpenTelemetry consists of three primary components: Specification, collector and instrumentation. The specification uses an API, SDK and the semantic conventions and protocol of OpenTelemetry to determine the cross-language requirements for all implementations. The collector removes the need for multiple data collection agents. It is a vendor-neutral method for receiving, processing, transforming and exporting data. The instrumentation libraries allow users to integrate their applications with OpenTelemetry using auto-instrumentation and support from multiple frameworks and languages. Together they provide a robust framework for companies to disrupt their existing observability practices and amplify them even further. 

    It’s All About the Customers

    Most modern enterprises have complex distributed microservices consisting of tens, hundreds or thousands of components that are all working together to deliver on customer experience. If organizations can’t correlate between those components or the different signals easily, it will slow down a business’s ability to effectively monitor, diagnose and troubleshoot issues. 

    For years, organizations have leaned into observability by relying on multiple infrastructure vendors to gather, enrich and ship data to vendors. Still, many still do not maximize observability’s most pronounced ability: Correlating complex systems and signals. 

    As end users desire deep insight into their data, they also expect vendors to offer the best experience to identify and resolve issues within their systems with the help of observability. With true observability, organizations can give customers back their time to focus on what matters, which is managing the digital experiences for their customers and empowering them to meet their customers’ needs. 

    A Similar Road to Adoption

    Much like Kubernetes has been essential for the growing adoption of software containers, the industry is already witnessing the same adoption approach around OpenTelemetry for observability. Kubernetes won the war for becoming the standard for container orchestration. Now, OpenTelemetry will bring the same standards needed for observability with comprehensive visibility into the health and performance of every layer of the stack. 

    OpenTelemetry has become widely adopted by the software development community, but it’s still at the beginning of its journey. The developer community made Kubernetes a success, and we will see the same with OpenTelemetry as customers demand better integration and less lock-in. For vendors involved around applications and software—from more traditional approaches like application performance management to software development pipelines and management tools and through to cloud-native services—supporting OpenTelemetry will be table stakes in the future. 

    With extensive portability, greater developer control and strong vendor and cloud provider support, OpenTelemetry has opened a new world to customers by helping to improve achievable results across the application infrastructure, from the software development process to areas like security.

    How can businesses continue to move the needle on observability while balancing growing customer demands? Quite simply: If organizations aren’t thinking about OpenTelemetry now, they are losing competitive advantage by lacking the proper visibility to deliver on the service level objectives that customers expect in this day and age. 

  • New Relic: Observability Tops 2022 DevOps Agenda

    New Relic: Observability Tops 2022 DevOps Agenda

    If 2021 was the year when IT organizations became aware of the need for observability, then the coming year should see observability aspirations realized.

    A report published by observability platform provider New Relic postulates that there are no less than 11 factors converging in a way that will make observability an absolute requirement for organizations.

    At the top of that list is a simple fact: Observability is now required to drive digital business transformation initiatives that typically span multiple processes. In fact, a global survey of 1,300 IT leaders, software engineers and developers conducted by New Relic found 90% of respondents view observability as critical to the business.

    Second, as a result, observability is now starting to be applied in both pre- and post-production environments as organizations take a more holistic approach to managing the software development life cycle, the report noted.

    Third, the report suggested that IT teams are looking to make better-informed decisions based more on facts than intuition. To do this, teams are using data collected from across the enterprise application environment. In fact, the fourth factor advancing observability in the enterprise is that the cost of instrumenting application environments is dropping, which the report notes is making it easier to collect relevant data.

    The fifth and sixth factors reflect a desire to rationalize the number of monitoring tools that IT teams currently use and the fact that usage-based pricing has simultaneously made it less expensive to leverage observability platforms as needed.

    The seventh factor is the overall shift toward open source software that enables observability platform providers to collectively leverage innovations faster. The eighth factor, closely linked to the open source trend, is the rise of more complex open source platforms such as Kubernetes that require much higher visibility to successfully deploy and manage.

    The ninth trend identified in the report is increased adoption of DevOps best practices that encourage IT teams to view observability as a core tenet of any IT management strategy. The tenth factor cited by the report is the need to drive increased collaboration among IT professionals.

    Finally, the eleventh factor driving further adoption of observability is the need for increased application reliability as more organizations appreciate their dependence on software to deliver essential services.

    Buddy Brewer, a group vice president and general manager for New Relic, said now that observability platforms are becoming more accessible, the next big challenge will be teaching IT professionals what questions to ask as they troubleshoot both legacy monolithic applications and a wide range of emerging microservices-based applications. The results of those queries then need to be easily shared with both developers and internal IT operations teams that must work together to maximize application performance and availability, he added. The latter requirement is why New Relic decided to acquire CodeStream to make telemetry data available with integrated development environments (IDEs), noted Brewer.

    There is, of course, no shortage of observability platforms for IT teams to adopt. In fact, the biggest challenge now is getting every application stakeholder to agree to standardize on an observability platform—no small feat, as many DevOps teams may already have a preference for one approach over another.

  • Best of 2021 – 11 Open Source DevOps Tools We Love For 2021

    Best of 2021 – 11 Open Source DevOps Tools We Love For 2021

    As we close out 2021, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the eighth in our series of the Best of 2021.

    DevOps isn’t just a cultural shift — it requires great tools to come to fruition. Below, we’ve pulled together a list of some of the most well-loved DevOps tools available today. But, throwing loads of money into fancy SaaS solutions can quickly gobble up the cloud budget. These DevOps tools all are open source, and enable everything from container builds and orchestration to microservices networking, configuration management, CI/CD automation, full-stack monitoring and more. Here are some of our favorite open source DevOps tools for 2021.

    1. Kubernetes

    With the ubiquity of microservices and container-based software, it’s no surprise that Kubernetes tops this year’s list of open source DevOps tools. Kubernetes, adoption of which rose by 48% in 2020, is used to orchestrate containers. Instead of releasing microservices manually, Kubernetes can automate deployment, maintenance and scaling of groups of containers in production. Kubernetes, sometimes written as K8s, is hosted by the Cloud-Native Computing Foundation (CNCF).

    2. Docker

    Docker, the software, is a free and open source platform used to build, ship and run an application as a lightweight container. Containers package up the binaries, libraries, configuration files and dependencies required for a program to run. Over the past decade, containers have played a key role in agile development, and Docker containers led the revolution. At its core is the Docker Engine. Docker Hub is also an excellent resource for finding and sharing prepackaged functions as containers. Also, to plug container vulnerabilities, it may be helpful to use open source container auditing tools like Docker Bench or Anchore.

    3. Istio

    Microservices are a handy development style, yet they bring new development and architecture concerns. Namely, how do we apply networking policies like security, encryption, observability and telemetry elements consistently across all our services? Well, service mesh is one answer. Service mesh places a sidecar proxy next to each container and abstracts these networking capabilities to a control plane. Istio is one such open source service mesh that has seen widespread adoption. Istio is built on Envoy, opening it up to plugins and extensibility options. We should also mention Linkerd and Kuma as viable open source service mesh alternatives.

    4. GitHub Actions

    GitHub is arguably the most popular source control and software collaboration platform on the planet. The GitHub platform itself, based on Git, has seen some significant updates in the past few years. Most notable is the GitHub Actions capability. GitHub Actions enable software packages hosted on GitHub to accept inputs and trigger other processes. This could help automate some cool DevOps workflows within GitHub, such as code reviews, branch management or CI/CD processes — the combinations of possibilities here are endless. GitHub Actions are, essentially, YAML files hosted in GitHub repositories that leverage GitHub webhooks. Though this is more of a feature than an open-source tool, we feel it’s important to include here. Actions is free for public repositories with a limit of 100 actions.

    5. Jenkins

    A big part of the DevOps philosophy is finding ways to automate and deploy new iterations more efficiently. Part of this goal is creating a streamlined continuous integration and continuous delivery (CI/CD) pipeline. Jenkins is an open source automation server with hundreds of plugins to automate the building, deployment and testing of software projects. Although GitHub Actions could theoretically replace a CI server in the future, CI tools like Jenkins, CircleCI, TravisCI and GitLab Community Edition still are preferred by many DevOps teams.

    6. Prometheus

    Metrics and alerting systems are crucial for site reliability engineers to visualize applications and react to issues. Prometheus, a graduated CNCF project, is a well-loved open source monitoring solution. A Prometheus server collects time-series metrics by scraping HTTP endpoints and generates a system to interact with this data, offering deep querying, visualization, storage and other capabilities. Check out this Awesome Prometheus list for Prometheus introductions and additional resources.

    7. Ansible

    Ansible is all about automation. Ansible, an open source project sponsored by Red Hat, can be used to automate things like cloud provisioning, networking, deployment, configuration management and other tasks. Ansible has a simple yet effective architecture that is relatively easy to assemble — all you need is a text editor and command line. You describe your infrastructure in a text document and organize your desired states in a playbook. For an example in practice, see how OpenIO uses Ansible. “Ansible is our standard tool not only to deploy the OpenIO core, but also our WebUI, OIO-FS and all upcoming options,” writes Cédric Delgehier, Ops at OpenIO.

    8. Chef

    Chef is another infrastructure-as-code (IaC) solution for automating your configuration management. Chef uses Ruby to automate server configurations and works well with all major cloud service providers (CSPs). This can be very useful when creating and provisioning large quantities of machines. Like other automation tools on this list, the user describes their components and their states in a declarative format. In Chef, these are known as “recipes,” which can be grouped together into “cookbooks.” You can’t diss Chef for not being on-theme!

    9. Terraform

    Terraform is another IaC tool that can be used for initiation of building, versioning and further automation using configuration files. “Terraform is a tool for building, changing and versioning infrastructure safely and efficiently,” as it is described on GitHub. Terraform follows an “execution plan” that a user creates with high-level syntax. One unique aspect of Terraform is its emphasis on versioning — this allows you to version the blueprint of your service just as you would your software.

    10. JAMStack

    As I’ve covered before, JAMStack combines JavaScript, APIs and markdown for constructing web-based applications. While more of a “headless development” methodology than a single open source tool, JAMStack projects are often built using open source components. For example, JAMStack often leverages open source headless content management systems such as Ghost, Strapi and/or Netlify CMS.

    11. ELK Stack

    ELK Stack is the union of three open source projects maintained by Elastic: Elasticsearch, Logstash and Kibana. With these three components, developers can take in and log data from any source and create helpful visualizations. This centralized logging is enabled with a NoSQL database for storage with Elasticsearch, processing and data collection with Logstash and visualization with Kibana. Increased visibility is vital for data analysis and helps identify errors to reduce mean time to recovery (MTTR).

    Rounding Out The DevOps Toolchain for 2021

    Programming isn’t only about shipping quality code, it’s about efficient execution. This drive to improve operations across the board is truly putting Ops into everything. With awesome open source DevOps tooling, it means more and more architects can adopt a DevOps approach within their deployment models.

    DevOps as a practice, as well as its underlying technology, is always evolving. In 2021, a lot of effort is being placed on wrangling the effects of introducing a microservices development style. We’ve noticed the maturation of container orchestrators and service mesh, as a result. Blueprinting infrastructure-as-code and creating automated, repeatable configurations is another must-have for implementing automated build and release pipelines. Not to mention, many no-code and low-code platforms are opening up DevOps capabilities to non-programmers, albeit from a proprietary perspective.

    It’s also helpful to note that some popular open source DevOps tools have been acquired — like Docker and Chef — blurring the lines between their business and their open source roots. When it comes to open source tools, it’s a good practice to adopt vendor-neutral tools with solid community support representing diverse stakeholders. This will help future-proof your project. Even though open source is “free,” users must ensure the benefits outweigh the time-to-onboard as well as the operational overhead of self-hosting.

    DevOps tools are beneficial to automate software deployment. Here, we’ve attempted to list some of the most exciting tools to help automate that process. Of course, many new tools are emerging in the DevOps space, and we have hardly scratched the surface. Do you have a favorite DevOps tool that’s not mentioned above? Let us know in the comments below!

  • Best of 2021 – 7 Popular Open Source CI/CD Tools

    Best of 2021 – 7 Popular Open Source CI/CD Tools

    As we close out 2021, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the fourth in our series of the Best of 2021.

    DevOps is a software development strategy that incorporates agile practices for fast, efficient product creation and release. It focuses on integration of development and operations teams, continuous integration/continuous delivery (CI/CD) and automation of tasks and processes.

    Typically, DevOps teams use pipelines to streamline and standardize processes. DevOps pipelines are toolchains that teams can use to automate tasks and provide visibility into the software development life cycle. In this article, we’ll cover seven popular open source CI/CD tools.

    How DevOps Will Change in the Near Future

    Although DevOps has been around for just over a decade, it has continued to evolve. New DevOps technologies and changing infrastructure have caused teams to adapt and technology advancements that will continue to drive change. Below are a few of the ways you can expect DevOps to evolve in 2021 and beyond.

    AI and Data Science

    Artificial intelligence (AI) and machine learning have significant potential to improve current DevOps processes. Teams can use AI to identify inefficiencies in pipelines, optimize testing or evaluate user needs. Increasing amounts of data about environments, customers, markets and individual habits can all be applied to increase productivity and, ultimately, revenues.

    Serverless Architectures

    Serverless platforms, such as the AWS serverless ecosystem, including AWS Lambda and Fargate and the Azure serverless platform, including Azure Functions and Azure App Service, are becoming increasingly popular. These platforms enable DevOps teams to focus on just their code and eliminate many other variables, such as concern for host operating systems. These architectures provide scalability, deployment speeds, and user experiences that are difficult to match. This value will further secure FaaS in the coming years.

    Cloud-Native

    The growth of cloud-native applications has both driven and been driven by widespread adoption of cloud services. The containers and microservices that make up many cloud applications are used in many pipeline deployments and environments. Efficiently developing cloud-native applications require agility and automation that are difficult to meet without DevOps strategies.

    For the above reasons, DevOps has become almost synonymous with cloud-native. As cloud-native technologies change, tooling and processes adapt, as well. Additionally, since cloud growth isn’t slowing down, you can expect that DevOps will continue to focus on and refine cloud-native development.

    DevSecOps

    When developing applications in an automated CI/CD pipeline, security cannot be an afterthought. Especially in complex microservices architectures, security must be built in from the planning and development stages. Everyone must be aware of security requirements – developers, DevOps engineers and infrastructure teams.

    Transitioning to a DevSecOps paradigm requires a culture change, but also a new technology stack. Shifting security left requires tight integration of application security tooling into the development life cycle, and implementation of cloud security solutions that can help gain visibility, and apply automated security controls, to complex cloud environments.

    Which CI/CD Tools will You Use in 2021 and Beyond?

    As technologies change, so does tooling. While you can continue to use the same tools you started with, you should at least consider if other tools can increase your efficiency. Below are a few popular, open source tools you might want to integrate.

    Argo CD

    Argo CD is a CI/CD tool for Kubernetes development. It is an open source project which is currently in the incubation status at the Cloud Native Computing Foundation (CNCF). It uses Git repositories to store the state of Kubernetes applications, monitors applications and can resync clusters to the desired state, as represented in the git configuration.

    This innovative approach also allows you to store multiple desired states of a Kubernetes application, using branches, tags, or by pinning manifest versions using a Git commit. This provides a flexible environment for managing Kubernetes configurations during the development process.

    CircleCI

    CircleCI is an open source CI/CD tool. It includes features for job orchestration, resource configuration, caching, debugging, security and dashboard reports. CircleCI integrates with a variety of tools, including GitHub, Heroku, Slack and Docker.

    CircleCI is available in three tiers, one of which is free. You can use it in the cloud or on-premises with Linux, Mac or Windows machines.

    GitHub Actions

    GitHub Actions are a recently released open source tooling feature you can use to automate workflows. It enables you to build, test and deploy code directly from GitHub. You can base your automation on any GitHub events, including push or issue creation.

    GitHub actions include features for matrix workflows, hosted runners for all major operating systems, built-in secrets storage and live log reviews. It supports all common programming languages.

    GitHub actions includes hosted runtime environments that are free to use for the first 2000 minutes of use per month. If you need more time for testing, you can purchase time by the minute or in bulk. You can use GitHub actions by including them in your YAML files.

    GoCD

    GoCD is an open source build tool that lets you build CI/CD development pipelines. At its core, it is a continuous integration (CI) server, which lets you work with complex release workflows. It goes beyond classic CI to let you construct a full continuous delivery (CD) pipeline, all the way from automated and secure deployment to production.

    GoCD defines pipelines using code checked into source control – this infrastructure-as-code (IaC) approach lets you test, manage and reuse pipelines across multiple projects. Pipelines can be represented by JSON or YAML templates.

    Another feature of GoCD is the ability to run pipelines both sequentially and in parallel, with configurable dependencies. You can view entire workflows from development to production and track a feature from initial commit to production deployment, using value stream mapping.

    GoCD is extremely flexible, providing an ecosystem of plugins and the tool also allows you to develop and contribute your own.

    Gradle

    Gradle is an open source build automation tool that has been around for about a decade. It continues to be well-adopted and supported. Although it is not a new tool, it can be a good alternative if you’re looking to replace Maven as your build management solution. Gradle is more extensible and faster than Maven, and can be used for native development of C/C++ applications.

    Gradle includes features for incremental builds, build caching and parallel execution. It supports a variety of languages and platforms, including Java, Scala, Android and C/C++. You can integrate Gradle with a range of tools, including Eclipse, IntelliJ and Jenkins.

    You can use Gradle on-premises or with a virtual machine in the cloud. Gradle is free to use; however, if you want enterprise support or training, you need to pay an additional fee.

    Helm

    Helm is an open source Kubernetes package manager. You can use it to define, install and upgrade Kubernetes-orchestrated applications. Helm includes a wide range of pre-built charts (sets of distributable Kubernetes resources) that you can use to deploy services faster and more easily. You can use Helm on Linux, Mac and Windows devices.

    Jenkins X

    Jenkins X is an updated version of the well-known Jenkins. It builds on the existing features of Jenkins with capabilities tailored to Kubernetes and Docker workflows. It includes features for pre-built pipelines, integrated GitOps support, automatically generated preview environments and feedback integration.

    Jenkins X integrates with Tekton, Prow, Helm, Knative and Skaffold. You can use it on Linux, Mac or Windows machines.

    CI/CD and Automation Go Hand-in-Hand

    Agile teams need automation tooling, especially when creating and working with CI/CD pipelines. DevOps environments are dynamic, built for fluid workflows and constant change. Automation helps DevOps teams maintain a high level of visibility and reduce time spent on repetitive tasks. When adding AI and data science into the mix, DevOps teams gain the ability to make data-driven decisions, in addition to automated alerts and response.

  • Best of 2021 – Torvalds’ Bug Warning is a Lesson for Linux Users 

    Best of 2021 – Torvalds’ Bug Warning is a Lesson for Linux Users 

    As we close out 2021, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the third in our series of the Best of 2021.

    Linux does, occasionally, raise security concerns. While many users see it as the most secure, robust and versatile operating system available — that’s this writer’s opinion, as well — security precautions still have to be taken.

    A recent, widely publicized case illustrated this point; Linux creator himself, Linus Torvalds, warned against the use of the Linux 5.12 release. He described a “nasty bug,” and wrote that the situation is a “mess,” due to the use of swap files when adding Linux updates. This nasty bug, in fact, had the potential to destroy entire root directories.

    Some of the main takeaways following this “mess” include: tread very carefully when installing early Linux releases, especially those that involve swapping files instead of partitions, and especially, despite Linux’s well-known security advantages, avoid becoming complacent, because Linux security is not always foolproof.

    Hence, while the “state of Linux security today is quite good, and has evolved in a positive way with more visibility and security features built, like many operating systems, you must install, configure and manage it with security in mind; that is how cybercriminals take advantage, [via] the human touch,” said Joseph Carson, chief security scientist and advisory CISO at Thycotic, a provider of privileged access management (PAM) solutions.

    A Patch for Nastiness

    As Torvalds noted a few weeks ago, “most people don’t use a swap file, but a separate swap partition and the bug in question really only happens when you have a regular file system, and put a file on as a swap.”

    “The bad news is that the reason we support swap files in the first place is that they do end up having some flexibility advantages, and so some people do use them for that reason. If so, do not use [release candidate] RC1,” Torvalds wrote. “Thus, the renaming of the tag.”

    After issuing the warning, Torvalds released a patch that he says prevents the bug from destroying swap file systems. However, it may have already been too late for early adopters of release 5.12. Ubuntu, a leading Linux distro, can swap files by default.

    “It is nasty bug if you are still using swap files,” Carson said. “If you do still use swap files, then you could be impacted, resulting in potential data loss or a corrupted system.”

    DevOps teams – or anyone else running Linux and installing patches, whether on multi-servers or on individual workstations – still need, of course, to follow strict best practices. “Like any operating system, security depends entirely on how you use, configure or manage the operating system,” Carson said. “Each new Linux update tries to improve security; however, to get the value, you must enable and configure it correctly.”

    Linux Goodness

    The fact that Torvalds was so forthcoming about the bug, as well as the level of transparency that the Linux kernel offers, also demonstrates one of the many reasons Linux remains popular. Given that the Linux kernel, in one variety or another, is used “not only in about 50% of the internet servers of the world, but also in a substantial part of all our smartphones, it is good to see this level of transparency at ‘root level,” said Dirk Schrader, global vice president, security research at New Net Technologies (NNT), which providers cybersecurity and compliance software.

    “The security of Linux is based on its transparency; the ability to review the code of a distribution,” says Schrader. “Quite often forgotten is that transparency also involves talking about the mistakes, the errors, those nasty bugs.”

    Citing National Institute of Standards and Technology (NIST) vulnerability database statistics, Schrader described how, compared to the Windows family of desktop and server operating systems, for example, the Linux kernel shows better results for overall vulnerabilities. The number of vulnerabilities have also declined over the past four years, while Microsoft’s operating systems do not display the same trend, according to NIST’s national vulnerability database.

    Since Linux’s famous kernel is open source and transparent, it is possible to extrapolate that there are a greater number of potential vulnerability watchdogs compared to those monitoring vulnerabilities in closed systems. Some may argue that Microsoft has been, at times, less successful at detecting vulnerabilities and issuing much-needed patches.

    However, Linux users still must remain vigilant.

    “Still, for any of the Linux distributions, anyone using the early release candidates — RC1 in particular — should make sure that their own development or build process is undergoing change control, so that no mishaps will transfer the nasty bug into a production environment,”  said Schrader.