devopsy.kinsta.cloud

Tag: ebpf

  • Latest eBPF Advances Are Harbingers of Major Changes to IT

    Latest eBPF Advances Are Harbingers of Major Changes to IT

    The open source community advancing the extended Berkeley Packet Filter (eBPF) gathered at a virtual eBPF Summit today that featured a demonstration of eBPF being used in a Windows environment. eBPF enables applications to run within a sandbox in the Linux microkernel.

    Microsoft has signaled its intent to employ eBPF within Windows like the latest version of Linux already does today. That capability makes it possible to, for example, run networking, security, observability and storage software closer to the kernel to dramatically improve performance versus running that software in user space.

    Liz Rice, chief open source officer for Isovalent, a provider of networking software based on eBPF, said that while it’s clear eBPF will have a major impact on IT, the transition to eBPF won’t be a single event. Instead, it will be more of an evolutionary process that will occur as organizations upgrade their operating systems and the various tools and applications they employ.

    The eBPF Foundation, an arm of the Linux Foundation, was formed to expand the use of eBPF across all operating systems. Members of the eBPF Foundation include Facebook, Google, Isovalent, Microsoft and Netflix.

    In effect, eBPF changes the way operating systems are designed. It bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that were historically completely independent. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput at a time when brute-force cyberattacks are overwhelming existing cybersecurity tools.

    Similarly, networking, storage and observability tools should be able to run at much higher levels of scale. That’s especially critical for DevOps teams embracing observability platforms that need to dynamically process massive amounts of data in near-real-time.

    Less clear is the degree to which eBPF might one day drive convergence across distinct categories once multiple tools and applications run at the microkernel level.

    IT organizations would be well-advised to ask their vendors when they plan to support eBPF as they plan their upgrade cycles. It’s not likely IT teams are going to want to upgrade every tool and application that can benefit from eBPF all at once. In fact, updates to these applications that don’t support eBPF might even be postponed in the knowledge that a much larger and arguably more critical update is on the immediate horizon.

    In the meantime, as the number of organizations running the latest versions of Linux continues to increase, more hands-on experience with eBPF will be gained. IT teams may not need to concern themselves with what is occurring in the microkernel of the operating systems, but they should at least understand how eBPF ultimately helps reduce the total cost of running IT and how it makes the Linux microkernel more programmable. Those IT teams, however, will soon discover how stark a difference in performance there will be.

  • Cycode Leverages eBPF to Secure CI/CD Pipelines

    Cycode Leverages eBPF to Secure CI/CD Pipelines

    Cycode today added a Cimon extension to its application security platform that uses extended Berkeley Packet Filtering (eBPF) to thwart cyberattacks against continuous integration/continuous delivery (CI/CD) pipelines.

    Alex Ilgayev, head of security research for Cycode, said Cimon uses eBPF to inspect network connections, running processes and file modifications within a pipeline to learn standard behaviors. Armed with those insights, Cimon will then surface any anomaly in real-time, he added.

    DevOps teams can then craft and embed security and compliance rules within their pipelines based on the severity of the anomalies discovered, said Ilgayev.

    eBPF makes it possible to run sandbox programs at the kernel level without adversely impacting performance. In effect, eBPF bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that, historically, were completely independent of one another. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput to improve overall scale at a time when the volume of cybersecurity attacks continues to increase.

    Cycode is taking advantage of eBPF to enable DevOps teams to not only defend against known attacks involving malicious package installation, typosquatting, repojacking, dependency confusion, dependency hijacking and other types of dependency attacks but also zero-day threats, noted Ilgayev.

    Cycode provides a platform for mapping metadata and events using graph technology that makes it simpler for application development and cybersecurity to consolidate alerts generated by their DevSevOps tools. Most recently, Cycode added a Cycode Application Security Orchestration and Correlation (ASOC) module that will automatically discover all the tools that make up a DevSecOps workflow without requiring any integration effort.

    The approach enables the Cycode platform to discover, for example, dependency issues that would otherwise be missed by software composition analysis (SCA) or static application security testing tools (SAST).

    While a lot of progress has been made in terms of adopting DevSecOps best practices, many organizations are still struggling with securing their software supply chains. Most of the members of a DevOps team have limited cybersecurity expertise, so they need the help of a cybersecurity professional to determine what specific actions are required to remediate a vulnerability. Cybersecurity professionals, conversely, don’t have a lot of application development expertise.

    As the number of regulations specifically focused on application security steadily increase, every organization that builds software will soon need to embrace DevSecOps to better secure software supply chains. Unfortunately, developers have too often viewed cybersecurity as an obstacle to deployment while cybersecurity teams have historically considered developers to be a primary cause of the breaches they are expected to clean up. The first step toward bridging that divide is, of course, to realize that application developers and cybersecurity teams need to be invested in each other’s success.

  • Libbpf Vs. BCC for BPF Development

    Libbpf Vs. BCC for BPF Development

    If you’re into Linux development, you’ve probably heard BPF mentioned over the last few years. BPF stands for Berkeley Packet Filter, and the technology has a large number of use cases. BPF apps can get deep access into an operating system and enable you to perform tasks such as high-performance load balancing, DDoS mitigation and more simply and easily.

    Libbpf and BCC-Tools are both sets of tools to help with BPF development. However, both of these have their own strengths and weaknesses. In this guide, you will learn in detail about the two tool collections and understand when to use them.

    What is BCC?

    BCC stands for BPF Compiler Collection and is one of the oldest ways to develop BPF applications. It helps you embed your BPF code into your user-space program in the form of a plain string. When the user-space program is executed by the kernel, BCC invokes its embedded Clang/LLVM, pulls in system-wide kernel headers and compiles the program on the spot. 

    Since BCC compiles BPF programs on the host machine, it ensures that the memory layout your BPF program expects is precisely the same as that of the target host.

    BPF programs are designed to be injected directly into a kernel, so BCC tools seem to be the perfect solution for developing BPF applications. However, they have proven to be bulky in the modern context due to their heavy reliance on the Clang/LLVM combination, which is resource-intensive. This has led to the need for a better, more modern solution to developing BPF apps.

    What is Libbpf?

    Libbpf is one of the hottest new BPF tools on the market. It is usually coupled with BPF CO-RE (which stands for compile once, run everywhere). BPF CO-RE enables you to generate binaries that run on multiple kernel versions.

    The idea behind Libbpf is to make BPF development as similar to other forms of development as possible. With BPF CO-RE, Libbpf does this by compiling BPF programs into small binaries that can be deployed to multiple deployment hosts. Libbpf does the setup work like loading and verifying programs, creating maps, attaching to hooks, etc., which enables developers to focus on more critical tasks at hand, such as program performance and correctness.

    Libbpf aims to eliminate the overheads associated with BPF app development and deployment by reducing the dependency on system-wide kernel headers as well as Clang/LLVM libraries for compilation on runtime.

    Libbpf Vs. BCC: Key Takeaways

    Now that you understand how each of these tools works, let’s compare some key benchmarks to learn when to use them.

    Dependency Management

    BCC relies on kernel header packages that need to be installed on the target host machines. While it is not a problem in most cases, it can get difficult to set up and maintain if you are working on multiple machines. 

    Libbpf, on the other hand, eliminates this issue by relying on a vmheader file. This file includes multiple kernel types which helps you to remove dependency from system-wide kernel headers.

    Compilation Style

    For programs written with BCC, compilation occurs at runtime. To facilitate this, such programs require Clang/LLVM be present on the host machine. This adds to the code footprint of your programs.

    Furthermore, the Clang/LLVM libraries are resource-intensive. Complete libraries need to be available and run at compile time, even for small programs. This can upset a perfectly balanced BPF workload on the host machine.

    On the other hand, Libbpf enables you to generate binaries that are compiled once and can be run anywhere. You do not need any system-wide dependencies to be present on the target machine for running such apps. Hence, it reduces the overall application size as well as resource consumption on runtime.

    Error Detection

    For apps developed with BCC tools, errors are detected only at runtime since that is when the programs are compiled. This leads to a rather sluggish development experience compared to apps developed using Libbpf. Since Libbpf enables compilation while developing, finding and fixing bugs is easier.

    Final Thoughts

    In this guide, we walked you through what Libbpf and BCC tools are and compared them based on some important aspects of application development and delivery. 

    All in all, Libbpf appears to best BCC tools in most aspects. BCC tools offer fast prototyping and experimentation possibilities, but when it comes to production deployments, BCC tools turn out to be a rather expensive choice compared to Libbpf. 

    You can learn more about why BCC became the popular alternative and how it was eventually replaced by Libbpf here. Even if you currently use BCC tools for your apps, you can easily migrate to Libbpf by following a few simple steps. And, in most cases, it is better to opt for Libbpf than BCC tools for a better development experience and application performance.

  • Open Source Observability Comes of Age in 2022

    Open Source Observability Comes of Age in 2022

    Observability is growing in importance to meet envisioned business objectives. Despite the benefits to performance data, organizations have faced challenges in realizing the full potential of observability based on implementation, distributed software tools, and other technological hurdles. But with new innovation, the era of true observability is upon us.

    Innovative changes

    The open-source observability landscape, in particular, is going through a phase of innovation. Powered by distributed tracing “coming of age” with OpenTelemetry, and eBPF taking center stage in terms of restructuring as the next frontier of observability and monitoring on Linux, innovation is at an all-time high. 

    Kernel technology

    Extended Berkeley Packet Filter (eBPF) has presented a fascinating technology native to the Linux kernel. Born with the intent of having a programmable way of dealing with networking, eBPF has evolved into a rather generic and performant facility to have safe, dynamic access to the Linux kernel, that enables, among many other things, to extract telemetry for observability and security purposes. eBPF is powerful and growing in popularity, and Microsoft has been working on a port to Windows.

    Versatility

    In terms of observability, eBPF proved to be remarkably versatile in 2021. For example, there are tools that use eBPF to perform distributed tracing, like the Pixie project that was open-sourced in mid-2021 by New Relic. But the most promising aspect of eBPF for 2022 is the fact that it is possible, with eBPF, to implement continuous profiling of applications in production. Examples of this are the aforementioned Pixie project and Parca. As proven by tools like Google’s Cloud Profiler and various commercial observability tools, having a low-overhead, always-on profiler in production is a powerful way to troubleshoot latency and memory issues in production environments, which are notoriously hard and time-consuming to reproduce in test labs.

    Clear differences

    While production profiling with eBPF is happening now, it is not yet a general capability for use with most applications. The way profiling works is surprisingly different across different runtimes and programming languages. While pretty much all of the languages behave similarly in terms of using the CPU and allocating memory (and even in such foundational computing matters, there are differences), the way process concurrency is notably different, pretty much across the board. 

    For example, Java is using threads inside the Java Virtual Machine that are mapped to threads in the Operating System (to be fair, there are also other options, like NIO and libraries like RxJava, Reactor, Vert.x, plus Project Loom will eventually ship). In Node.js, on the other hand, concurrency is mostly dealt with the Event Loop, and “real” threads at the Operating System are abstracted away from the developer. Differences like these matter for a profiler, because the data shown to the person troubleshooting an issue must be “translated” to the model for concurrency used by the programming language, and this requires explicit support in the profiler for specific runtimes.

    Open source observability comes of age in 2022

    Fortunately, 2022 is going to be the year that brings us full-featured, open-source, eBPF-based production profiling tools that understand the specifics of the many runtimes that are prevalent in today’s cloud-native applications, like Java, Node.js, Python and .NET, as well as compiled languages like Go and Rust. And that will be a game-changer for improving software and solving issues faster, for all DevOps and SRE and operators.

    As we look out to the year ahead, revolutionary technology like eBPF means the rate of innovation happening at the operating system level is only picking up and setting the stage for the year of true observability. 

  • LF Networking (LFN) Adds to Open Source Portfolio

    LF Networking (LFN) Adds to Open Source Portfolio

    At the online Open Networking & Edge Summit + Kubernetes on Edge Day event LF Networking (LFN), an arm of the Linux Foundation focused on networking, today added several projects that promise to make it simpler to programmatically manage networking services based on open source software.

    Walmart is contributing a L3AF project for managing network applications that make use of an extended Berkeley Packet Filter (eBPF) capability that allows applications to run in a sandbox at the microkernel level of an operating system. Intel and Aarna Networks, meanwhile, are donating the seed code from an EMCO project that provides a universal control plane to securely connect and deploy workloads across distributed computing environments that span everything from the edge to the cloud.

    Arpit Joshipura, general manager for networking, edge and internet of things (IoT) for the Linux Foundation, said the L3AF project adds a control plane for managing those applications to the LFN portfolio that addresses load balancing, rate limiting, traffic mirroring, flow exporter, packet manipulation, performance tuning and other management tasks. Earlier this year, the Linux Foundation set up a separate eBPF Foundation to further advance development and adoption of eBPF across multiple operating systems.

    At the same time, LFN also revealed today that Project Alvarium, a tool for measuring how well data is being delivered to devices that was originally created by Dell, and eKuiper, an existing open source lightweight IoT data analytics and streaming platform, are now part of the LF EDGE portfolio of projects.

    Finally, an open source mobile edge computing project, dubbed Edge Gallery, that was originally created by Huawei has been contributed to the Linux Foundation.

    Joshipura said the LFN continues to gain momentum as more vendors contribute core technologies that, ultimately, don’t provide them with enough differentiated value to continue to develop on their own. The LFN today also revealed that VMware and F5 Networks have joined the LF EDGE consortium as premier members, while mimik and Veena, Inc. have become general members of LFN and the Eclipse Foundation has become an associate member.

    The LFN also provides a mechanism that promotes adoption of those technologies more widely than any one vendor can achieve on its own. The total cost of delivering next-generation edge computing platforms based on wireless 5G networks will also be sharply reduced as LFN projects continue to mature, said Joshipura.

    In general, the rate at which various LFN projects are updated continues to accelerate, said Joshipura. The Akraino project for creating blueprints for edge computing platforms recently delivered its fifth update to address use cases involving smart cities, cloud-native multi-tenant environments and vehicular networks. An EdgeX Foundry project for IoT platforms, meanwhile, recently revamped its application programming interfaces (APIs) to make them more secure and enable more message-based communications. The Home Edge project will also soon issue a Drewberries update that will address code stabilization, scripting, APIs, data synchronization and security.

    It’s now only a matter of time before these LFN projects become more integrated with DevOps workflows as IT teams seek to give developers more programmatic control over network services. The issue now is deciding not so much whether to integrate network operations and DevOps teams but rather determining to what degree those teams will need to fold into each other.

  • Foundation Proposes Advancing eBPF Adoption Across Multiple OSes

    Foundation Proposes Advancing eBPF Adoption Across Multiple OSes

    The eBPF Foundation, which proposes to advance adoption of an approach that enables sandbox programs to run faster at the kernel level, is being launched today as an arm of the Linux Foundation.

    The technology known as eBPF was originally developed for Linux. The eBPF Foundation is now committed to expanding use of eBPF across all operating systems. Members of the eBPF Foundation include Facebook, Google, Isovalent, Microsoft and Netflix.

    Thomas Graf, CTO and co-founder of Isovalent, a provider of networking and security tools that leverage eBPF, said IT organizations should expect to see the technology become part of Windows, along with other operating systems, as the core technology is tested and validated across a much wider range of platforms.

    In effect, eBPF changes the way operating systems are designed. It bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that, historically, were completely independent of one another. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput to improve overall scale at a time when the volume of cybersecurity attacks being simultaneously launched continues to increase.

    Currently, eBPF is widely employed by web-scale companies such as cloud service providers. Facebook is using it as the primary software-defined load balancer in its data centers while Google is using open source Cilium networking software within its managed Kubernetes offerings.

    However, adoption within on-premises IT environments that have deployed Linux has been more limited simply because the number of vendors that have optimized networking, security and storage offerings to take advantage of eBPF is still fairly limited.

    Sysdig, for example, recently launched made available an instance of the open source Falco container security platform available that employs eBPF. Tigera has also made available an instance of its container networking platform that leverages eBPF at the kernel level of Linux.

    Ultimately, Graf notes the biggest benefit of eBPF is efficiency. The total cost of processing for security, networking and storage platforms should decline as more vendors take advantage of a capability that will one day be widely employed across multiple operating systems.

    In the meantime, IT organizations would be well-advised to ask their vendors when they plan to support eBPF. Cloud service providers generally require it to deliver their own managed services more efficiently, so any vendor that hopes to sell a platform to those providers will need to support eBPF to drive performance at scale. The issue then becomes determining which platforms that have embraced the technology are also available in on-premises IT environments looking for similar benefits.

    Regardless of approach, it’s clear there will soon be a reason to upgrade not just networking, storage and security platforms but also instances of operating systems that don’t yet support eBPF. DevOps teams would be well-advised to plan accordingly because the number of platforms that will ultimately benefit from eBPF spans the entire enterprise.