Tag: open source

  • IBM/Red Hat Sparks Anger at GPL ‘breach’ as RHEL Source Locked Up

    IBM/Red Hat Sparks Anger at GPL ‘breach’ as RHEL Source Locked Up

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Red Hat Enterprise Linux (RHEL) goes closed source. Well—no, not really; but IBM’s lawyers are accused of skating so close to the edge of legality that they risk violating the GNU Public License. (more…)

  • Integral Makes Generative AI Code Review Tool Open Source

    Integral Makes Generative AI Code Review Tool Open Source

    Integral this week launched an open source Robin AI project that leverages OpenAI’s generative pre-trained transformer (GPT) platform to review code changes and provide constructive feedback.

    Integral is a data analytics platform used by health care providers. The company originally developed Robin AI using a set of Bash scripts to create a generative artificial intelligence (AI) tool to assist its internal development team.

    Integral CTO John Kuhn said Robin AI is essentially a bot that functions like a coach for developers. It first reviews code and then surfaces optimization suggestions that developers can instantly apply, said Kuhn. It’s up to each developer to decide whether to accept those suggestions, but Integral is already using it to, for example, make sure all the code being written is human readable, he added.

    Robin AI currently works best on JavaScript code repositories, but can be applied to any codebase residing in a Git repository. Integral has also already developed a GitHub Actions script.

    In addition, as an open source project, Robin AI can be integrated with any number of generative AI platforms that might be built using more domain-specific large language models (LLMs), noted Kuhn.

    Most developers are already using AI tools such as GitHub Copilot to write better code. Robin AI is now extending generative AI capabilities to the code review process as part of an effort to improve the quality of the code being written. In effect, it provides a means to surface common mistakes developers make and enables them to fix those before they are discovered by a colleague.

    It’s not clear how much application development will be driven by AI, but Kuhn said he believes it’s only a matter time before the entire software engineering process is automated. As those advances are made, the cost of building applications will effectively drop to zero, he added.

    Of course, not everyone in the application development community agrees with that assessment. But one thing that is clear is that generative AI capabilities will soon be pervasive. In fact, the next frontier will arguably involve integrating generative AI platforms with frameworks that make it possible to automatically apply suggestions and recommendations made by generative AI platforms across an entire enterprise IT environment.

    In the meantime, DevOps teams should be evaluating generative AI technologies in terms of the capabilities they can provide today and their future potential. Many of the manual tasks that conspire to make application development and deployment tedious will become increasingly automated. DevOps teams committed to ruthlessly automating as many processes as possible will naturally be at the forefront of adoption. The challenge and the opportunity lies in determining whether AI platforms can be trusted or if a human must always be in the loop to ensure there are no unexpected outcomes.

    After all, while AI can be applied to automate deployments, it’s not quite as clear whether they can be rolled back if a mistake is made.

  • Supercharging Ansible Automation With AI

    Supercharging Ansible Automation With AI

    Whether the latest incarnations of generative AI will live up to their seemingly great promise remains to be seen. This does not always happen with new technology, but the torrent of AI advancements is promising disruption across nearly every industry. Instead of posing a threat to human roles, AI now aims to make tasks easier, faster and more efficient. “This is precisely what we have been trying to achieve with Ansible and automation through generative AI models,” according to Richard Henshall, senior manager of Ansible Product Management at Red Hat. “At the end of the day, AI is another form of automation, and while we do not have thinking computers yet, we do have highly advanced and powerful models that can automate access to data.”

    Case for Integration

    Picture two scenarios: Consider a user who is already skilled in using Ansible. Their limitation is often the amount of time it takes to find the right way to do something, as Ansible is very broad and encompasses many different technologies. The user may be an expert in one or a few of these technologies, but not all of them. In this case, AI can act as an assistant, helping the user find answers more quickly and efficiently. For example, if they want to automate something in AWS, the AI can help them find the necessary resources, and the user can then tweak and curate the automation to create a final product that can be distributed to a large number of people. “This essentially superpowers the individual user,” said Henshall.

    Another type of user may not know how to automate, but they do know how to use a specific technology. “They may not automate that technology because it’s difficult for them to learn something new,” Henshall added. “AI capabilities can help these users get started with automation more quickly, enabling more people to write automation and manage both new and existing technologies.”

    In both cases, the purpose of automation is to help users solve problems. AI serves as an accelerator for these capabilities. “Once users have created automation, the AI model can be trained on the organization’s DNA, including best practices, security posture, policies and enforcement points,” Henshall asserted. This allows the AI to become a co-worker, empowering users and helping them maintain and manage their technologies more effectively.

    “As the AI model learns from the organization and its users, it can help train junior engineers, who will eventually become senior engineers, continuing the cycle of knowledge transfer and improvement,” Henshall predicted. “In this way, generative AI technologies can be integrated into the Ansible ecosystem, benefiting users and organizations in a wide range of contexts.”

    Solving Problems With AI

    As the DevOps leader of cloud testing organization LambdaTest, Shahid Ali Khan has led the company through open-source engagements and investments in next-generation automation solutions, addressing key challenges with Ansible. “We are dealing with a very large scale inventory in terms of geologically separated data centers and multiple OS. Managing this inventory on the basis of these characteristics becomes a hefty task and requires continuous efforts from the team,” he said. “With AI, Ansible can analyze the inventory and automatically group the hosts based on their characteristics, such as their operating system or their geographical location. This makes it easier to manage the inventory and apply changes across multiple hosts at once.”

    Khan also highlights the issue of auto-healing in the industry. “Auto-healing has always been a problem, and while tools in the market have somewhat addressed it, they still require defining errors and categorizing fixes,” he said. “With current AI technology in place, Ansible can detect and automatically fix issues on the basis of historic data in the inventory without the need for manual intervention, reducing the risk of human error.”

    “Inventory forecasting has always been a pain in order to calculate how much inventory we will need based on usage and on what fronts like location and OS we need to scale,” Khan shared. “AI here with Ansible can help detect the usage and forecast based on exact demand.”

    Benefits and Challenges for the Ansible Community

    In the Ansible community, which consists of people who want to solve problems, the benefits and challenges of AI-driven automation are similar to those faced by individuals in their jobs, just on a different scale.

    Community members want to see how generative AI can help them accomplish more tasks, whether or not they are part of an enterprise. Having access to a community model allows them to contribute to training the AI, ensuring its usefulness, as they may eventually work in an enterprise setting during their careers. “The access to community services enables them to take advantage of the benefits offered by AI-driven automation,” Henshall remarked. “It is a mutually beneficial relationship between the community and AI-driven automation.”

  • Friend or Foe? ChatGPT’s Impact on Open Source Software

    Friend or Foe? ChatGPT’s Impact on Open Source Software

    OpenAI’s ChatGPT took the world by storm, amassing 100 million users in the first two months after its public launch. The continued interest in the tool has created a buzz among developers, especially those in the open source community. But this begs the question: How will ChatGPT impact open source software?

    Amid the excitement and promise that ChatGPT holds, many wonder whether the open source community, just like any other organization, should fear or embrace the technology. Certain concerns for open source contributors and developers are sparking debate as questions surrounding the origin of generated source code surface, and the possible ethical and legal implications are discussed. While valid, developers should not fear or stray away from using ChatGPT but rather shift their focus to understanding how to embrace it for positive outcomes.

    The Top Three Open Source ChatGPT Concerns

    Collaboration between contributors is the cornerstone of open source projects, which some fear will be disrupted with the increased use of AI-based tools. However, the level of synergy within the community cannot be easily taken away or replaced by tools. Instead, ChatGPT or other AI-based tools, including GitHub Copilot, allow developers to produce code more quickly and more efficiently.

    As developers increasingly use AI tooling to assist with new or enhanced code, project collaboration and oversight will help improve the AI-generated code.

    Debunking and navigating the top concerns will be imperative to take full advantage of the technology’s promise and potential.

    Validity. While the creation of code by ChatGPT has generated excitement among developers, critics argue that—without context—the validity of code can be questioned. Some open source developers worry that teams will begin to rely entirely on ChatGPT to generate code, but this concern fails to consider the humans involved in the process—today and far into the future.

    Developers don’t take ChatGPT’s output as the final word; rather, they use it as a baseline and jumping-off point to streamline their code. In fact, code is rarely created from scratch anymore. Developers rely on others’ source code. For example, code from Stack Overflow, GitHub and thousands of open source libraries that are available on public registries like npm, Maven, Nuget and PyPI. Bringing ChatGPT into the fold will not significantly change how developers source their code, but will improve development velocity, saving valuable time and associated cost.

    Training Data. Machine learning (ML) and deep learning (DL) model training must be fair, robust and explainable to avoid biases. If the data is wrong, the results will be as well. Garbage in, garbage out. Coming from ML/DL models, the code ChatGPT returns in response to prompts could raise concerns surrounding its accuracy. Like any other source (Stack Overflow, GitHub, etc.), ChatGPT’s code outputs are not guaranteed to be perfect, and developers must be mindful of this.

    However, there are more benefits of ChatGPT-trained models concerning code outputs. ChatGPT can also explain new or existing code, it can also effectively provide unit tests for code, which will help with writing better software faster.

    Ownership. Questions of ownership have surfaced regarding the use and distribution of the code that the AI tool generates. While the code that ChatGPT generates is the result of ML/DL inference from many sources, it’s the developer’s responsibility to use that code ethically and safely. It should be treated as any other public data or open source software, as not a final product and must be used in the context of your requirements. It’s also important to carefully review any code generated by ChatGPT and ensure that it doesn’t introduce vulnerabilities.

    Similar to GitHub Copilot, ChatGPT is trained by millions of lines of open source software. Code posted in prompts is likely to make it to models, too. Unlike copyrighted pieces of art and writing, the code that ChatGPT outputs should not be considered in a final state and should not be used in a way that is subject to licensing restrictions or legal implications that have dominated the conversations around its use.

    Ownership closely relates to the ethical dilemma of using AI-generated text, code and more. To address that, there are new tools that help detect whether AI generated the content and how much, which will be useful for educators concerned with students overusing the tool. This emerging area of checks and balances exemplifies the constantly evolving and improving technologies that allow humans, and in the case of open source, contributing developers to improve their craft and produce better open source software.

    ChatGPT’s Impact on Open Source Talent

    Discussions about whether ChatGPT will require new skills for existing positions or new jobs with specialized experts are ongoing. While new and exciting, ChatGPT won’t immediately create new or different jobs. Like any new tool introduced to developers, it takes time for developers to become familiar with the technology and understand how to use it best. ChatGPT is no different. For example, consider the previous push to use low-code/no-code technology. While this great technology is used to speed up the creation of apps and improve the usability for non-developers, it has taken time for organizations to properly use low-code/no-code technology. A similar perspective and trajectory will extend to the use of ChatGPT for all software development, including open source projects.

    In the coming weeks and months, it will be essential to encourage the open source community to embrace ChatGPT and explore its possibilities. Technology has already proven to be an effective educational tool. Consider asking ChatGPT for book recommendations about programming languages and coding; it delivers short descriptions for each book. Or, prompt it for the top takeaways from one specific book. By doing this, individuals can make learning so much easier. By learning from other developers and sharing resources, including results from ChatGPT, the community roots of open source will keep thriving.

    Human Involvement is Always Needed

    The public’s reaction to ChatGPT may be novel, but the idea behind the tool’s relationship to open source is not. Analyzing how ChatGPT works within the open source community reveals that it will be useful because it enhances the capabilities of developers, it doesn’t replace them.

    Whether it’s reviewing code, pair programming or learning from fellow developers, humans will not be replaced by generative AI, only enhanced. Leveraging the tool will decrease the time and effort needed to complete tasks, increasing developer quality and efficiency. The use of ChatGPT does not create new positions or overhaul current skill sets; it makes them better and empowers developers to expand their opportunities and enhance their abilities.

    Soon, ChatGPT and other AI tools will be standard practice within the open source community—or until a new technology is created and shakes up software development once again.

  • Linux Foundation Europe to Host RISE Open Source Project

    Linux Foundation Europe to Host RISE Open Source Project

    The Linux Foundation Europe, in collaboration with RISC-V International, announced today it will host a RISC-V Software Ecosystem (RISE) Project. Through the project, multiple organizations will contribute engineering and financial resources to harden critical open source software for use in commercial applications and enterprise IT environments.

    Founding members of the project include Andes, Google, Intel, Imagination Technologies, MediaTek, Nvidia, Qualcomm Technologies, Red Hat, Rivos, Samsung, SiFive, T-Head and Ventana.

    RISC-V International is defining a set of application processors that includes software development tools, virtualization support, language runtimes, Linux distribution integration and system firmware that is intended to be used primarily in edge computing environments.

    Amber Huffman, chair of the RISE Project, said at a minimum, each of the founding members contributing €80,000 and additional engineering resources. The RISE Project deliverables will be prioritized by the RISE Technical Steering Committee (TSC) as the project looks to address performance, security and reliability issues involving open source software, she added.

    The TSC will also be able to shift skills from one initiative to another as priorities continue to evolve, Huffman added.

    The overall goal is to use a more holistic approach to eliminate duplicate work that each member of the RISC-V community would otherwise do separately to create an application processor, noted Huffman.

    The Linux Foundation Europe is an arm of the Linux Foundation that has been created to host projects that are of particular interest to organizations investing in open source software based in Europe. The RISE project is being launched at a time when the European Union debates a proposed Cyber Resilience Act that requires organizations that sell hardware platforms that connect to the internet to ensure that the devices their software runs on comply with cybersecurity best practices.

    The Linux Foundation also provides the funding for  the Open Source Security Foundation (OpenSSF) that is marshaling resources to enable maintainers of open source software to address vulnerabilities and adopt DevSecOps best practices to better ensure the security of downstream applications that incorporate that code. In addition, the OpenSSF is building out an incident management capability that will assist maintainers in the event a zero-day vulnerability is discovered.

    It’s not clear when all these investments might eventually lead to higher-quality open source software. However, it’s clear that vendors that benefit from open source software are rallying to address a range of quality assurance issues. Less clear is the degree to which enterprise IT organizations that benefit from open source software are making similar commitments.

    In the meantime, it’s not likely any of these issues will have a material impact on organizations’ reliance on open source software. The benefits of reduced costs and increasingly faster pace of innovation enabled by open source software development far outweigh the risks. However, as cybercriminals become more adept at embedding malware within software supply chains, there’s a clear need for increased vigilance.

  • CDF Marries Emporous Repository to Ortelius Management Platform

    CDF Marries Emporous Repository to Ortelius Management Platform

    The Continuous Delivery Foundation (CDF) has added the open source Emporous repository for software artifacts to an Ortelius platform for managing the software artifacts that make up a microservice.

    Ortelius was originally created by DeployHub and subsequently contributed to the CDF, which also oversees the development of the open source Jenkins continuous integration/continuous delivery (CI/CD) platform.

    Emporous is based on a Universal Object Reference (UOR) conceptual model that enables dynamic schema registration and decentralized content-addressable reference of a wide spectrum of software artifacts. Originally developed by Red Hat, Emporous will now be managed as a sub-project within the larger Ortelius incubation-level project by the CDF.

    DeployHub CTO Steve Taylor said this integration would make it simpler to store, organize and search metadata related to software artifacts, along with the artifacts themselves, within the context of the Ortelius platform for managing microservices.

    Ortelius provides insights into the relationships and dependencies between different software components and applications. It does this by collecting and aggregating pipeline data from multiple sources within a software supply chain. Emporous will be incorporated within Ortelius to enhance metadata search capabilities and provide a single repository to store any type of artifact, from containers to jar files. It will now also be able to store metadata and also the artifacts themselves within the same Emporous catalog.

    Those capabilities will significantly improve security and compliance reporting across a software supply chain, added Taylor. DevOps and cybersecurity teams will be able to automatically apply and enforce supply chain policies and procedures, he said.

    DevOps teams need a way to aggregate all the data from the various tools they use and the data must be easily searchable. Armed with those insights, it should become simpler to surface insights into, for example, which components of an application contain known vulnerabilities.

    In the longer term, IT organizations will be able to apply machine learning algorithms and other forms of artificial intelligence (AI) to automatically surface insights that will enable DevOps teams to automate workflows, said Taylor.

    Microservices-based applications are considerably harder to build and manage than monolithic applications, but they are also easier to maintain because individual microservices can be ripped and replaced.

    Most microservices today are built using containers, but there are also microservices that are written in Java, for example. In some cases, developers view monolithic applications as being a large, less flexible microservice that they invoke via an application programming interface (API).

    There’s no doubt that the rise of microservices made application environments a lot more complex. The tools needed to simplify the management of the next generation of applications are finally starting to become more widely accessible. In theory, the number of organizations that can successfully build and deploy these types of applications should steadily increase in the months and years ahead.

    In the meantime, there’s clearly still plenty of work that needs to be done to improve the overall state of software delivery. A recent CDF survey found only 22% of respondents were using continuous integration and continuous delivery (CD) to automate the building, testing and deployment of code in a production environment. As CD becomes more distinct from CI, however, the number of organizations that can achieve that goal when deploying microservices-based applications should steadily increase.

  • OpenAI to go Open Source — Elon Musk was a ‘Huge Idiot’ ¦ Mojo Risin’

    OpenAI to go Open Source — Elon Musk was a ‘Huge Idiot’ ¦ Mojo Risin’

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Sources say it’s an open source future for OpenAI, and improving AI perf with Mojo. (more…)

  • Economy Forces Shifts in Tech Talent Hiring

    Economy Forces Shifts in Tech Talent Hiring

    A survey of 418 hiring managers published this week by the Linux Foundation found 59% of respondents changed or revised their technical hiring plans for 2023 because of economic uncertainty. Among those organizations, the primary strategy for addressing these concerns was to freeze hiring (34%), followed by a focus on hiring to address specific skills requirements (32%) and staff reductions (13%).

    Conducted by Linux Foundation Training & Certification and Linux Foundation Research to coincide with the Open Source Summit North America conference, the survey found 30% of respondents worked for organizations that froze hiring, while another 8% reduced headcount. Another 6% opted to do both, the survey found.

    In contrast, a total of 44% said they planned to increase their hiring of technical talent, with another 12% reporting that they would still be hiring even if they had frozen some hiring or reduced headcount.

    The survey revealed that organizations are looking to add technical staff with experience in cloud/container technologies (50%), cybersecurity (50%), artificial intelligence/machine learning (46%), database and data management (37%) and advanced analytics and data science (37%). The most acute need for technical expertise is in the areas of augmented reality/virtual reality, AI/ML, blockchain, continuous integration/continuous delivery (CI/CD) and DevOps and Kubernetes, the report found.

    Anna Hermansen, ecosystem manager for Linux Foundation Research, said in addition to looking to recruit talent, it’s also apparent more organizations are investing in training to fill gaps, with 70% of respondents reporting they invest in training whenever a new technology is introduced. Investments in upskilling often enable organizations to add critical capabilities faster than finding and then vetting potential candidates, she noted. Well over half of respondents (58%) worked for organizations that sought training for their existing employees when they were unable to find suitable technical candidates, the survey found. Only 38% of respondents said they would rely on consultants to fill those gaps.

    Upskilling, however, is not a panacea. While 55% of organizations agreed that upskilling could help, there is often a need to backfill positions and 47% of respondents said upskilling takes too long or is ineffective at training for complex roles.

    In comparison, 48% or respondents said it would take more than three months to fill a technical position, with nearly two-thirds (64%) also noting recruitment is costly and time-consuming and often does not lead to the right candidate. A full 84% agreed there is a need for a streamlined way to recruit new technical staff with proven skills and 64% said that onboarding takes valuable internal resources away from other critical projects for too long.

    The survey also found that 38% of organizations lost up to 10% of their new hires within four to six months, with nearly one out of three new hires, on average, departing within six months of being onboarded.

    There’s no doubt that an uncertain economic outlook is adding more stress to already-flawed recruitment processes. The challenge and the opportunity is determining whether to focus on recruiting top talent versus growing a farm team that will be able to step up to handle more complex tasks. It’s not a question of choosing between those approaches as much as it is striking the right balance between them.

  • AWS Converts Cedar Policy-as-Code Tool to Open Source Project

    AWS Converts Cedar Policy-as-Code Tool to Open Source Project

    At the Open Source Summit North America conference today, Amazon Web Services (AWS) announced it is making Cedar, a language for defining permissions as policies that includes automated reasoning to mathematically prove an IT environment is secure, available as an open source project.

    In addition, AWS launched SnapChange, an open source fuzz testing tool that cybersecurity researchers can employ to discover vulnerabilities by replaying physical memory snapshots in a KVM virtual machine. Fuzz testing uncovers software security issues by monitoring how the system behaves while processing random data.

    David Nalley, director of open source marketing for AWS, said both projects are part of an ongoing AWS effort to contribute intellectual property to the open source community that plays a pivotal role in the applications that organizations deploy on the AWS cloud.

    AWS has been employing Cedar to provide IT organizations with the ability to write authorization policies as code that can be deployed anywhere. It is most widely used within the managed AWS Verified Permissions service that manages those processes on behalf of organizations that deploy workloads on its cloud.

    Cedar differs from other policy-as-code approaches because it employs automated reasoning, a computer science discipline that is considered a subset of artificial intelligence (AI), to enable machines to prove theorems and checks. That latter capability is especially critical to enable organizations to mathematically prove that compliance mandates have been continuously met, noted Nalley.

    Otherwise, IT organizations become wholly dependent on an assessment of intermittent audits conducted by humans that are likely to arrive at varying subjective conclusions, he noted.

    The latest open source contributions from AWS come at a time when there is more scrutiny being applied to how open source software is developed and employed. The scrutiny increased in the wake of a series of vulnerability disclosures involving open source software, such as the Log4j monitoring software project.

    The maintainers of many open source software projects often don’t have the resources or capabilities required to immediately patch a vulnerability every time it is discovered. That’s become especially problematic, because more research into open source software vulnerabilities is being conducted; it’s only a matter of time before a vulnerability is detected in software that was largely created by developers that have limited cybersecurity expertise. Enterprises that consume open source software need to pay careful attention to what dependencies they are creating when that software gets incorporated into an application environment. A patch for a zero-day vulnerability that is discovered might not be readily available.

    Hopefully, consumers of open source software will become more involved in its development by contributing code or making available the financial resources that maintainers need to devote time and effort to creating a timely patch to a vulnerability. In the meantime, however, they should also be reviewing what open source software is included in applications running in production environments. Cybercriminals are increasingly looking to discover and exploit vulnerabilities in not just existing applications but not also in components they know are likely to be employed across a wide range of downstream applications.

    As such, finding ways to apply more cybersecurity expertise to application development by embracing DevSecOps practices has become an imperative rather than a recommendation.

  • Nixpkgs: How to Build Better, More Open Software

    Nixpkgs: How to Build Better, More Open Software

    The world we know today depends in fundamental ways on open source software. As providers of this software, package managers serve their respective ecosystems from their centralized, opaque hubs in ways that are sometimes difficult to reconcile with the open source nature of the software they’re serving. I want to talk about the extent to which that’s true and discuss Nixpkgs—the largest open source package repo in existence today—in this context.

    People sometimes focus on Nix the language or Nix the package manager, but it’s the Nixpkgs package collection (pronounced ‘nix packages’) that is core to the success of Nix. But it’s more than that: We can also understand Nixpkgs as a powerful tool to ensure the continued growth, transparency and resilience of open source software, first and foremost because it democratizes the process of curating this software and its build instructions. It is the living compendium of the efforts of thousands of contributors who both stand on the shoulders of giants (the software they’re packaging) and provide the ladder for others to climb further.

    A Purely Functional Grimoire

    The Nix package manager is different from other package managers in several ways, but one key aspect is that Nixpkgs contains only recipes to build packages, not the packages themselves. This is a fundamental idea to Nix: If we build a package reproducibly, we can separate the specification of a build from its built output. When we build a “recipe” for a package, we determine the output hash of that build–without actually building it. This means you don’t need a cache of build artifacts: Recipes are enough to rebuild your software world.

    Nixpkgs is just that: The big fat recipe book of open source. Anybody can view it, fork it, patch it and use Nix to inspect the largest set of free software recipes in the world. Anybody can create a cache that uses those build recipes (or your own!) to publish packages: The default cache is the official one run by the NixOS Foundation at cache.nixos.org.

    Derivation and Substitution

    So what happens, in practice, when I download a package from the Nix cache? When you query for a package, the CLI creates the specification (we call it a derivation) of the package to be built. This derivation includes a hash: The summary identifier of all source code, patches, build flags and hashes of the dependencies themselves.

    With that output hash in hand, Nix the package manager just asks whether that path exists in its cache. If it does, Nix fetches the contents of the package you asked for; if it doesn’t, Nix will build the package locally. We call this move substitution: Creating a derivation from a Nixpkgs expression gives you all the information you need about the output of the build, so the cache can substitute a binary with that output hash if we have it. Of course, if you end up having to build the package locally, it will result in the expected output hash anyway.

    Democratizing Package Curation

    The term ‘lingua franca’ refers to the dialect spoken by sailors across the Mediterranean in previous centuries, and it was the shared language with which one could communicate and do business from Morocco to Greece. Today we package software for independent islands, each with its own rules, languages and cultures and, until Nix, we had not found a way to “speak” across the many ports-of-call of our software archipelago. What this allowed for is packages being available on more platforms, which drew more developers in, which resulted in Nixpkgs becoming more complete, and so on in a positive feedback loop.

    Nix is a cross-platform, cross-architecture package manager. If we had to design the next-generation package manager, this would probably be at the top of the requirements list, yet few package managers enable this today. Each of the teams at RedHat or Debian maintains their own packaging format and package cache and has a fixed team of vetted contributors to keep packages secure and up-to-date. This results in few who effectively contribute to packaging open source projects and do so for one of many platforms.

    An open book of package recipes written in this lingua franca results in another surprising effect: Nixpkgs maintains more packages than anybody with an order of magnitude fewer maintainers [1]. It makes sense: Since it’s a common language, people can review the packaging code for packages they don’t necessarily know intimately, which in turn leads to contributors and reviewers cycling through updates faster and providing more up-to-date packages, which reinforces the previous feedback loop from earlier.

    Even if the rest of the Nix ecosystem stalled out, we’d still have this incredible resource in Nixpkgs to curate and share open source software; flox sources its packages exclusively from Nixpkgs for this (and many other) reasons. As of February 2023, there were around 6,000 contributors on the Nixpkgs repository and growing.

    (via (repology.org)[https://repology.org/repositories/graphs])

    Stay Unstable, my Friends

    Nixpkgs embodies a new model for package management and one that opens it up to more people to contribute. Providing cross-platform and cross-language packages is obviously attractive, but it’s the fact that it provides a lingua franca for software packaging that makes it a pillar of open source moving forward. Because Nixpkgs unifies a broader swath of software and makes it easy for anybody to contribute new packages and fix existing ones, it continues to grow and serve the open source community: A positive feedback loop propelling the existence of a more free and open world of software.

    References:

    Nixos.org

    https://reproducible-builds.org/

    https://wiki.debian.org/SecureApt

    https://rpm-packaging-guide.github.io/