Tag: docker

  • Cloud: Why Must We Be Confusing?

    Cloud: Why Must We Be Confusing?

    When I was a kid learning to play Dungeons & Dragons (D&D), the concept of level was never confusing—though the authors should have seen that it very well could be. For those unfamiliar, D&D has many uses of the word level—character level, caster level, spell level, dungeon level and, increasingly, as time went on, monster level. I can almost guarantee that I missed some uses of the word level in the game, but you get the idea.  In fact, if I missed your favorite, feel free to add a comment.

    We used context to determine what was meant, and we clipped along without any (game) world-shattering consequences. Though some people were horribly confused.

    (Side note: I worked with D&D co-creator Gary Gygax for a while and he knew about the over-use but didn’t feel that after D&D’s surge in popularity it could be fixed. His later RPG creations did seek more clarity and less word reuse, but most RPG systems available today still have many uses of the word level.)

    I am comfortable declaring that those of us who grew up on RPGs are now running vendor companies and doing analysis. Otherwise, why would we have so many uses for unique words? I mean, we no sooner got DevOps than we made a total mess about what, exactly, you meant when you said “CD”. Was it continuous deployment? Continuous delivery? Both? Similarly, we malformed continuous integration. Let’s call it CI—and then make an entire market around it really being continuous test while it is still used to refer to continuous integration, because that is not at all confusing.

    But then we jumped to cloud, and spent some time being really busy trying to confuse everyone about what “cloud” actually means. Is it only public cloud, or does it include private cloud? If private cloud, does that include virtual private cloud only, or does it include cloud in a co-host or data center? Is SaaS actually cloud, or is that a completely different beast?

    And that brings us to the reason I came here to write this particular blog post. We are now being barraged with “cloud” that includes Docker hosted on Kubernetes. I think of this as akin to one too many uses of the word “level”.

    Kubernetes is not cloud. It offers similar infrastructure in a smaller format, but the key to Kubernetes is that it supersedes cloud. Kubernetes doesn’t care about your cloud (or lack thereof) once it has been implemented. Cloud simply becomes infrastructure-as-a-service (IaaS) when Kubernetes is deployed. This is terribly important for a variety of reasons, not the least of which is that if you drop Kubernetes onto a cloud provider, you have two sets of infrastructure to lock down. Two sets that have a lot of internet exposure—because it’s not called “public cloud” for nothing.

    I do understand where referring to Docker hosted on Kubernetes as ‘cloud’ comes from; there are a ton of similarities. But Kubernetes needs a platform—hardware or virtual—to run on. And hand-waving the implications of that nested relationship is asking for trouble. And when cloud is either a thing of the past or so totally commoditized that people don’t think about it, Kubernetes or its successors will likely be the reason. So let’s take it easy re-using cloud too much.

    In the short term, though, my advice is that if someone in the industry mentions cloud in their product offering or analysis, ask which use of ‘level’ they mean. Because honestly, Kubernetes, if implemented on hardware or IaaS, can be picked up and moved to whereever it is needed. And that’s not “cloud”.

    Keep kicking rear. Doesn’t matter where your apps are running, it matters that you’re keeping them running and locked down. Keep it up, and call things what they are—container orchestrators are not cloud platforms, no matter which functionality they share. And to all my RPG peeps, keep having fun. There are always new adventures to be had… No matter the level.

    Image: Photo by S L on Unsplash.

  • Dagger: Standardizing CI/CD is the Holy Grail of DevOps

    Dagger: Standardizing CI/CD is the Holy Grail of DevOps

    Continuous integration and continuous delivery (CI/CD) has become a hallmark of quick software release life cycles. Nowadays, many teams support CI/CD pipelines for their software, offering a repeatable pathway to build, test and deploy code. A CI/CD pipeline is necessary to support an iterative, rapid deployment approach—yet, maintaining one is not an easy task.

    Enter Dagger. Dagger is a utility that aims to streamline how developers create and manage their CI/CD pipelines. Dagger, led by founders of Docker, recently unveiled their public beta launch along with a Series A $20 million funding raise led by Redpoint Ventures. While many open source CI/CD projects already exist on the market, Dagger is unique in offering easier configurations, pre-built components and more reusable actions.

    Akin to traditional supply chain management, software supply chain management is an increasingly complex endeavor involving many third-party dependencies. Development teams truly require a set of standard building blocks to piece it all together. For this reason, Solomon Hykes, co-founder and chief executive of Dagger, told SiliconANGLE that “solving this problem is the holy grail of DevOps.”

    Below, we’ll look at the common stressors inherent in creating and managing modern software delivery chains. We’ll also take a peek into the Dagger beta to see what it’s attempting to accomplish.

    Issues Facing CI/CD

    Software development supply chains are growing larger and larger to support a complex network of software architectures. But DevOps engineers, who must automate this chain, are drowning in tool sprawl—and stitching these components together to make custom pipelines can become a big bottleneck.

    Especially as an organization expands its distributed microservices architecture into the cloud, it can be tricky to scale a repeatable CI/CD pipeline across various department and disparate tech stacks. This often results in many bespoke CI/CD pipelines. Even within the same team, nuances between codebases can complicate each CI/CD process and require engineers to rewrite scripts.

    Simultaneously, organizations are under intense pressure to deliver to meet new digital innovation demands. But, until DevOps can make the release process more efficient, software development automation risks introducing more toil than it takes away.

    Diving Into Dagger

    So, how is Dagger alleviating these concerns? Well, Dagger creators described it as a “portable devkit for CI/CD pipelines.” Using Dagger, DevOps engineers can use a standard set of building blocks to assemble their CI/CD pipelines and then run them on their chosen cloud. It’s like a better developer experience for CI/CD pipeline creation.

    So, how does Dagger work under the hood? Dagger is powered by Buildkit, the Dockerfile-agnostic builder toolkit. Developers then use Cue to specify how their code is run in a container and to link up actions. Cue is an open source configuration language developed at Google.

    According to the docs, it takes about five minutes to install Dagger and the necessary dependencies. Dagger also provides boilerplate code to integrate with pre-existing CI environments. In essence, Dagger doesn’t need to replace your CI tool—it adds “a portable development layer on top of it.”

    Then, there are the actions themselves. Dagger supports a growing library of actions that it calls Universe. At the time of writing, the Universe of actions is limited and is being staged in the main Dagger repository. These actions are reusable Cue packages. For example, this package details how to run and deploy the Nginx web server in Cue:

    package nginx
    
    import (
        "universe.dagger.io/docker"
    )
    

    And then,

    #Build: {
        output: docker.#Image & _pull.image
    
        _pull: docker.#Pull
        *{
            flavor: "alpine"
            _pull: source: "index.docker.io/nginx:stable-alpine"
        } | {
            flavor: "debian"
            _pull: source: "index.docker.io/nginx:stable"
        }
    }
    

    Benefits

    CI/CD can certainly benefit from standard building blocks and more fluid wiring between tools. There are several potential benefits from Dagger’s proposition:

    • Unification: Unifying developer and CI environments could avoid headaches.
    • Localization: Dagger enables testing a closed-loop pipeline locally. This helps for debugging.
    • Reusability: Building blocks help avoid rewriting automation scripts repeatedly.
    • Multi-cloud: Easier ability to migrate a pipeline from one cloud to the other, thus avoiding vendor lock-in.
    • Compatability: Dagger can run in any environment compatible with Docker, meaning it’s probably workable with your current toolset.

    Final Thoughts: Lego Blocks For CI/CD

    Docker revolutionized the software industry by easing the process of managing containerized applications. And now, Docker founders Solomon Hykes, Sam Alba, Andrea Luzzardi and others at Dagger are tackling another meaningful problem within today’s complex software ecosystem. Given the success of their previous endeavors, it’s not too difficult to imagine a network of reusable CI configurations emerging with an excitement similar to Docker Hub.

    Of course, Dagger is still in its early stages and there’s much development ahead. It will undoubtedly be an interesting project to follow in the years to come. Dagger is being built as an open source project on GitHub. The company has also opened a Discord channel to engage with the community.

  • DevOps and Containers

    DevOps and Containers

    “Works on my machine!”

    We’ve all had that co-worker. I have had one at pretty much every company I’ve worked for. At several jobs, it devolved into a running joke.

    There is a belief that containers alleviate this issue. And they may—or they may simply mask the issue. The choice is up to IT.

    Inevitably, something that works on one machine and not another can be traced back to on-the-fly configuration changes. If “we use containers for DevOps” means that developers have full control of the container environment, then it must also mean that any configuration changes be codified into a documented container image that can be spun up repeatedly and even recreated in a Dockerfile if needed.

    If, instead, it means that devs have a playground in containers and, once they get it going, they can chuck it into production—even if you don’t say it that bluntly—then you are making things worse with containers, not better. At least in the days of “works on my machine,” the issues had to be resolved because the only time you heard that phrase was if something didn’t work on the machines it needed to be deployed to!

    As a developer, it is tempting in both container and virtual environments to upgrade a package or tweak a setting without documenting it. But a clean DevOps environment needs to be able to build the product from Dockerfiles, not from a pre-existing image. Yes, you can just clone the existing image, but then there is no documentation of what has actually been changed and it takes a lot more work for those following you and debugging future problems.

    So take the time; keep an up-to-date Dockerfile (or whatever equivalent script is used to build your containers). It is not that significant an investment to stop for a minute, open a file and type a few lines. And it pays off in spades if, in the future, someone is trying to figure out what the heck is going on in there. Consider it self-documenting configuration. I’m not a fan of “self-documenting code,” but for config, it is pretty clear if you go, “Oh, we needed version 10.2, not version 10.1; let me update the init script.”

    There are a fair number of shops out there that modify things running in production, and the world hasn’t ended yet—though that may well be how it happens. So, a good practice that will help those trying to follow behind you is clearly not a necessity. But each of us has gone back after a year and said, “Why did we do that this way?” which was (or wasn’t) then answered with, “Oh, here is exactly how the environment was configured.” But it might help answer the ‘why.’

    Your apps are powering the world—or your specific apps are powering your little piece of it. And you’re kicking it just keeping them running. Now, go make them better! Consolidate changes, manage configuration through Dockerfiles, Ansible, init-scripts, whatever combination you use … And keep rocking.

  • Docker Expands Trusted Content Offerings for Developers

    New collaborations with AWS, Datadog, Mirantis, Red Hat, VMware and other industry leaders expand access to trusted application building blocks to more than eight million registered Docker developers

    Palo Alto, CA, May 27, 2021 – Docker, Inc.™, today announced an expansion of the company’s trusted-content offerings for software developers. Among today’s announcements is the launch of the Docker Verified Publisher program, which provides access to Docker differentiated and trusted content that developers can leverage as reliable building blocks for their applications. Datadog, Red Hat and VMware are the latest three publishers to join a program that has seen 100 companies join in the last six months. Adding to trusted content distributed through Docker, the company also announced the upcoming availability of Docker Official Images into public and private registries from Amazon Web Services, Inc. (AWS) and Mirantis. In a separate press release, Docker announced updates and enhancements to the Docker Collaborative Application Development Platform.

    “We are thrilled to announce the Docker Verified Publisher program’s availability to even more publishers and the distribution of Docker Official Images to even more developers through even more registries,” said Docker CEO Scott Johnston. “This greatly expands choice for developers to complement Docker Official Images and solidifies the Docker platform and Docker Hub as the de facto standard for trusted, secure container images.”

    Secure software supply chains are becoming increasingly important to development teams as they work to lower the risk of exposure to malicious content while they build applications. Using reliable content at every stage ensures applications are secure and minimizes time and money spent on resolving security issues. With Docker, developers no longer need to worry about the source of their images. Instead they can quickly and confidently discover and use images in their applications from known, trusted sources. Docker Verified Publisher offerings include popular developer components such as Bitnami and Spring software from VMWare, RedHat Universal Base Images (UBI), Canonical Ubuntu and more.

    Leveraging the success of Docker Official Images and the existing Docker Open Source program, the Docker Verified Publisher program extends to commercial content incorporating trusted solutions for developers that are part of the application delivery pipeline. With the Docker Verified Publisher program, developers can build, share and run applications quickly with confidence — knowing that each component is secure at every stage of the software supply chain.

    Millions of developers rely on Docker Hub as their primary source for trusted content, including Docker Official Images and Docker Verified Publisher images. Developers will soon have the flexibility to download these images from their choice of registry including Amazon Elastic Container Registry Public (Amazon ECR Public) and Mirantis Secure Registry. This offers developers the ultimate flexibility to benefit from the trusted content provided by Docker using the registry of their preference.

    With over 200 ISVs using Docker Hub to distribute their software to developers, users who see the Docker “Verified Publisher” badge know that the images they are pulling come straight from, and are supported by, the publisher. With 13 billion image pulls per month from nearly 8 million repositories by more than 13 million developers, Docker Hub is the industry’s leading public container registry. Docker Hub delivers developers the largest breadth and depth of container images and plays a central role in building and sharing cloud-native applications.

    Trusted Content Powered By Docker

    Nick Coult, General Manager, Amazon ECR & Amazon ECR Public, AWS

    “Our customers expect the highest levels of availability and performance for their container applications, and a container registry is one of the key elements of performance and availability for many customers adopting containers. With Amazon ECR and Amazon ECR Public, AWS customers can rely on a highly available, high performance container artifact registry that works with their container applications. And soon, AWS customers using the Docker Official Images can take advantage of the high availability and fast downloads of Amazon ECR Public.”

    Michael Gerstenhaber, Senior Director of Product Management, Datadog

    “More than half of applications run on containerized infrastructure, and Docker Hub is the primary source for container images, according to our published studies. It is critical that we provide a secure and robust source for our images, and we are excited to be recognized as a Docker Verified Publisher. You can find any Datadog image to start securely monitoring the performance of your infrastructure and applications.”

    Adam Parco, CTO, Mirantis

    “Developers are increasingly reliant on container images to quickly deploy cloud native applications. We’re excited to have our Mirantis Secure Registry be part of the Docker Verified Publisher. This initiative lowers the risk of exposure to malicious content and gives developers confidence by ensuring applications are secure and minimizing the time and money spent on resolving security issues.”

    Mithun Dhar, Vice President and General Manager, Developer Business Unit, Red Hat

    “With cloud-native applications being deployed into production across industries, we see development teams increasingly looking for container content that is based on trusted, reliable and verified code. Built from Red Hat Enterprise Linux, Red Hat Universal Base Images provide the foundation to create the next wave of enterprise-ready cloud-native applications, and we’re pleased to be able to bring UBI to Docker Hub users as part of Docker’s Verified Publisher program.”

    Ashok Aletty, Vice President of Engineering, VMware

    “VMware is pleased to join the Docker Verified Publisher’s program. This provides developers unrestricted access to our artifacts, and allows them to safely adopt the popular open-source technologies we’ve made available. We are excited that VMware Tanzu customers, in particular, will benefit from a wider range of complementary services they can leverage as they quickly get apps to market.”

    Join the Docker Verified Publisher program

    Software publishers and ISVs interested in joining the Docker Verified Publisher program can get more information by visiting https://www.docker.com/partners.

    DockerCon Live 2021

    Join us for DockerCon Live 2021 on Thursday, May 27. DockerCon Live is a free, one day virtual event that is a unique experience for developers and development teams who are building the next generation of modern applications. If you want to learn about how to go from code to cloud fast and how to solve your development challenges, DockerCon Live 2021 offers engaging live content to help you build, share and run your applications. Register today at https://dockr.ly/2PSJ7vn

    Resources:

    About Docker

    Docker helps millions of developers efficiently and collaboratively build, share and run applications. The Docker collaborative application development platform provides developers with an unmatched experience for an integrated, reliable and secure workflow that accelerates app delivery from code to the cloud. Through a combination of the world’s largest marketplace of components and integrations with leading tools, Docker allows teams to rapidly create innovative applications. For more information, visit www.docker.com.