devopsy.kinsta.cloud

Tag: containers

  • Containers, Service Mesh and API Gateways: It Starts at the Edge

    Containers, Service Mesh and API Gateways: It Starts at the Edge

    Anyone embracing container technology such as Docker or Kubernetes has no doubt heard about the associated next big thing: service mesh, which promises to homogenize internal network communication between microservices and provide cross-cutting nonfunctional concerns such as observability and fault-tolerance. However, the underlying proxy technology that powers a service mesh also can provide a lot of value at the edge of your systems—particularly within an API gateway.

    Although it may appear that service mesh technologies have suddenly sprung up overnight, the reality is that many organizations have been using what we now identify as a service mesh for quite some time (including Verizon, eBay and Facebook). One such organization is Lyft, a U.S.-based ride-sharing service with a $1 billion annual revenue. Lyft also happens to be the creators of the open source Envoy proxy that is powering a lot of development in the service mesh space, such as the Kubernetes-native Istio control plane and Ambassador API gateway.

    The State of SOA Networking

    In a talk last year, Matt Klein, one of the creators of the Envoy Proxy, described the state of service-oriented architecture (SOA) and microservice networking in 2013 as “a really big and confusing mess.” Debugging was difficult or impossible, with each application exposing different statistics and logging and providing no ability to trace how requests were handled throughout the entire services call stack that took part in generating a response. There was also limited visibility into infrastructure components such as hosted load balancers, caches and network topologies.

    “It’s a lot of pain,” he said. “I think most companies and most organizations know that SOA [microservices] is kind of the future and that there’s a lot of agility that comes from actually doing it, but on a rubber meets the road kind of day in and day out basis, people are feeling a lot of hurt. That hurt is mostly around debugging.”

    Maintaining reliability and high-availability of distributed web-based applications was a core challenge for large-scale organizations. Solutions to the challenges frequently included either multiple or partial implementations of retry logic, timeouts, rate limiting and circuit-breaking. Many custom and open source solutions used a language-specific (and potentially even framework-specific) solution that meant engineers inadvertently locked themselves into a technology stack “essentially forever.” Klein and his team at Lyft thought there must be a better way. Ultimately, the Envoy proxy project was created to be this better way.

    Working Outside-In: Edge Proxy Benefits

    Although the open source release of the Envoy Proxy project made Klein and the Lyft engineering team look like an overnight success in September 2016, the reality was that the journey was filled with challenges over the four years from the initial hybrid-SOA Lyft architecture to their current microservice- and service mesh-enabled system. In a more recent talk at the 2017 Microservices Practitioner Virtual Summit, Klein talked about the essential need for—and associated challenges of—demonstrating business value for a technology-focused migration toward a service mesh network topology.

    Klein’s first piece of hard-won advice was, “Start with [an] edge proxy.” Microservice-based web applications need edge reverse proxying to avoid both the exposure of the internal business service interfaces (which would violate the principle of loose coupling) and the high operational overhead of exposing each service via an independent URI or RPC endpoint. Existing cloud offerings are “not so good” in this edge proxy or gateway space, or are presented to engineers as a potentially confusing range of differing products. Instead, Klein recommended starting an implementation of modern proxying technology at the edge, as this provides business value in the form of improved observability, load balancing and dynamic routing. Once an engineering team has understood how to operate proxy technology at the edge, the benefits can be rolled inward toward ultimately creating an internal service mesh.

    Evolution of the Edge: From Proxies to API Gateways

    AppDirect, an end-to-end commerce platform for managing cloud-based product and services with an estimated $50 million annual revenue, has undertaken a similar journey to Lyft, as highlighted in a recent blog post, “Evolution of the AppDirect Kubernetes Network Infrastructure.” The dynamic and ephemeral nature of cloud technology and container orchestrators such as Kubernetes, which provide many benefits in terms of scalability and resilience, mean that they are additional challenges for exposing public endpoints for business functionality provided via a composite of microservices.

    The AppDirect engineering team took a measured approach to solving these challenges, beginning with making core parts of the configuration static (such as service ports exposed) and placing a load balancer in front of each application. Their second iteration embraced more dynamism using HashiCorp’s Consul distributed key/value store and the HAProxy reverse proxy that supported “hot reloading” of configuration as it changed at runtime. Ultimately, however, the team were keen to leverage the richer functionality provided by a more fully featured API gateway.

    “The goal of our API gateway was to leave the exposed public APIs untouched and accessible even to legacy URLs and partner-customized domains, yet allow us to grow by ‘injecting’ and replacing old components one by one,” according to the blog post.

    After evaluating a series of open source and commercial offerings the AppDirect team deployed the Kubernetes-native Ambassador API gateway, which builds upon the Envoy proxy:

    “Relying entirely on the Kubernetes native API—which we know and love—Ambassador is lightweight, stateless, and uses no external datastore. Ambassador exclusively makes use of Kubernetes annotations to drive the active route configuration (i.e., it is the control plane of Envoy’s data plane),” the team noted in the blog post.

    Although AppDirect has not fully implemented a service mesh for internal communication, the company is already learning about the benefits of technology like the Envoy proxy and, critically, how to handle deployments of this in production.

    Making Sense of it All

    The adoption of service mesh technology within cloud native implementations and migrations is only just beginning, but it is already possible to identify that this technology fills a gap currently identified with modern container-based application platforms such as Kubernetes. All of the benefits of a service mesh—such as rate limiting, circuit-breaking and observability—also can be leveraged at the edge of systems. If you want to explore and learn about this technology, starting at the edge of your systems and working inward can be an effective strategy. This can also allow the technology to demonstrate value, such as improved observability and resilience, earlier than attempting to work inside out.

    — Daniel Bryant

  • Jump-Start Your Kubernetes Journey with JenkinsX

    Jump-Start Your Kubernetes Journey with JenkinsX

    Containers are very quickly being adopted as the stable way of deploying services. More and more organizations are adopting immutable architecture and containers provide an excellent way to achieve that.

    But along with the stability comes the complexity of deployment, along with managing workload, scaling and updating. That’s in addition to having to create a reliable and repeatable process to continuously build and ship these containers to a production ready state. Kubernetes is one option: It provides features to reliably manage production workloads for your services.

    So now developers have to learn Kubernetes, as well as how to prepare their application for Kubernetes and create the pipeline for continuous integration and continuous deployment (CI/CD). Developers also must determine what is the best approach to get this done and where to start.

    JenkinsX: CI/CD for Kubernetes

    JenkinsX is an open source CI/CD solution for the modern cloud platform for developing and managing applications on Kubernetes. It’s the automation that helps an application move to Kubernetes, deploying various services on Kubernetes to manage an application’s CI/CD journey.

    Developers simply focus on their product and its features and JenkinsX will manage the Kubernetes deployment. It’s opinionated yet flexible, and provides extensive customization. Users can create a Kubernetes cluster using JenkinsX on AWS, Azure, Google or even locally on minikube or minishift. Developers who already have their own Kubernetes cluster and its RBAC (Role Based Access Control) enabled can simply install JenkinsX on top of it.

    JenkinsX provides various templates, and, since it’s open source, more templates are added daily. Developers can do a quickstart with a templated version of spring boot application, a golang project, node, open-liberty, python-http and more. It will create a git repo for a template application and then connect to developer’s github account to push this template to a repo. Also, it sets up the required webhooks to talk to Jenkins, which is installed on Kubernetes as part of the JenkinsX setup.

    JenkinsX also creates namespace environments in Kubernetes. By default, it will create a staging and production environment that can be modified.

    Once changes are made to a templated quickstart application and a pull request is created, the process kicks in and pushes the payload using webhooks to jenkins. The pipeline jobs then kick in to start the build process.

    After the application builds in its own container, the next stage begins to create a docker image and push it to the internal docker registry. Helm charts are generated for the application and stored in the internal hosted chart museum. It is then deployed to a preview environment and developers can view or run tests against the application even before the pull requests are merged.

    As the pull request is reviewed and merged to the master branch, a job kicks in to deploy that application on to the environments. JenkinX implements GitOps, so any and every change is triggered from the github repository. It creates a pull request to the environment repository and adds a change to deploy this application in this environment., validates the pull request and automatically merges it, kick-starting the final deployment stage.

    An environment is mapped to a specific namespace, the application container is then deployed in that namespace, a service is created and all the required mappings are configured. Also, a URL is spit out for developers to hit their application.

    Developers then push their changes, grab a coffee (this step is optional) and few minutes later they get the URL to access the application in that environment.

    That’s the standard workflow. As you might have noticed, JenkinsX is very different from the traditional Jenkins approach. Jenkins provides complete flexibility, whereas JenkinsX provides an opinionated approach. Given everything is in the template repo it creates, you can always modify them to your requirements.

    A Jenkinsfile is created for the pipeline jobs that can be edited to add another stage. Helm charts are created and can be updated if required. Users can disable the default environment it creates the first time and have their own set of persistent or disposable environments.

    This is a great approach toward automating of various different tools we use today to build and deploy our applications on Kubernetes. There are a few features I would love to see added, such as different deployment strategies, support for internal docker registries and maintenance features to clean up obsolete images. That said, JenkinsX removes much of the learning curve and jump-starts your Kubernetes and container journey.

    — Saurav Sen

  • Reducing Risk in Applications Using Docker Containers

    Reducing Risk in Applications Using Docker Containers

    Docker. It seems like in this day and age you are either using Docker containers or you are going to use Docker containers. If you haven’t jumped on the bandwagon yet, check out a previous article, “Docker: The New Ordinary.” If you are on the wagon or are thinking about it but have concerns about their security, it’s time to read on.

    José Manuel Ortega (jmortega.github.io ) is a software engineer and security researcher in Spain. I recently watched his presentation online titled, “Testing Docker Images Security.” He gave an overview of typical Docker deployments, explained the attack surface and threats, presented how to detect vulnerabilities and outlined a couple of best practices. In short, his advice will help you learn how to better secure your Docker containers.

    (New to Docker? Read this paragraph; all others skip ahead.)  If you aren’t sure what Docker is, José offered this explanation: “Docker containers wrap a piece of software in a complete file system that contains everything it needs to run: code, runtime, system tools, system libraries—anything you can install on a server, regardless of the environment it is running in.”

    That is, containers are isolated but share an operating system and, where appropriate, binaries and libraries. Docker provides an additional layer of isolation, making your infrastructure safer by default. This makes the application life cycle faster and easier to configure, reducing risks in your applications.

    For starters, José laid out Docker’s default mechanisms for security:

    • Linux kernel namespaces
    • Linux Control Groups (cgroups)
    • The Docker daemon
    • Linux capabilities (libcap)
    • Linux security mechanisms like AppArmor or SELinux

     

    Screen Shot 2018-04-08 at 7.17.58 AM

    José walked through others tools, add-ons, best practices, etc., to increase Docker container security. I will cover most of them here.

    Docker Inspect Tool. The Docker Inspect Tool is built into Docker. It provides information about the host name, the ID of the image, etc. and it comes up when you start Docker.

    Docker Content Trust. It protects against untrusted images. It can enable signing checks on every managed host, guarantee integrity of your images when pulled and provide trust from publisher to consumer.

    Screen Shot 2018-04-08 at 7.19.01 AM

    Docker File Security. Docker files build Docker containers. They should not write secrets, such as users and passwords. You should remove unnecessary setuid and setgid permissions, download packages securely using GPG and certificates, and try to restrict an image or container to one service.

    Container Security. Docker security is about limiting and controlling the attack surface on the kernel. Don’t run your applications as root in containers, and create specific users for testing and policing the Docker image. Run filesystems as read-only so attackers can not overwrite data or save malicious scripts to the image.

    José provided a useful checklist to check the security of a Docker container, but it’s not a short one. Remember, if you are going to deploy hundreds or thousands of these containers, you’ll want to ensure consistent handling of security concerns to keep the hackers at bay:

      • Do not write secrets to Docker files
      • Create a user
      • Follow version pinning for base images, packages, etc.
      • Remove unnecessary setuid, setgid permissions
      • Do not write any kind of update instructions alone in a Docker file
      • Download packages securely
      • Do not download unnecessary packages
      • Use COPY instead of ADD
      • Use the HEALTHCHECK command
      • Use gosu instead of sudo whenever possible
      • Use no-cache (if applicable) when building
      • Enable Docker Content Trust
      • Ensure images are free from known vulnerabilities
      • Ensure images are scanned frequently throughout your DevOps pipeline
      • Ensure your images, packages are up to date
      • Use file monitoring solutions for image layers (if required)

    Auditing Docker Images. You can scan your images for known vulnerabilities with a wide variety of commercial and open source tools such as:

     

    All of these solutions can be integrated in one element of your CI/CD pipelines — some can be integrated in multiple places.

    José explored these solutions and best practices in more detail and offered up technical implementation tips in his full talk, available for free here. If you are working on or interested in Docker security, his talk is worth your time.

    Thirsty for more DevOps discussions led by practitioners (not vendors)?  You too can binge watch 99 other talks from the All Day DevOps conference here.

    — Derek E. Weeks

  • Shippable Extends Scope and Reach of CI Service

    Shippable Extends Scope and Reach of CI Service

    One of the attributes of building a continuous integration (CI) platform on top of Docker containers is that it’s highly portable. Shippable is making good on that promise by delivering versions of its namesake CI platform that can be employed on Mac OS X, CentOS and Windows platforms as well as to the Ubuntu distribution of Linux.

    In addition, Shippable is make available a beta release of an analytics module that promises to improve overall DevOps efficiency.

    Manisha Sahasrabudhe, vice president of product management for Shippable, said the company’s namesake CI platform for managing application release cycles now supports every type of platform commonly employed in a DevOps environment.

    DevOps teams can now run jobs across multiple operating systems that share access to the same GitHub or Bitbucket repositories. The latest release of Shippable also provides those teams with the ability to granularly allocate a pool of available resources for each job running, Sahasrabudhe said.

    Other new capabilities include support for custom scripts and images, as well as the ability to run tasks inside or out of a container.

    Finally, Sahasrabudhe said DevOps teams can now also make of an analytics add-on module that DevOps teams can employ to identify bottlenecks, for example, and track various key performance indicators. The analytics module is intended to help organizations move more rapidly down the DevOps maturity curve.

    Individual organizations are move down that maturity rate at significantly different paces. But when it comes to DevOps, Sahasrabudhe noted the larger the organization is the more chaotic IT tends to be. Because Shippable is made available as a software-as-service (SaaS) application, the company is in a unique position to enable organizations to compare how they are performing various tasks compared to the rest of the Shippable customer base. Those same organizations are also keenly interested in what’s holding up application development. They may not necessarily be able to eliminate a bottleneck, but deeper insights into the overall process should enable the organization to plan better, Sahasrabudhe said.

    For example, organizations typically have a choice of programming tools at their disposal. Analytics could be applied to inform them as to how well any specific programming tool is being used to build specific applications by not only that organization, but also the entire Shippable customer base in aggregate, Sahasrabudhe said. Further on, Shippable also expects to be able to apply both machine and deep learning algorithms to the data its collects to provide even more predictive analytics, he said.

    One of the primary reasons many organizations are starting to shift all kinds of management platforms into the cloud is to create a central repository of knowledge that can be easily accessed from anywhere in the globe. Most application development projects today are highly distributed endeavors. Hosting the tools needed to manage those projects enables developers to continue working locally on their own code, while still giving the organizations insights and control over the overall project.

    Analytics provided via the cloud is not a substitute for a project manager. But it does provide a means for everyone across the organization, including business leaders, to have a much firmer grasp of the realities of modern application development.

    — Mike Vizard

  • How to Isolate Development Environments: Vagrant vs. Docker

    How to Isolate Development Environments: Vagrant vs. Docker

    “It works on my machine.” This line has become an excuse often overused by developers to escape the burden of having to investigate problems experienced by end users that they are unable to replicate on their development machines. While this may excuse developers—after all, they didn’t experience the problem during development and therefore couldn’t have anticipated it happening to some users—it doesn’t solve the problem.

    Sometimes, issues are caused by end user mistakes, such as skipping a step in an installation process. Often, however, the problem arises from some disparity in the development and server or user environments. With the vast assortment of machine configurations in use—different operating systems, hardware specifications and software versions—even the slightest differences in development and production device configurations can cause dramatic differences in the behavior of many applications.

    This can be a serious issue, as it requires members of the development and operations team to identify the root cause of the problem, which potentially introduces unexpected and often significant delays in deployment and production schedules. Especially in the DevOps culture where rapid development, frequent deployment and highly dependable releases are of highest priority, such unexpected delays are completely unwelcome.

    Thankfully, there are a number of tools that have emerged to address this very problem. In this article, we take a close look at two popular tools with two different approaches to solving the problem of environment disparity: Vagrant and Docker. Like many DevOps tools and technologies such as continuous integration, these tools work well with all types of project setups, including IT outstaffing (dedicated development teams). Let’s compare these two solutions and see which types of scenarios each one is best fit for.

    Vagrant vs. Docker

    Vagrant offers to solve the aforementioned problem by enabling developers to easily create and manage virtual machines (VMs) in which they can develop, test and deploy applications. By using a VM, a developer or DevOps engineer is assured that as long as the application is run within the virtual machine, its behavior will be the same regardless of the actual hardware and operating system of the computer that the VM is running on.

    Docker offers a similar solution, but uses a different technology. Instead of using virtual machines, it uses containers, which are similar to Linux Containers (LXC) that are included in Linux systems. Containers serve a similar purpose as virtual machines, but at a much less extensive level and, therefore, are much more lightweight. Rather than isolating an application in its own virtual computer with a full operating system, containers simply isolate an application inside an environment with the minimum components necessary to run the application. This includes system libraries and tools, settings and configuration files and a runtime environment. Compared to virtual machines, containers consume much less resources from the actual machine on which it is being hosted. Docker has a comprehensive documentation library that includes information on what Docker and Docker containers are, how to install Docker and how to execute a Docker run.

    Vagrant and Docker share some common goals. For instance, in software development, both tools enable a more streamlined and consistent software development and deployment workflow by eliminating the need to worry about the application behaving differently in different machines. Moreover, it offers some degree of security to the host machine: Since the application is isolated from the rest of the system, issues caused by one application should remain within the scope of its environment and should not break the rest of the system.

    There are, however, a number of important differences between Vagrant and Docker that make them suitable for different types of applications. Let’s examine more closely how these two differ.

    Scope of Isolation

    While both Vagrant and Docker wrap applications into isolated environments, they differ in the degree to which these applications are isolated—or, effectively, the scope of the isolated environments. For Vagrant, the scope is the entire operating system: An application can access any and all components of the full virtualized machine and operating system it is being run in. For Docker, on the other hand, the scope is much narrower, as an isolated environment is limited to a single application or service: Each container is created specifically for an application. Effectively, this implies that with Vagrant, a single isolated environment can run multiple applications at once, while with Docker, running multiple applications will typically require the use of a separate isolated environment for each one.

    Resource Consumption

    Since Vagrant virtualizes an entire machine with a full operating system, it naturally has a high overhead cost and consumes a lot of resources from the host machine. With Docker, on the other hand, an app within a container shares the same kernel—the core module or component of an operating system—with both apps in other containers and the host machine itself. This enables containers to have much less overhead and subsequently consume much less resources from the host machines than do virtual machines. In effect, a host machine running apps with containers can run up to three times as many applications as it can with virtual machines. Moreover, since launching apps within containers won’t necessitate having to wait for the virtual OS to boot up, launching apps can also be substantially faster with Docker than it is with Vagrant.

    Limitations

    Vagrant and Docker also have their own limitations. Docker, for instance, is currently natively available to two operating systems: Linux, where it uses the Linux Kernel, and Windows, where it uses the Windows kernel. Docker is also available for MacOS, although it runs Docker containers on top of a Linux virtual machine. Moreover, Docker has some security issues—its use of containers make it less secure than Vagrant. Since applications within Docker containers share the same kernel with each other and with the host machine, it’s possible for a malicious program within a Docker container to affect other containers and even harm the host machine itself.

    While Vagrant, which uses virtual environments, is much more secure in this regard, it is limited in that it is meant only as a tool for development and generally not recommended for use in production environments. In other words, it is meant to provide a convenient way to virtualize the environment of the server to which an application is eventually going to be deployed, but once in production, the application must be run directly on the host machine. On the other hand, Docker provides a consistent environment for applications throughout the entire software development workflow, from creation to production.

    Conclusion

    Being able to isolate applications into their own controlled environments has proven to be extremely useful for software development, as it eliminates many hardware and software incompatibility problems that applications may run into when executed in different machines. This is particularly useful in DevOps environments, as it streamlines the whole software development workflow by eliminating sources of unexpected problems, thereby potentially shortening development cycles.

    We have looked into two popular tools for environment isolation, Vagrant and Docker. Both isolate applications from the host machine, although to varying degrees and using different technologies. Vagrant uses virtual machines, which can support multiple applications at once, but can be heavy on resource consumption and is only recommended for use during development. Docker, on the other hand, uses containers, which support only single applications, but is much lighter in terms of resource consumption and can be used both during development and during production.

    As such, these two tools cater to different business needs. If you require the development of a complex system of multiple applications interacting with each other, using Vagrant may be beneficial, but if you’re aiming to create a single portable application and want to minimize resource consumption, then Docker is the way to go.

    — Myron Monets

  • Persistent Storage: You’ll be Hearing More

    Persistent Storage: You’ll be Hearing More

    As containers take over the IT environment, a problem that has existed since virtual machines (VMs) became popular has become more urgent: how to keep persistent storage, and keep it consistent.

    Just as a business designed for foot traffic of 50 to 100 people per hour does not work well when it sees 500 to 1,000 people per hour (think Black Friday in most places these days), storage designed to hold data that persists across instances and even installations often struggles in containerized environments.

    There are a couple of reasons why this is so.

    First, we now treat our servers as disposable resources. If one dies, we leave it dead and simply spin up another. Normal processing keeps the dead server for later evaluation, but many shops do not follow this best practice. The issue has to do with servers that die when writing to persistent storage—what was the state of the write? Did it get interrupted mid-transfer? Luckily, applications can be designed to account for this, and the increasing prevalence of transactional processing in applications that are not based on a database management system (DBMS) shows that people are doing just that.

    Second, the number of server instances hitting those persistent stores is higher in a containerized environment—they are smaller, lighter and faster to spin up and their container density is generally higher than VM density. Contention issues go from zero to full-on when an app goes from “one server is writing to this persistent store” to “two servers are writing to this persistent store.” And often there are hundreds of servers writing to the persistent store. As mentioned, the stop-gap is to implement some form of transactional processing to prevent writes and overwrites.

    Persistent Storage: It’s Got Issues

    The problem is, we end up with transactional processing written into dozens or hundreds of server instances to protect a small number of persistent disks. This is not optimal, and at a minimum increases software complexity.

    That’s why you’ll be hearing more. Here are a few places working on simplified but powerful persistence ideas:

    {code} Rex Ray

    Originally part of EMC, {code} is Dell’s open source group and it has led the charge on the container storage interface (CSI) that facilitates persistent storage. Examples the company has implemented include databases and Minecraft, but the technology works for anything that must be stored on disk.

    Container Orchestration Engines and Cloud Providers

    All of the well-known container orchestration engines (COE) and infrastructure-as-a-service (IaaS) cloud vendors have methods to handle persistent disk—some more transactional than others and some more work than others. Expect this process to become standardized (and possibly for that standardization to take the form of CSI above) in the next year or so. If persistent data is critical to your container environment, it is worth looking at how each solves the problem, as some of the solutions are procedural (such as Docker’s recommendation), and thus are somewhat portable across COE solutions. The weakness of IaaS providers is portability between systems, but that is true of all IaaS procedures/solutions at this time.

    Virtualization Engines

    Both VMware vSAN and Microsoft NTFS have systems for providing persistent disk. While much of this is the simple case of “disk that doesn’t disappear when the container does,” that is the bulk of persistence needs. Expect the support to continue to expand, perhaps even going so far as to implement 100 percent transactional shares.

    Storage Array Vendors

    Storage array vendors have useful tools for this area. While what you use to access their storage is not too terribly large a concern to these vendors, they do have a vested interest in seeing that their storage meets the needs of the changing enterprise. As such, they are delving into methods to help make persistent storage available and consistent. This is not a huge stretch for the vendors, as the systems they produced were always intended for multiple access. Where they lag (and where Rex Ray steps up) is in interoperability/interchangeability of vendor solutions.

    Conclusion

    For those with controlled access to persistent/shared storage, these issues are not that big a deal. For high-volume or high-turnover DevOps installations, though, persistent storage is another link in the chain of reliability that must be protected. Keep kicking it, and hopefully before your need becomes urgent, a solid system will be in place to manage and protect persistent stores.

    — Don Macvittie

  • DevOps Gets More Exciting in 2018

    DevOps Gets More Exciting in 2018

    As 2017 winds down and the IT industry prepares for 2018, there are a number of DevOps trends that everyone should be keeping their eyes on. It’s not so much that something incredible and explosive is expected in the new year. Rather, 2018 will be a time when DevOps starts to mature alongside other trends important to developers.

    These trends don’t guarantee organizations will be immediately successful with DevOps. Most still will struggle with the management aspects of DevOps, especially the need to keep feuding silos within IT working together as one happy group. Assuming developers, QA, and Operations can all learn to get along and act as a team, the technology supporting DevOps will continue to improve and the ecosystem will grow.

    Here are some of the DevOps trends that Amalgam Insights is following for 2018:

    Everyone Loves Containers and the Ecosystem Grows

    Containers have become a common technology in many IT departments. Although the majority of applications remain housed on single servers and virtual machines, container deployments are growing. Containers are an important enabling technology for DevOps—by creating small, transportable bits of applications, containers make it easy for IT silos to work from the same page.

    In 2018, growth in container deployments is expected to accelerate as large companies outside the software industry, such as those in financial services and health care, adopt microservices infrastructure. Containers that are especially suited to microservices will continue to expand rapidly.

    Increased container growth is also due to an expanding ecosystem. More companies have entered the market with products necessary to large-scale production applications. Products now exist for securing, managing, deploying and enhancing container services that weren’t available a few years ago. What’s more, containers are now available in all the major cloud services including Oracle Cloud, Microsoft Azure, AWS, IBM Cloud and Google Cloud. Containers are becoming enterprise-ready and large enterprises are feeling more confident using them.

    Takeaway: Growth in the ecosystem contributes to growth in container deployments while expansion of containers entices companies to enter the ecosystem. Containers have reached mainstream normalization and we expect to see fast expansion throughout 2018.

    Kubernetes Eats the Container Orchestration World

    In the container orchestration space, Kubernetes has become the tool of choice. Orchestration is the deployment and management of container clusters and is necessary to deploy them at scale. Kubernetes, originally a Google open source project and now managed by the Cloud Native Computing Foundation, has become the most popular orchestration tool available. That became obvious when Docker decided to distribute and support Kubernetes, despite having invested in its own orchestration system called Swarm. CoreOS, Microsoft Azure, Red Hat OpenShift, Docker and others have their own enterprise distributions as well.

    Kubernetes brings to the container ecosystem a relatively easy way to manage containers at scale. In large container deployments, containers are constantly being spun up, shut down, moved and copied. Once container clusters are in place, they must be monitored for security and service failure.

    During development, there will be a number of clusters that must be managed for development, test and final release to production. Doing this manually is close to impossible when even a moderate number of containers are in the DevOps pipeline. Kubernetes provides the tools and workflow to simplify these tasks.

    Takeaway: As container production clusters grow and as they are integrated into the DevOps pipeline, the need for orchestration becomes acute. Kubernetes has won the game and is now the tool of choice for orchestration. 

    The Pipeline is Getting More Integrated

    DevOps is often described of as a “pipeline,” In this context, a pipeline is a linear workflow that moves from build to test to release. As DevOps becomes more common in organizations, the pipeline grows to include additional test phases, monitoring and continuous improvement to create more of a cycle than a linear flow.

    One of the great impediments to making DevOps a reality is a lack of tools that support the entire pipeline. There are plenty of tools that help build applications, such as IDEs and code repositories from companies such as IBM, Microsoft, GitHub and Oracle, as well as dozens of open source products. Testing is well-supported with test data management and test plan management tools from companies such Atlassian, Informatica and Oracle. Release management and deployment products are widely available from Atlassian, Microsoft, IBM and many other companies. The problem lies not with the support for the individual steps in the pipeline, but with the integration of these tools to provide a complete end-to-end workflow. This creates a disjointed workflow, prone to errors and supportive of silos instead of collaboration.

    This is changing, however, and announcements late in 2017 herald toolsets that can provide a continuous DevOps pipeline. A great example of this is the newest versions of Microsoft’s developer tools that were debuted at Microsoft Connect(); in November 2017. Tying together the Visual Studio debugger with Kubernetes allows developers to test and debug as close to the production environment as possible and then deploy cluster to the test environment. That, plus the upgrades to Team Server, shows the commitment Microsoft has to creating integrated DevOps systems.

    Takeaway: While DevOps tools are still more or less disjointed, we are seeing a movement to fully integrated toolsets that enable the end-to-end DevOps pipeline.

    2018 is the Year of DevOps—Unless IT Management Gets It Wrong

    This coming year could be the year that DevOps truly takes off and becomes mainstream. Those who believe it is already mainstream now are fooling themselves. Management may believe their organization is implementing DevOps, but too many barriers still exist—especially management itself. IT departments remain heavily siloed and independent of each other, as do development tools. DevOps at scale requires, above all else, a collaborative and cooperative approach. Sadly, that does not yet exist in many IT departments. As tools reach maturity and best practices become common, managers become the inhibitors of DevOps. New management structures, incentives and thought processes need to be in place or the money spent on tools is wasted.

    Takeaway: All the DevOps pieces are in place except for the breaking down of functional silos that exist in IT departments. 2018 may be the year of DevOps if IT managers can create a new environment of cooperation.

    About the Author / Tom Petrocelli

    Tom Petrocelli is a contributing analyst with Amalgam Insights. His area of interest is collaboration and new ways of work, developer tools, IT project efficiency, governance and methodologies and DevOps. Most recently, Tom worked for a large, global banking corporation. Previously, he was the research director for Enterprise Social, Mobile and Cloud Applications at Neuralytix. He is an experienced marketing, technology and business executive with 30+ years in the computer technology industry. Tom’s background spans software engineering, systems architecture, IT, product management and marketing and general management. Connect with him on LinkedIn and follow him on Twitter.

  • DevOps Predictions for 2018

    DevOps Predictions for 2018

    The new year is upon us, and that can only mean one thing: outrageous tech predictions! This year, we have a few that are easy to believe and a few that are completely off-the-wall. See if you can pick them out.

    Serverless Overtakes Containers

    Serverless is the future and containers are dead. Some (or maybe even a majority) of the industry will come to realize this. Serverless is largely defined as using functions as a service (FaaS) as the backbone of your architecture. These services are a layer on top of containers that provide an opinionated way to run code that scales with demand and ties true compute usage to costs at sub-second billing increments. Much like virtualization in the data center was dwarfed by cloud providers doing opinionated virtualization a decade ago, serverless offerings will overtake the container market.

    DevOps Backlash Gets Loud

    DevOps has been happily spreading through the industry as an extension of Agile to the rest of IT. But in 2018, the pushback comes. In the early days of DevOps, there was a movement called “No-Ops.” It was championed by some, but then later, reversed back to DevOps. This year, we will see stories of failed digital transformations and DevOps attempts at conferences across the globe. At first, they will be marked as “not doing DevOps right,” but as more and more happen, they will be branded with the old-new stigma of No-Ops—and they won’t come back.

    Chaos Engineering Overtakes the Airwaves

    You remember the Chaos Monkey Netflix released to the IT world that captured the hearts and minds of software engineers everywhere? Well, this is the year when chaos reigns! We’ve already had a taste of it through the new open-source project ChaoSlingr on the Modern Security Series show at Signal Sciences, but my guess is that we are far from done. With the release of Nassim Taleb’s book “Antifragile” in 2014, this has been brewing for a few years. Expect to see more chaos-themed engineering books, conference talks and podcasts in the coming year.

    Software Flaws Mean Prison Time

    Unit testing used to seem like such a good idea, but this year it’s going to become illegal to release software without testing rigor applied. Congress has been on the edge of threatening software accountability legislation for years now, but at this point it looks unavoidable. Will this be the year that a landmark legal battle happens and an engineer who didn’t do proper testing gets thrown in jail? Don’t laugh. We have already seen the first signs of this with the VW diesel scandal; just imagine if it was a pacemaker failure. In 2018, lawyers make the new rules for software engineering.

    Promise Theory Finds New Adoption

    Mark Burgess brought physics to IT through his groundbreaking book on Promise Theory and its implications for the way our applications and infrastructure will be built over the coming years. Instead of making assertions of obligation from machine to machine, networks will be composed of voluntary agents making promises. At the time it was released, Promise Theory received a lukewarm reception. But with the new services-based architectures such as serverless and even the rise of container orchestration platforms, the industry is now ready to adopt Promise Theory thinking. Networks, applications and systems are already functioning as voluntary agents working together through promises. This year, we will see more and more mainstream adoption of the concepts and more examples of how to get it right. Get ready to learn how systems can be self-organizing.

    Application Security Becomes More About the Application

    The perennial web application security checklist is to avoid XSS, SQLi and all the rest of the OWASP Top 10. Application security will finally shift from language and framework vulnerabilities to more conversations around how applications are getting attacked. In the new year, the conversation hinges on account takeovers, fraud detection and abuse inside the application. This is good for the industry because we will move past fighting automatic scanning by adding real instrumentation to detect bad actors.

    Happy New Year!

    The next 12 months are certainly going to be an adventure, and I hope you have a happy new year.

    P.S. Please don’t get thrown in jail!

    — James Wickett

  • Axway Extends API Management Reach

    Axway Extends API Management Reach

    With application programming interfaces playing a larger role in the DevOps environment, Axway has added to its API toolkit for developers. The company expanded its AMPLIFY API management platform to include a visual tool for composing and integrating APIs and enhanced its API Manager tool so that it now can be invoked to scale resources up or down as needed. In addition, Axway unveiled API Central Service that makes it easier to integrate APIs across an ecosystem of organizations that make their services available via the AMPLIFY marketplace.

    Other new capabilities include an open-source MQTT connector for building internet of things (IoT) applications, a connector to the Microsoft Azure Cloud and tighter integration between Axway’s MFT file transfer software and the Syncplicity file synchronization software it acquired earlier this year.

    Axway also merged the Titantium SDK for building native mobile applications using JavaScript into the Core SDK for developers and is previewing the Query Builder Analytics tool, which enables users to do ad hoc query of mobile statistics. The Titanium code is based on technology developed by Appcelerator, which Axway acquired early last year.

    The announcements were made at the recent Axway SPARK 2017 conference. Vince Padua, vice president of platform and products for Axway, says these and other enhancements to the Axway API portfolio are intended to make it easier for organizations to craft microservices spanning multiple platforms. In some instances, those microservices will be crafted by developers, but Axway believes more advanced users will be stitching together microservices on their own using visual tools, says Padua.

    While most early use of APIs was driven by web-based applications facing external users, using APIs to interconnect services inside the enterprise now dwarfs the number of instances of published external APIs, he notes. The proliferation of those APIs has created a significant DevOps challenge for organizations and requires an API management platform to streamline the management of various back-end service. At the same time, Padua notes that DevOps teams working in collaboration with line-of-business units have taken over much of the API management, which previously was left up to individual developers. Now, IT organizations are trying to strike a balance between maintaining control and developer productivity, he says.

    The shift in API management responsibilities has also driven a wave of mergers and acquisitions over the last two years as API management has become embedded in mainstream application development platforms. But there’s also a fierce debate over the degree to which legacy API management platforms can support microservices. As microservices based on emerging technologies such as containers or more mature Java programming tools continue to evolve, API management is becoming more critically important. Each microservice generates its own API that needs to be maintained as each microservice gets continually updated. That requires an API management platform that can support all levels of API call activity that once would have been assumed to be unimaginable.

    — Mike Vizard

  • Core Factors Influencing Java Memory Consumption

    Core Factors Influencing Java Memory Consumption

    Java is known for having an insatiable appetite for resources, requiring strict controls to not exceed budget limits. However, the latest JDK updates and other developments have helped to make Java runtime extremely flexible in terms of memory usage. When configured smartly, Java can be cost-effective for different types of applications—cloud-native or legacy, microservice or monoliths. Today, it’s possible to make Java very slim from the start and scale it up gradually without consuming unnecessary resources.

    Here we’ll take a closer look at the factors influencing resource usage inside JVMs, see what tools can help and how processes can be tuned to gain more elasticity and scalability, ultimately reducing TCO.

    Scaling Possibilities: Virtualization vs. Containerization

    Within JVMs there are several scaling options: vertical, horizontal and a combination of both. To maximize results, it is important to start with a vertical scaling configuration that optimizes RAM and CPU usage inside every instance, based on current load levels. As a next step, these settings will be replicated across all instances while scaling horizontally.

    VMs Scaling

    While starting to run a project inside a VM, it is important to pick the right size machine, as too small an instance can lead to performance issues or even downtimes during load spikes. However, over-allocating will result in wasted unused resources during normal load or idle periods that still must be paid for. This can be a tough choice.

    Start with a smaller VM and scale it as the project grows. When just a bit more resources are required, the cloud vendor can offer you to double the size, as this is the most commonly used scaling step (e.g. see AWS offer below).

    During such scaling, the current VM normally is paused to migrate and redeploy the application. This inevitably causes some downtime and requires manual intervention.

    Memory ballooning provides a possible way (although rather complicated) to modify the amount of resources inside VMs on the fly without incurring downtime. However, this process is not fully automated and requires special tools to monitor the memory pressure in the host and guest OS and then activate vertical scaling based on the needs. As a result, memory ballooning doesn’t work well in practice, as the memory sharing should be automatic to present the expected advantages.

    Unfortunately, lack of VM elasticity and efficiency in terms of resource usage often leads to overpaying.

    Containers Scaling

    Containerization presents a new way of application packaging that makes resource consumption more granular and flexible, by automatic resource sharing among containers on the same host using cgroups. Unlike VMs, containers’ boundaries can be easily scaled up and down without the need to pause and reboot running instances.

    There are two types of containers: application and system containers. Both use the same scaling approach but they are intended for different purposes. An application container (such as Docker or rkt) typically is more lightweight, runs a single process and is used mainly for creating new projects, as it is easy to build them from pre-existing Docker image templates.

    A system container (LXD, OpenVZ) behaves like a full OS and can run full-featured init systems. This type of container is preferable for monolithic and legacy applications, as it can reuse the originally created design and important “old school” settings.

    Even system containers are much thinner compared to virtual machines, so scaling up and down takes much less time. And the horizontal scaling process is very granular and smooth, as a container can be easily provisioned from scratch or cloned.

    Garbage Collector Choice

    To maximize the efficiency of the above mentioned vertical scaling inside a Java-based container, it is important to select the appropriate garbage collector (GC) and tune it according to the project needs.

    Let’s review five widely used garbage collector solutions:

    • G1: From JDK 9, this modern shrinking garbage collector is enabled by default. One of its main benefits is the ability to compact free memory space without lengthy pause times and uncommit unused heaps. This GC is considered the best production-ready option for vertical scaling.
    • Parallel: A default GC in JDK8, it is a good option for low pauses but not for memory shrinking. This makes it inappropriate for flexible vertical scaling.
    • Serial, ConcMarkSweep (CMS): Both of these are shrinking garbage collectors with the ability to scale memory usage in JVMs vertically. But in comparison to G1, they require four Full GC cycles to release all unused resources. A new JVM option (-XX:-ShrinkHeapInSteps) in JDK 9 can  speed up memory release by bringing down committed RAM right after the first Full GC cycle.
    • Shenandoah: Shenandoah is a rising star among garbage collectors which promises to be the best solution for JVM vertical scaling. Its main benefit over the others is the ability to shrink in live mode without necessity to call Full GC, which helps eliminate significant performance degradation.

    Let’s compare memory behavior with shrinking G1 and non-shrinking Parallel:

    With Parallel, the unused RAM is not released back to OS. JVM keeps it forever, even disregarding the explicit Full GC calls. So even if your application has low RAM consumption, the unused resources cannot be shared with other processes or other containers as it’s fully allocated to the JVM.

    The modern shrinking G1 garbage collector significantly improves resource utilization compared to the previous example. The reserved RAM increases slowly according to the real usage growth. And all unused resources within the Max Heap limits are not wasted by standing idle, but made available to other containers or processes running in the same host.

    To summarize, Java applications with non-shrinking GC or non-optimal JVM start options hold all allocated RAM. For scaling Java vertically, the garbage collector should provide memory shrinking in runtime. In other words, it has to package all the live objects together, remove garbage objects, uncommit and release unused memory back to the operation system.

    Charging Model of Cloud Providers

    Nowadays, most of the cloud vendors offer a “pay as you go” billing model. This means that it is possible to start with a smaller machine and then add more servers or migrate to a twice-bigger instance as the project grows. But as described above, usually you reserve more than the application actually consumes to avoid the risk of downtime during the load spikes. As a result, many organizations overpay because the account is charged per VM but not based on the consumed resources inside.

    At the same time, there are providers offering  “pay as you use” billing approaches that became possible with the help of containerization, resource granularity and automatic vertical scaling. The payment is based on the resource units inside container. And the required capacity is provided and reclaimed on the fly in accordance with the load at the current moment. As a result, you are charged based on actual consumption and are not required to make complex reconfigurations to scale up or down.

    Taking into consideration all these factors and implementing the right settings can significantly reduce the costs for Java hosting resources, as well as positively influence application performance as a whole. Cloud and container technologies continue eliminating barriers and offering new opportunities for projects with different architectures and budgets.

    About the Author / Ruslan Synytsky

    Ruslan Synytsky is CEO and co-founder of Jelastic. He designed the core technology of the multilingual cloud platform that runs thousands of containers in a wide range of data centers worldwide. Ruslan worked on building highly available clustered solutions, as well as enhancements of automatic vertical scaling and horizontal scaling methods for legacy and microservice applications in the cloud. Ruslan is actively involved in various conferences for developers, hosting providers, integrators and enterprises. Follow him on Twitter.