devopsy.kinsta.cloud

Author: Chris Tozzi

  • DevOps Lessons from the Dyn Attack Fiasco

    DevOps Lessons from the Dyn Attack Fiasco

    Dyn’s DNS servers suffered a major DDoS attack last week, slowing a host of major websites. Does the drive for DevOps automation increase the risk of similar attacks? Maybe, but it doesn’t have to.

    As almost every geek knows by now, a DDoS attack on Oct. 21 overwhelmed Dyn’s DNS servers, and websites that depended on those servers slowed to a crawl. The attack was made possible because a large number of Internet of Things (IoT) devices were compromised using Mirai, a malware tool that breaks into devices using default access credentials.

    While it’s not yet clear exactly why so many IoT devices were configured with default login credentials, it’s likely that at least part of the reason included the following factors:

    • IoT devices are so numerous that securing them all is very difficult, especially when they are configured with weak security out of the box.
    • The need to give multiple users access to the devices encouraged installers and service providers to keep the default credentials in place. For example, if you have a smart thermostat that is installed by a contractor, managed by a utility company and owned by a homeowner, each of those parties could potentially need access to the device. Sticking with default login credentials is a lazy but effective means of providing it to all of them.

    In certain ways, the shift to DevOps makes challenges like these more common. Consider the following:

    • DevOps encourages the use of microservices. That means more services and logins to manage. It increases the risk that some services will not be secured properly because default login credentials will remain in place.
    • Under DevOps, the entire software delivery team is supposed to have access to all platforms and resources that are part of the delivery chain. That’s different from waterfall development, in which developers only had to be able to access developer tools; IT ops only needed to access management tools; and so on. The need for broader access could encourage teams to take the easy way out by using default logins—or even just sharing login information, which is risky even if the credentials are not the product defaults.

    A shorter way of saying this is that to increase agility, DevOps increases complexity. Greater complexity translates to more potential security risks.

    Conclusion

    The Dyn attack was not a DevOps problem, and none of the above means that securing a DevOps environment is not possible. It is, but it takes more work.

    Still, the Dyn outage is a reminder of just how badly things can go if developers and admins make mistakes such as relying on default login credentials. This challenge will become more difficult, not easier, as DevOps continues to reshape the way software is written and deployed.

    — Chris Tozzi

  • Continuous Integration vs. Continuous Delivery: There’s an Important Difference

    Continuous Integration vs. Continuous Delivery: There’s an Important Difference

    If you read DevOps blogs today, it can be easy to assume that Continuous Integration (CI) and Continuous Delivery (CD) are synonymous terms. People often use them that way. But that’s a mistake.

    As an example of conflation of the terms CI and CD, take this page, which purports to be a “history of CI/CD tools.” The author does provide a truly excellent history of CI platforms.

    But CI tools are the only subject that the page discusses. It’s not a history of CI and CD. It’s just a history of CI platforms. To work the history of CD into the picture would require discussion of how software testing and delivery practices have changed over time, not just an overview of the history of CI software over the past several years.

    I don’t mean to knock the page. It’s a great read, and the mistake the author makes in treating CI and CD as synonymous is hardly rare. He or she is just repeating a common confusion. You see the same lack of distinction between CI and CD on Docker’s discussion of CI/CD, for instance.

    The Difference between CI and CD

    It’s unclear how this conflation of the two terms originated. But it’s easy enough to clear up.

    Simply put, CI is the process of integrating code into a mainline code base. In almost all scenarios today, CI is done using platforms designed specifically for the purpose. Implementing CI is, therefore, as simple as using the right tools.

    CD is more complicated. CD is about the processes that have to happen after code is integrated for app changes to be delivered to users. Those processes involving testing, staging and deploying code. There is no single type of tool that does all these things. These processes take different forms, depending on the culture of the team and the type of app it is creating.

    In actuality, then, CI and CD share little in common. Besides starting with the same word (“continuous”) and being different stages in modern software delivery pipelines, they don’t have much to do with one another.

    Why the Difference Matters

    Why is it important to steer clear of this confusion? The main reason is because, by conflating CI with CD, organizations can end up doing only the former but thinking they have implemented the latter.

    Setting up a CI server such as Jenkins or Bamboo to integrate code changes continuously does not mean you are doing CD. It just means you are using a CI server.

    To get to CD, you also have to organize your software testing, staging and deployment processes in a way that automates them as much as possible and makes them continuous. Doing so requires more than simply turning on a certain type of tool.

    How you actually do CD is fodder for a separate, much longer article (or you could just read the book about it). But recognizing that CI is not CD, and avoiding confusing formulations including CI/CD, is a start.

    — Chris Tozzi

  • DataOps: Time to Bring DevOps Ideas to Big Data

    DataOps: Time to Bring DevOps Ideas to Big Data

    What can data scientists learn from DevOps? A lot, actually. By applying DevOps principles to the world of big data, you get DataOps, which streamlines data analytics.

    If you’re reading this site, you’re probably already familiar with DevOps. The DevOps philosophy emphasizes continuous collaboration between software developers (the “dev” people) and system admins (the operations, or “ops,” folks)—as well as everyone in between, such as software testers.

    It’s possible to apply similar ideas to the world of big data and data analytics. There, as in the world of software development and delivery, different teams traditionally have worked in isolation from one another. Those teams include storage admins, who keep data storage systems running; data analysts, who write and execute programs for interpreting large volumes of data; data quality specialists, who ensure data consistency; and data security teams, which protect data from attackers.

    As DevOps professionals know, keeping different teams isolated from one another is not efficient. It limits collaboration and maximizes confusion when problems occur.

    The data world, therefore, has much to gain by ending this isolation between teams and adopting a DataOps workflow. This will keep data operations flowing smoothly and provide continuous insight into the data that organizations collect.

    DataOps is not a brand new concept. People like Andy Palmer have been writing about it for a little while now. But the term and the idea have yet to gain nearly as wide a following as DevOps. (There is not yet even a Wikipedia entry for DataOps—and you know an idea is still in its infancy when no one has taken the time to write about it on Wikipedia.)

    Making DevOps Bigger

    Meanwhile, from the DevOps perspective, the growth of interest in DataOps is significant because it exemplifies the broad applicability of DevOps principles. So far, the DevOps conversation has focused on the relatively narrow context of software development and delivery. But it can extend much further, too. DataOps is a start.

    So, if you’re a data analyst, storage admin, data security professional or someone who works with data in any other capacity, it will be worth your while to borrow some ideas from your DevOps brethren. And if you work in software development, testing or delivery, it may be worth your time to educate your data friends on DevOps principles. It will make their workflows more efficient, while introducing DevOps to a wider audience.

    — Chris Tozzi

  • Reasons Why DevOps May Not Work for You

    Reasons Why DevOps May Not Work for You

    DevOps is a great way for many organizations to optimize software design and delivery. Yet, like every other technological innovation in history, DevOps is not perfect. Here are some examples of drawbacks associated with DevOps.

    By no means do I think DevOps is a bad idea in general. I want to be very clear about that.

    However, organizations considering making the move to DevOps—and those that are already doin’ DevOps, as we say here on devopsy.kinsta.cloud—can gain useful perspective by recognizing the difficulties that DevOps can present. Those include …

    Narrower Team Member Roles

    Traditionally, software engineers were expected to bring “full-stack” skills to the table. That meant they should understand the ins and outs of all layers of an application stack.

    Under the DevOps model, each member of the software delivery team has a more rigidly defined role. Coders code, QA engineers test, IT Ops admins administer and so on. These people all talk to each other, but they do not share in the same tasks.

    In many cases, this approach is better because it allows each professional to focus on what he or she does best. But this can be a drawback for full-stack developers. Their broad skill sets are no longer valuable. This is why people such as Jeff Knupp complain that DevOps is “killing” developers.

    There’s a potential downside here for organizations, too. Under DevOps, they have to hire and manage more personnel to make sure there is a specialist who can cover each stage of the software delivery pipeline, rather than investing in a smaller number of full-stack developers.

    Software Feature Bloat

    One of the key advantages of DevOps is that it promotes continuous software delivery. That means new features can be rolled out faster.

    But the ability to add features more easily also raises the risk that software delivery teams will add functionality to software just because they can, not because it’s needed. More features are not always better, even if they can be implemented efficiently.

    Infrastructure Changes

    To make the most of DevOps, most organizations will have to upgrade at least part of their infrastructure to optimize software delivery. Many will need to migrate apps running on virtual servers to containers, for example.

    There is nothing inherently wrong with such infrastructure upgrades. But they are potentially expensive and complicated endeavors, and if you don’t complete them properly, you might not actually get very much out of the other DevOps processes that you implement. Organizations considering DevOps should keep this in mind.

    Half-Baked DevOps

    The final major risk of a DevOps migration is not actually a downside of DevOps, per se. It’s a risk that results from the chance that organizations will implement DevOps imperfectly.

    This can be easy to do, because organizations can mistakenly believe that doing DevOps means simply running a continuous integration (CI) server or changing job titles. In practice, DevOps also requires significant cultural changes and new methods of communication. If you implement a half-baked DevOps environment that includes the tools but not the culture, you’ll likely be worse off than you would without having attempted DevOps at all.

    Conclusion

    Again, I want to be clear that there are lots of good reasons to do DevOps. I’m not trying to go on record as the guy who hates DevOps, because I absolutely don’t. Almost all organizations have much more to gain than to lose by adopting DevOps, if they have not already.

    Still, being aware of the imperfections and risks associated with DevOps is important. DevOps is not a magic pill that will solve all of your software delivery problems with no side effects. If you want to avoid mistakes or unrealistic expectations when doing DevOps, you need to appreciate the drawbacks you may face.

    — Chris Tozzi

  • Native Docker for Windows Arrives: Are Containers Finally Serverless?

    Native Docker for Windows Arrives: Are Containers Finally Serverless?

    Is Docker finally ready for the age of serverless computing? That’s the message from the announcement this week of native Docker container support for Windows Server 2016.

    If you’ve been following Docker for a while, you know that Docker containers originally supported only Linux operating systems. Then, Docker made its platform available for Windows users, too—but with the major caveat that Docker Engine ran inside a Linux-based virtual machine image on top. So, you could run Docker from Windows, but Linux was still the middleman.

    Real Docker Containers

    This week, that all changed. On Sept. 26, Docker announced the general availability of native Docker container support on Windows Server 2016. By “native,” Docker means that the containers run using new primitives—the equivalent of namespaces in Linux—that are built into the Windows kernel itself. There’s no more virtualization.

    But that’s not all. The complete Docker tool set also now runs natively on Windows. The Docker CLI, Swarm orchestration, data volumes and all the other building blocks of a Dockerized infrastructure are officially compatible, as well as Docker containers themselves.

    The support is the result of more than two years of collaboration between Docker and Microsoft, which is now a Top Five contributor to the Docker open-source project.

    Caveats

    Native support for Docker is a big accomplishment. But there are still some limitations, including:

    • Docker containers can only run natively on Windows Server 2016 and Windows 10. Other versions won’t work with Docker because they lack the kernel enhancements necessary to support Docker containers, Scott Johnston, Docker COO, explained in an interview.
    • Some advanced networking configuration features for Docker are not yet supported because the “plumbing” required to make those features work is not complete inside the kernel, Johnston said, adding Microsoft is working to change that.
    • Most notably, Docker containers on Windows can only run Windows apps inside the containers. In other words, you can’t run an app compiled for Linux inside a Docker container running on Windows. You would need a Windows host to do that. (And the opposite holds true: You can’t run Windows apps inside Docker containers on a Linux host.) This is not something that will change, according to Johnston.

    Docker Becomes Serverless

    Docker the company is celebrating the support as an opportunity to expand the Docker ecosystem immensely. Support for Windows “doubles the potential market opportunity” for Docker partners, the company said in a presentation about the announcement. (By the way, Docker has good reason to want to keep its partners happy by giving them a larger ecosystem to work with. The integration of Swarm into Docker earlier this year undercut the value of some of Docker’s partners’ products. But I digress.)

    The most interesting aspect of this news from a DevOps perspective, however, is that it means Docker is now much more serverless. A large part of the value of Docker containers is their ability to simplify application delivery. As long as you can package your app as a Docker container, you can deploy it on any server that has Docker running.

    You don’t have to think about the underlying server hardware or configuration. In other words, Docker allows you to become serverless, because you no longer have to think very much about the server you are deploying to.

    Previously, however, this flexibility was limited by the fact that you did have to think about the server OS. Now, with native Docker support for Windows, that’s no longer the case. It doesn’t matter whether your infrastructure is composed of Windows servers, Linux servers or both. As long as you install Docker on them, you can deploy to any server.

    Of course, there is still the issue of having to make sure your app is compiled for the right operating system. But that is easy enough to manage in most cases during the build process.

    — Chris Tozzi