devopsy.kinsta.cloud

Tag: Co-location

  • Case Study: DevOps Key to AWS Cloud Migration

    Case Study: DevOps Key to AWS Cloud Migration

    DevOps practices halved the time it took this SaaS company to migrate to AWS

    Australia-based Whispir provides communications applications as a service. One of its goals is to get products into users’ hands as soon as possible, and that requires development, operations and product staff working together, said Jonathan Swift, head of product and operations.

    Whispir formerly operated from a co-location data center, a model that continued as it expanded internationally. But the time and effort needed to run the systems was not cost-effective for an organization of its size, so the company switched largely to AWS. Governance issues faced by some clients force the company to still run its own systems in certain geographical locations.

    In the process, “[We] learned a lot about our app and how to make it cloud-ready,” said Swift. “Cloud isn’t free, cloud isn’t cheap” and is potentially more expensive than placing servers in co-lo data centers if the software isn’t built correctly.

    For example, it makes sense to make the most use of a physical server as it is largely a fixed cost. But in the cloud, the fewer resources you use, the less it costs.

    Crises Lead to Spikes

    On the other hand, a lot of Whispir’s business is crisis-focused (eg, Caltex uses Whispir at its Kurnell oil refinery in Sydney to send emergency messages), and it takes a certain amount of time to spin up additional instances to deal with spikes in message volumes. “That was a real challenge,” said Swift.

    Those crisis-communications clients expect Whispir to have sufficient resources to meet sudden demand. That’s an expensive proposition with co-located hardware, whereas Ccloud resources can be scaled up quickly, though not instantaneously.

    Seasonality has to be taken into account. One relatively predictable driver of crisis messaging is Australia’s bushfire season. Unfortunately, this overlaps Christmas, which is a peak time for marketing messages. So Whispir plans to run at around 200 percent of its normal baseline capacity during the fire season. This is easily done with cloud infrastructure, which Swift describes as “a beautiful thing” in this context.

    Some organizations are scared by multi-tenant environments. The ability to stand up dedicated instances on AWS is a great feature that Whispir can offer, especially to its clients in the financial sector.

    The company’s DevOps practices had been focused on the co-location environment, so even though kanban still “works very well for us” after moving to AWS, some previous choices had to be revisited. In particular, new monitoring tools were needed to cope with the elasticity inherent in the cloud and because the focus expanded from the hardware to the whole stack.

    Selected tools included the Zabbix open-source monitoring platform and Sumo Logic for trend analysis.

    Earlier this year, Whispir moved all of its Australian customers into a “pure AWS” environment. There are some great tools for migration, Swift said, “but it’s not as straightforward or seamless as you might think.” He recommends doubling any initial estimate of the time required for such projects.

    There were some nasty surprises: The first month’s bill from AWS was “huge” and it took a couple of months to determine the cause, even with the aid of AWS’s “really beneficial” premium support service.

    Security requirements in various jurisdictions had to be taken into consideration, with updates to various procedures to ensure the new cloud environment was compliant. For instance, the procedures required when a physical system was decommissioned were well-understood, but how could they be translated into a cloud environment? One specific example is that an in-house storage device can be destroyed when it reaches the end of its life, but in the cloud it’s largely a matter of relying on the provider’s controls.

    Avoid Lock-In

    But Swift warned that “the challenge is to not lock yourself in” to a particular cloud provider. For Whispir, which operates a mixed cloud/on-premises environment (no cloud provider offers the right onshore facilities in New Zealand and the hardware that Whispir has installed in Singapore is too new to be replaced by IaaS), that is a requirement rather than a strategic choice.

    Whispir would have avoided AWS-specific features that would make the platform easier to use and more difficult to get out of, despite the fact that using provider-specific toolsets can yield significant cost savings.

    Whispir is keeping an eye on Azure—”great tools, great partnerships, great ecosystems,” observed Swift—and may migrate to Microsoft’s cloud in around 18 months.

    A particular advantage is that Azure Stack can be run on private hardware, which could be useful if Whispir decided to enter a market where Microsoft Azure wasn’t present.

    Either way, the intention is to go 100 percent cloud to the extent that is possible.

    ‘Infrastructure as Code’ Means More Power for Ops

    The move to the cloud has made Whispir’s operations people more powerful, he observed. The infrastructure-as-code concept allows for rapid deployment, enables operations to contribute more to product development, reduces risk and enables the company to be more responsive to the market.

    Furthermore, dealing with on-premises hardware still very much a waterfall situation, thanks to the delays and dependencies involved. But moving to IaaS means infrastructure issues sit comfortably within the kanban model.

    IaaS also means that various network-related tasks simply disappear, but other issues arise in their place. For example, some external services used by Whispir (eg, SMS gateways) require the use of VPN connections from a known IP address, but with cloud infrastructure the best you may be able to do is specify a range of addresses. Whispir’s short-term fix was to route such connections through a specific server in the co-lo center, and in the longer term it found providers with less-restrictive arrangements.

    Moving from co-lo to cloud would have been “incredibly hard” without DevOps, Swift said. It would have taken at least twice as long, he noted, and the company wouldn’t be able to run the mixed environment as well as it does.

    “There’s no way we would have been so successful” without DevOps, he said.

    — Stephen Withers

  • DevOps’ Cultural Myths and Logical Fallacies

    DevOps’ Cultural Myths and Logical Fallacies

    Remember that guy you knew in college who was working on his fifth or sixth degree with no plans to stop? At some point the idea of real work must have terrified him, and instead what appealed to him was a continual state of learning. If you have the funding to master five or six disciplines, and perhaps not the ambition to get going in any one of them, why not stay and explore the next one?

    But if you work in IT, chances are you did not stay in college for more than a few areas of study, and since you entered the work force, your future options dwindled down to two: become a specialist (deep thinking in a singular discipline, but not too broad) or become a generalist (broad thinking across multiple disciplines, not too deep in any one of them).

    Managers tend to start their careers as specialists who do so well they are promoted to manager, where their career shifts and they become generalists. From time to time they dabble in a particular area of study to speak the lingo, use the right buzz words and understand what is coming, but the role they play is inherently generalist, as they are forced to deal with a number of topics beyond the realm of any one particular area of study.

    We tend to associate the title “architect” as being the specialist who is deep in an area we are forced to contend with. Examples might include, “We need to hire a network architect, or a database architect, or a software architect …” the list continues. When we do this, we have certain expectations. We are looking for the “guru” who can solve our network problems or database problems or security problems—wherever the skill set was that caused us to hire this specialist. If our new employee cannot solve our problems, we begin the search for another who can solve them.

    But the candidate has expectations as well: The network guy may have a little knowledge about databases but does not expect to have to perform at the level of his peers in that discipline. If we hire a network guy and then ask him to do everything in every other discipline, we are the ones out of line and he will likely flee our employment in haste, calling us crazy on the way out.

    The Cultural ‘Frankenstein’

    To create a true master of the multiple disciplines that exist within IT would require creating a modern Frankenstein in networks, servers, operating systems, storage, database, middleware, software and mobility. And, of course, this list is not limited to one expression of each type, but indeed multiple expressions of each type. So for example, our database discipline likely includes deep expertise in Oracle, DB2, SQL Server, NoSQL, MySQL … you name it. If it is popular or relevant to our current stack, our Frankenstein must know that one.

    As DevOps has gained in popularity, we have conflated certain aspects of it and driven ourselves off the highway of logic. Too often, proponents have taken the concept of co-location from the agile beginnings of DevOps and applied it to the operational endings of DevOps. Taking down silos is mistakenly applied to co-locating IT operational staff. This conflation occurs billed as the panacea it once was to co-locating business staff, with the developers producing products for them. In the strictly agile context, this works well. Applying it to IT personnel on the back half of delivery is the least effective means of creating Frankensteins. Co-location is a good way to share ideas, from user back and forth to innovator. In general, sharing ideas between a network guy (with limited interests outside his discipline) and middleware guy (who could care less how the network is configured in detail) has limited success.

    These operational IT disciplines do interact, but only as they have need, which is significantly less than having to sit in the cube 4 feet away from a counterpart involved in a differing discipline. Co-location is not the cure for breaking down silos or increasing communication—common backbones are. Imagine the example of the medical profession: A hospital provides the common backbone for the neurosurgeon, the orthopedic surgeon and the heart surgeon. These disciplines are inherently separate, and if you have a particular illness, you are going to want the one who specializes in the thing you need done. While each doctor understands general health aspects, they specialize for a reason—a reason you want and need—when it is your time to seek help.

    The hospital provides the common facility, the common support staff, the common electronic systems and tools we use. Our medical records are stored in one place, no matter what type of treatment we seek, so each specialist has to enter them there and can read them there. But distinctions by practice remain. There is a special surgical floor for heart surgeries, where they do not conduct orthopedic work or neurosurgical work. The tools may be slightly different; the medical journals are. The training is different. The experience is different. And none of these skills transfer by osmosis of co-location if you put all three doctors in one central cubical cluster. It takes years to get where they are. The same is true of our IT specialists.

    If instead of co-location we looked more at common systems for moving, documenting and articulating change. If we gave limited access to our DevOps tooling to the specialists, confined to the niche they have expertise in, then instead of co-location we could have remote-work-concurrence. Experts from all over the country could work on the movement of change without ever physically seeing each other, but with full visibility into what is happening in real time through a common lens. Network operations center (NOC) personnel remain physically in the NOC, but can see the entire stack from that location and change only their portion as changes are required from there.

    It is the common backbone of DevOps systems that breaks down the silos of change transparency and change participation. Prior to DevOps we were all using different tools to move change and track change. With DevOps we can begin to articulate change and document it, in the same tool or interface. Meaningful access is granted only to the portions of tools and only where it is needed. Database guys have full access to that platform and nobody else does (other than to see it). The same model works for every other specialty. And before you know it, you have a concert—or rather a symphony of moving parts—instead of disjointed sounds no one can comprehend.

    Organizations that hold to the idea that staff will become Frankensteins, equipped with deep skills in all areas because they share cubicle clusters, would do better buying lotto tickets. Where the culture will require attention is in demonstrating to the impacted teams why participating in the common backbone approach has merit for each of them. For engineers who have never had to use “something different” before, this takes work, but is the real win.

    When complete, what has truly happened is a conversion to a culture of unity. The work of the genius is still protected in this method; in fact, it is amplified as now it is visible to the entire organization. So even the prima donnas should be happy with the approach. And the ability to orchestrate change never will have been so high, with so much real-time interaction, as after the embrace of a common backbone of DevOps systems. This should offer executives new insights into product creation and evolution. In this sense, the cultural changes reach from top to bottom of the organization, and they are achievable.

    To continue the conversation, feel free to contact me.