Tag: infrastructure

  • ngrok Adds Load Balancing to Programmable Network Service

    ngrok Adds Load Balancing to Programmable Network Service

    ngrok today added a global server load balancing capability to its service for managing ingress via an application programming interface (API).

    Alan Shreve, ngrok CEO, said the ngrok Global Server Load Balancing (GSLB) service reduces latency and increases overall resiliency by automatically rerouting traffic across a global network of points-of-presence ngrok manages.

    The overall goal is to provide a higher level of abstraction for managing network operations (NetOps) that can be programmatically invoked. NetOps takes the place of what are essentially the equivalent of Assembly programming tools to provision networking equipment, he added. The long arc of computing is moving toward higher levels of abstraction to make IT capabilities more accessible, noted Shreve.

    That approach eliminates the need for IT teams to assign network operations teams to manage appliances in favor of a service that also provides a demilitarized zone (DMZ) via a service that accesses reverse proxies and firewalls, noted Shreve. The ngrok service ensures unauthorized requests are instantly blocked.

    The ngrok service provides a programmable approach to managing network services that makes it simpler for DevOps teams to invoke them. Instead of waiting for a network operations team to provision networking equipment, an API makes it possible to integrate the provisioning of networking services within the context of a larger DevOps workflow.

    One of the challenges that IT teams continue to encounter is that while virtual machines can be spun up in minutes, the provisioning of network services can require anywhere from minutes to days, depending on the age of the networking equipment. The ngrok service doesn’t eliminate the need for network professionals, but it does make it simpler for them to collaborate with application development and deployment teams, said Shreve.

    It’s not clear if NetOps teams will one day simply become an extension of a DevOps team, but as applications become more distributed, the need to minimize network latency is becoming more critical. A service like ngrok makes it simpler for DevOps teams to ensure the appropriate level of bandwidth is available so distributed application performance doesn’t slow to a crawl.

    Of course, the bigger challenge will be bridging the cultural divide that currently exists between many NetOps and DevOps teams. Both groups have their own distinct nomenclature for describing services that, in many cases, the other simply doesn’t comprehend. As distributed application development accelerates, each organization will need to determine how to demolish those silos so both teams can more easily achieve a common goal.

    In the meantime, IT leaders might to reconsider their strategy as networking services become increasingly programmable via APIs. Most modern networking equipment today is software-defined in the sense that network operations teams have access to graphical tools to provision networking equipment. However, as APIs become more accessible, the need for those graphical tools may not be nearly as high as it is today as networking becomes just another service to be managed as code.

  • How Platform Engineering Can Drive Digital Sobriety

    How Platform Engineering Can Drive Digital Sobriety

    If we’re being honest, the “green issue” currently isn’t the priority it needs to be for a lot of companies. At best, they are ticking the box marked ‘sustainability’ and paying lip service to a trendy topic. At worst, they are engaging in nothing more than greenwashing.

    Considering carbon emissions from cloud computing have now overtaken those from commercial aviation, this is disheartening for the IT sector. Unsurprisingly, most of these emissions come from inefficient use of the cloud due to overly complex processes and a lack of centralized oversight over cloud infrastructures. And if “emissions” sounds a bit vague, here’s a translation into dollars–$147 billion was wasted in 2022 in cloud spend.

    So, when it comes to reducing an organization’s cloud carbon footprint, there is clearly a financial incentive. Even for those who don’t believe in individual responsibility or worse, simply don’t care what kind of world we’ll leave future (and some present) humans to live in, there is money to be saved.

    The truth is that companies need to put sustainability first across the whole enterprise, and they need to do it now.

    A New Kind of Business

    Tackling departments one-by-one or singling out individual processes to make them more sustainable is not going to work. What organizations need is to stop thinking of sustainability as an afterthought and start incorporating it in the decision-making process from the get-go, maybe even as part of their very business model.

    This is the sort of business that places its people and the planet above profits. Progressive, modern companies that want to stay in business longer need to invest in their people and take care of the planet instead of chasing endless growth and ever-increasing profit because that, simply put, cannot last.

    As a culture, we’re currently at a turning point–late-stage capitalism is running its course, and more people are realizing that constant growth at the expense of well-being and the planet’s dwindling resources is not sustainable (in all senses of the word).

    For this to work, it’s about looking at business through a new prism.

    Driving Digital Sustainability From the Ground Up

    It’s not too late for businesses to become more sustainable. In fact, it is the best time to make life-saving changes. When it comes to tech specifically, there are a couple of things you can do, starting with two key things that need to change.

    First, cloud waste has been costing companies billions of dollars for years now. Organizations are set to spend nearly $600 billion on the cloud in 2023, but it seems that most people are still in the dark about the real environmental impact of cloud usage. As with most big corporate changes, it’s all about a change of mindset. We need to accelerate awareness and promote digital sobriety to inspire transformation and decision-making with GreenOps on the enterprise level.

    A cloud-first approach has long been the holy grail of IT. However, what we couldn’t anticipate is the sheer volume of resource usage and complexity that comes with it. We simply do not have the planetary resources to support this growth much longer. It’s time to become smarter, leaner and more environmentally conscious by using the “consume better” approach.

    Second, think about the KPIs. Change without success parameters will not work. While organizations have a key role in decreasing global cloud spend, they must have the right tools and references. They need to be able to assess the environmental impact of their digital choices at all decision levels.

    For the IT sector, this is where platform engineering comes into play. An internal developer platform can act as the critical piece that connects DevX, deployments, governance, security and observability.

    With green best practices and cloud carbon footprint visibility at every stage–from the dev deploying a new environment on the cloud to the CFO checking the monthly cloud bill–a platform engineering approach ensures not only organizational but also individual responsibility. When all tools are accessible and processes transparent, nothing gets lost in translation. This means less room for error and cloud waste.

    Ultimately, it’s About Consuming Better and Consuming Less

    Right now, these KPIs will be financially motivated, and rightly so. But there will come a time when progressive organizations will put their environmental impact at the forefront of their mission statement.

    The B-Corp certification, which evaluates companies in terms of their social and environmental performance, grew 63% in 2022. This shows the trend toward moving to a more inclusive and sustainable economy. Partly because it’s in the interest of the businesses’ long-term success and reducing costs, but partly because it is simply the right thing to do.

  • Dell Looks to Expand Pool of Available DevOps Expertise

    Dell Looks to Expand Pool of Available DevOps Expertise

    Dell Technologies has quietly launched a training and education initiative to significantly expand the number of IT professionals that have DevOps skills.

    Brad Maltz, senior director for DevOps and developer relations ecosystems for Dell, said the next era of IT will require IT professionals that understand how to automate the management of application development and deployment. Given its long infrastructure history, Dell is especially interested in helping IT teams to use a mix of graphical tools and command line interfaces (CLIs) to programmatically deploy software at much faster rates and at scale, he added.

    Dell has already hired more than 200 specialists to provide that training as part of a DevOps Transformation Services consulting services engagement, said Maltz. The primary area of focus will be working with IT organizations that are embracing platform engineering to centralize the management of DevOps and GitOps workflows, he noted. Many of those teams are now relying on declarative automation tools that ultimately make DevOps more accessible to a wider range of IT professionals, added Maltz.

    The number of IT professionals that employ graphical tools based on IT management frameworks such as ITIL still far outnumber the number of IT professionals that have DevOps skills. The ongoing debate between these two camps essentially comes down to the need to be able to consistently manage IT at scale versus being able to adroitly keep pace with the needs of organizations that are increasingly dependent on software. In the digital transformation era, it’s become clear that more of the organizations that hire IT professionals are placing a higher value on agility, said Maltz.

    Bridging the divide between ITIL and DevOps to address that requirement is largely a psychological challenge that requires IT teams to address a range of cultural issues in addition to mastering different types of tooling, he added.

    Regardless of whether organizations continue to manage IT infrastructure internally or rely more heavily on managed services provided by Dell, the need to continuously deliver secure software faster has become a business issue. The trouble is the level of DevOps expertise within organizations tends to vary widely. A recent survey published by the Continuous Delivery (CD) Foundation found only 22% of respondents are using both continuous integration (CI) and continuous delivery (CD) to automate all building, testing and deployment of code in a production environment. Most organizations that have adopted DevOps make extensive use of CI to build applications, but CD has proven to be more challenging to master. In fact, the CD Foundation survey found only 7% of respondents described themselves as DevOps engineers or specialists.

    The bulk of the managed services that Dell provides are focused on managing infrastructure rather than deploying software, but the company does have a managed DevOps offering based on the application development and deployment platform that an internal Dell DevOps team built to construct and deploy applications. Naturally, Dell has a vested interest in accelerating the rate at which software can be deployed on IT infrastructure it provides and manages.

    It may be a while before the overall size of the DevOps community expands, but the training initiatives such as the one launched by Dell will help. However, it may become a lot easier to find and retain that talent in the months and years ahead. In the meantime, organizations would be well-advised to invest in upskilling their own IT teams. After all, it’s less expensive to train someone that already works within an IT organization than it is to compete against thousands of other organizations willing to pay top dollar for the best DevOps expertise money can buy.

  • Start Automating Now

    Start Automating Now

    A bit of feedback I received from last week’s post is that I didn’t provide any background, I just jumped right in with examples. So let’s take a step back and talk about the bigger picture.

    It is always a surprise to me when I talk to enterprise friends and associates and hear that automation has not taken over their DevOps processes. Yes, we still (and might always) want the ability to have a person review what is being done and make course corrections if the automation has something wrong, but most of the automation that is available for DevOps is no longer new and should be a no-brainer.

    The ability for a build to block based upon any given criteria and notify Jira or ServiceNow—or whatever issue tracking system you use—while alerting the people responsible for the build is simply obvious … and yet, some organizations aren’t even that far yet. The ability to stop a build, create a ticket, assign it to the appropriate person and give it a priority is built into most of these products. It should be looked at to see if it’s suitable to the organization.

    This goes on and on. We’re now to the point where everything in the toolchain can be automated. IaC and GitOps are tackling the broad operations end, and if the application is 100% public cloud-based, CloudOps simply solves the problem. Automation from check-in to production is possible, with very few places where intervention from humans is needed.

    Given that most of this technology is mature—even though some of it has not been around as long but has broad enough adoption that it can be considered mature (Flux and Argo come to mind here)—it is time for enterprises to start another round of automation. Just like AI won’t eliminate most development jobs this year, automation won’t eliminate jobs, either. But we’ve been so strapped for time for so long that this may be a surprise. Automation might reduce the need to scramble for new team members while asking current employees to give a bit more … over and over.

    So get started. The math that made DevOps appealing in the first place—streamlining to reduce time to market and free up IT hours—is the same math that applies to the current round of automation. The more that can be automated, the more time team members will have to perform more productive work.

    Having mentioned AI in passing, I’ll use that as a segue to include AI in the “automation” discussion. At this time, nearly all of the tools we call AI are simply advanced automation tools and should be considered in the process of increasing IT automation. The ability for an app to run, learn your systems and improve them (be that improvement in development, security or ops) is just advanced automation. In the places where it works well in a given environment, it should be adopted. It is highly unlikely that AI will get worse over time, so even marginally acceptable AI results make it worth implementing, counting on technology advancements and continued training to improve that benefit moving forward.

    So, start automating now. Free your time, take a holiday at the beach, do the same job in less time or, to quote a previous generation, “Work smarter, not harder.” You’re doing the job that keeps the servers humming and users coming, now do it with less effort. And keep rocking it.

  • Back to Basics

    Back to Basics

    We here at devopsy.kinsta.cloud do sometimes suffer from The Curse of Knowledge. We write about DevOps regularly for an audience that is doing or managing DevOps. We assume knowledge, but there are still a fair number of people and organizations who are unclear about some of the terminology/technology that we talk about.

    So if you are a regular reader of my blog, this might be a redundant post—but read on because it might have info you haven’t had a reason to consider yet.

    I’m going to start with the obvious—how most readers got here to begin with: DevOps. DevOps is automating the build process so that builds are automated and repeatable. Teamed with Agile development, DevOps means that development teams can rapidly iterate through development and generate regular updates to their applications. Everything else in this blog is based on the premise that it is running in a DevOps environment.

    Continuous integration: Typified by Jenkins, continuous integration (CI) is the part of DevOps that runs the automated builds. It includes reporting and defect resolution only in the sense that it feeds into other systems that track defects. With CI in place, a developer or DevOps engineer can say, “Okay, ready to test a build,” and with one command, can kick off a build that will result in a finished artifact. Or result in a built product, anyway. ‘Finished’ is optional, and the target of CI is not generally a running application.

    Continuous delivery: One of the most confusing things for new DevOps people is CD. The way that the space evolved, there are actually two things that CD could stand for; continuous delivery–automated build of the complete application —or continuous deployment—delivery of a running application onto infrastructure. Continuous delivery is straightforward and means all the bits of the application are code complete–or at least the current state of the application is complete enough to build an application. Continuous deployment is much more convoluted. It might mean deployed to a development environment for developer-level testing or a test environment for running automated (continuous testing) tests on or in production. It might be a staged rollout, where users are siphoned over, or it might be a hard rollout where the old version is replaced by the new in a manner similar to ripping a Band-Aid off.

    Continuous test: Automated testing is at the heart of continuous test (CT), and it is only now becoming a real possibility for most short-staffed enterprises. Advancements in machine learning (ML) mean that tests can be run, results filtered to eliminate false or duplicate results, and IT staff is only notified of the important bits. Unlike traditional testing, staff is required only to create the tests and to look at meaningful results.

    Infrastructure-as-Code: Infrastructure-as-Code (IaC) is the ability to define all relevant elements of infrastructure with text files. Often this is done in YAML and sometimes in a variety of other formats. The idea is that infrastructure provisioning can be on-demand and adaptable. It is certainly more agile to change a text file from port 80 to 8080 and then run a command to process the text file–now the port is documented and whenever needed the command can be re-run to move to a known state. IaC largely relies upon software implementations over physical. While the idea of software-defined networking (SDN) has fallen out of favor, the underlying principles are everywhere and part of IaC, for example.

    GitOps: All the rage these days, GitOps is probably best defined as applying DevOps to operations. That does sound weird, but I like to break it up as “traditional DevOps is DevOps, while IaC (to a lesser extent) and GitOps (to a huge extent) are DevOps.” The point being that GitOps uses the text files generated in IaC and worker nodes generally designed for continuous deployment and then performs operations tasks based on those text files. If IaC defines the entire environment that an application needs and DevOps defines all of the sources that an application is composed of, bundling the two together will enable full-on automation from conception to decommissioning. When a new version of a source file or configuration file is checked into Git, the system then makes certain that every instance of the application is updated with the new version. Of course, this process, like every one listed here, is far more complex than that in implementation; we are talking about ‘what, not how’ at this level.

    All of this really is the very tip of the iceberg, but that was the goal here – to offer readers a place to start if they’re finding themselves looking at DevOps for the first time. As you dig deeper, things like DevSecOps and test driven development (TDD), or defect tracking and Agile methodologies start to play into the landscape, and maybe I’ll write an intro bit for those also. But this time, we’re out of space.

    As always, keep rocking it. No matter which or how much of the above you are using, if the organization’s applications are up and running, that’s you. And you deserve more kudos than you are likely getting. But keep grinding through—the rest of us get it, and thank you.

    Image source: Photo by Towfiqu barbhuiya on Unsplash

  • Understanding the Infrastructure Automation Maturity Model

    Understanding the Infrastructure Automation Maturity Model

    Nearly every company wants to evolve its digital transformation and increase the pace of software delivery and operations. And infrastructure automation is a new means to achieve these ends. It supports the emerging discipline of platform engineering and can increase scalability and reactivity to unforeseen events, enabling software ecosystems to be more anti-fragile. Increased infrastructure automation also helps free up engineers from focusing too much on managing DevOps when they really should be building out new features.

    But introducing more infrastructure abstraction can be met with certain challenges. This is often because departments have a wide range of cultures, processes and tooling already in place. With varying maturity levels across teams, it can be challenging to gauge where a company is before attempting to introduce new automation across the board. And as further evolution may require significant changes, having a baseline is key to planning for the future.

    I recently chatted with David Williams, senior vice president of market strategy at Quali, about an internal technique they’ve been using at Quali to gauge the maturity of their clients’ infrastructure automation. Their infrastructure automation maturity model breaks down the journey into four unique stages, moving from the least-mature ad-hoc automation to the most-mature autonomous solutions. Below, we’ll review this model and consider what questions need to be asked to ascertain where your team sits on the spectrum. We’ll see how enterprises can leverage such a model to hone in on improving their software operations.

    Four Questions to Determine Infrastructure Maturity

    DevOps is still a wild west in terms of how people approach it, said Williams. So before attempting to mandate sweeping cultural changes, it helps to first determine the base level of DevOps maturity within an organization. Here are four questions that can quickly assess the state of infrastructure automation within an organization.

    1. What Culture Already Exists?

    First, it’s good to evaluate the culture that already exists. How do teams deliver value and how is this value measured? Perhaps throughput or code quality is measured with specific metrics or SLAs. Understanding the DevOps attitude (and aptitude) within the organization will highlight pre-existing capabilities and show how teams collaborate.

    2. How Are People Organized?

    The genetic makeup of teams can dictate much about development and operations. How are teams organized? Are they large, siloed teams? Or are they two-pizza-sized teams? Understanding team organization and communication patterns are necessary to identify bottlenecks.

    3. What Processes Are Already in Place?

    Next, consider the development timelines and procedures already established and in use. Are teams applying SAFe, Lean, Agile or Scrum methodologies? What are these exact processes and how mature are they? Answers to these questions will demonstrate general release cadences and how teams drive productization.

    4. What Infrastructure Automation is Already Being Used?

    Lastly, you’ll need a window into what level of infrastructure automation, if any, is already in use today. How is it delivered? How is it managed? How is the success of this automation measured, and what is the overall outcome? You will likely find that automation is typically quite siloed and teams have differing approaches. This auditing is vital to identify what works well and what doesn’t.

    The Four Levels of the Infrastructure Automation Maturity Model

    Asking the above questions will quickly determine an organization’s “readiness state,” said Williams. But how do these answers map to the overall maturity of the organization’s infrastructure automation? Well, he shared four basic levels of infrastructure automation maturity that are commonly found in practice throughout today’s software development environments, ranging from bespoke scripts to more robust self-defining automation.

    Level 1: Ad-hoc

    The most basic level of infrastructure automation is ad-hoc integrations involving a mix of old and new technology. This level is chaotic and fragmented, with varying processes used throughout the organization. These bespoke integrations and scripts also may not have excellent visibility, limiting the potential for reusability within an organization.

    Level 2: Automation

    Next is having automation that is specifically tied to a certain technology stack. Nowadays, organizations are commonly hybrid multi-cloud, having adopted automation piecemeal with each new stack. This often results in multiple silos of automation stacks that are blind to each other. At this maturity level, one automation might fire off another automated procedure and so on, but engineers aren’t 100% aware of how they influence each other.

    Level 3: Frictionless

    The next level of infrastructure maturity begins to take the reins with more intentional automation adoption. At this stage, organizations embrace an internal developer platform, similar to Spotify’s Backstage, that helps engineers deliver and manage infrastructure more seamlessly. This tooling is more advanced than other levels integrating key commands into the developer’s IDE. Doing so eliminates the need to switch between environments and reduces friction.

    Level 4: Self-Defining

    Finally, the most advanced form of automation involves deeper intelligence within application operations, including the ability to dynamically optimize infrastructure. At this stage, infrastructure is self-defining, meaning production applications can automatically adapt, be self-governed and enable bursting and non-bursting capabilities. Such intelligence is typically proprietary to cloud providers, said Williams, yet companies often desire this intelligence inherent at the application level. Williams describes this stage as the “intelligent allocation of resources in line with application life cycle requirements.”

    Benefits of Self-Defining Infrastructure Automation

    Self-defining infrastructure automation has many benefits for companies, including having the infrastructure in line with application needs, scalability and measuring usage to optimize cloud spending. Many points within the software development life cycle could warrant increased automation, such as infrastructure-as-code (IaC), deployment release engines, configuration, traffic scalability, cloud management, containerization and documentation. But it’s not until you open conversations around infrastructure automation maturity and understand who is responsible for managing it that you can hone in on what areas can be improved.

  • 5 Tips to Consider When Introducing Infrastructure Automation

    5 Tips to Consider When Introducing Infrastructure Automation

    Although we discuss infrastructure automation often, the fact is that a lot of manual processes still exist throughout DevOps. Whether it’s rewriting scripts to spin up new servers, updating cloud-native policies or configuring computing environments, manual toil is an all-too-common reality. To make matters worse, engineers commonly create lengthy scripts that work in isolation but are too bulky to be reusable by other developers working with slightly different constraints.

    More infrastructure automation is the key to increasing deployment agility, but setting aside time to automate your daily workflow can easily take a back burner, especially when developers are already behind on their long list of to-dos. So, for the time-pressed, where should they start? What exactly should be automated? And how can we effectively reuse and share automation to help other areas of the business?

    I recently connected with John Laffey, technical product marketing director, Puppet. Laffey has thirty years’ worth of tech support and engineering operations knowledge under his belt and was “doing DevOps before it was called DevOps.” He shared with me five helpful tips to think about when introducing more infrastructure automation into your workflows. Below, we’ll expand on each of those points and consider other best practices around automation as the industry moves toward platform engineering.

    1. Automate Things That Interrupt Your Day

    First, Laffey recommended starting with the things that are easy to fix. These are the tedious actions that interrupt your day. Perhaps a disk runs out of space and troubleshooting takes 15 or 20 minutes. Minor interruptions like this might seem like part of the job, but they definitely add up over time.

    2. Automate Things You Can Reuse

    Secondly, try not to automate things for an ultra-specific purpose, said Laffey. Instead, focus on building automation that is more generic and can have more value to others. This follows the DevOps mantra of teamwork and collaboration. One-off automation that is too verbose or specific to a certain condition won’t apply well across groups.

    3. Automate Stuff That Doesn’t Change Much

    Next, automate the infrastructure that isn’t prone to change. These are fairly reliable things that you’re sure will be part of your systems for the foreseeable future. Because if a component does end up having to be updated often, supporting the automation could be more hassle than it’s worth. “Don’t trade fixing problems for fixing automation,” said Laffey.

    4. Start Small

    Here’s some sage advice applicable to most new projects with a big scope—start small. Laffey recommended starting by automating the easy stuff that has proven immediate value. These tasks should be modular and do one specific thing, like rebooting a machine or draining a log file. “Automate things in chunks that you can put into a toolchain.”

    Too often, teams try to automate everything all at once. But by taking big bites, you can quickly get off track and potentially add to technical debt. If you keep things modular, you’ll find it easier to change and reuse this automation.

    5. Share and Be Open

    This last tidbit is more of a cultural change—don’t code so that you’re the only one who knows about it! Sometimes, engineers don’t necessarily want the world to know about the automation they create. They might worry about automating their job away. Yet, Laffey said, these worries are misplaced. There will always be more work to do, and automating toil away can free up time to help you focus on more exciting projects.

    So, Laffey encouraged DevOps and SREs to be open about the automation they create. Ensure they’re well documented and stored in a centralized repository to allow other departments to reuse your code. This can avoid the duplication of having five different versions of the same script written in different languages within the same organization!

    Next Step: Move From Scripts to Industry-Standard Tooling

    It can be challenging to argue for investing time and energy into infrastructure automation, especially when engineers are hard-pressed to keep the lights on. Yet, small gains in automation can make huge waves in the long term.

    Eventually, said Laffey, you’ll want to move away from using scripts within your toolchain, which he describes as a low-end DevOps maturity. But as you update your process, you’ll want to replace these scripts with more modern industry-standard tools like Terraform or Puppet’s DSL. He recommended slowly revamping scripts in a toolchain as you update them one by one. “Start where you are and avoid a monolithic approach.”

    In essence, platform engineering, Laffey explained, is wrapping DevOps principles at a higher level to make them more efficient. And we will likely see an increased emphasis on infrastructure automation as platform engineering gains more mindshare. “It’s going to become a necessity.”

  • Tips for Better Governance – Techstrong TV

    Tips for Better Governance – Techstrong TV

    David Williams, VP of Porduct Strategy at Quali, and Charlene discuss David’s top tips to better governance in DevOps. The video is below followed by a transcript of the conversation.

    Charlene O’Hanlon: Hey, everybody. Welcome back to Techstrong TV. I’m Charlene O’Hanlon, and I’m here now with David Williams, who is the VP of product strategy over at Quali. David, thank you so much for joining me. Good to see you.

    David Williams: It’s good to see you, too, Charlene. It’s good to be here.

    O’Hanlon: Excellent, excellent. So, lots of stuff going on in the world of DevOps these days, but one of the things that we have been hearing a lot more noise about is governance in DevOps. And especially with just the insane amount of automation that’s going on to help kind of speed up the processes, obviously, it can be pretty easy for things to kinda go off the rails. So I wanna kinda get a sense from you as to what you see happening in the space with respect to governance in DevOps, and how we can kind of maybe move away from the herding cats mindset when it comes to governance.

    Williams: Yeah. It’s been a journey on DevOps. DevOps, when it was introduced many, many years ago as a concept immediately aligned itself with Agile, so it became only the ability to get things out as continuously as possible, and also as fast as possible. And so what happened is that at that time, things were very much managed. It was small teams doing lots of work. As enterprises started kicking in, with DevOps become a serious strategic way of actually delivering applications to their consumers, the amount of people got involved grew. 

      The amount of teams that were involved grew. Even though the DevOps philosophy was small teams, fast moving, application componentry going out into the ether as fast as possible, it stopped being that quite a while ago, and started becoming much more mainstream. So unfortunately, DevOps grew up a lot, and one of the things that became an issue was governance. And governance from that perspective wasn’t really just about security – y’know, SOC 2 compliance, and making sure that your secrets are held and managed efficiently across the pipeline – 

      but it was also around, how do you understand what people are doing? And what is the interaction that they’re having? And why are they having that interaction? And basically it’s a fragmentation challenge, more than anything else. DevOps pipelines are fragmented, and so how do you get control without inhibiting innovation? And how do you get visibility, having lots of teams that wanna innovate in their own areas, but not necessarily give you that full end to end? So that’s the challenge that has emerged. 

      And I think it’s going to continue to do that for quite a while to come, because the applications are getting bigger and more contention-based in regards to infrastructure sharing. So governance is gonna be the thing.

    O’Hanlon: It’s interesting, because I think of it a lot like a company itself growing out there. When you have two or three people working in a company, it can be very difficult to get a lot of achieved without everybody kinda going all in. But as a company grows, you’ve got more people who are doing the work, but maybe they’re not necessarily talking to each other, and not everybody has all the information they need. So you kind of have to kinda build a technology, or at least a process, around being able to get the information out and helping everybody be on the same page. 

      And it kinda sounds like the same thing is happening in the DevOps space with respect to just having a greater understanding. Not just an understanding of the process, but just, I guess, a better way of enabling the process from an understanding and a communication perspective – as well as just making sure that all the right things are getting done at the right time. So what do you guys propose? What do you see that you believe could make a real difference?

    Williams: You used two words that always make me smile when I talk DevOps. I used to be an analyst, talking to a lot of DevOps practitioners, and the two words that I used to run for the hills were “process;” y’know, that was one thing. And so they wanted collaboration, but they wanted to be able to collaborate without being governed in any way. So I think that that process word is one, and anything to do with “abstraction.” So abstraction’s not a good thing. It takes you away from the core. 

      So what I’m saying to you is that any solution that enables you to get greater control on governance and understanding has got to be able to do it without impeding people’s ability to do their jobs. And it’s gotta be something that is subsumed within their normal breathing in and out. It can’t be something that’s another job to do. That defeats the whole point of, anything that’s to do with lean practice is about removing stuff. We’re the software industry, so the way that we make things less complex is by adding something, y’know; that’s the normal inclination. 

      I think DevOps, when you actually use it, whatever you add into the mix has got to be subsumed within your normal day-to-day activities. So it can’t be yet another thing to do. For example, security has gotta be there, but not intrusive. Why I came to Quali was because I’d spent a lot of time with companies where infrastructure, the ability to provision into the cloud, do all the things, was becoming a bit of a mystery. People would assume infrastructure. 

      So the governance around what you’re deploying and how you’re provisioning your infrastructure not only became fragmented through your pipelines – Because you plan something, and then you code it, you build it, you release it, and test it, et cetera. Each one of those is a very defined area. It might be one person; it might be many teams. But each one of those is a very specific thing you have to do. But wouldn’t it be cool if you could subsume testing into your code? Wouldn’t it be cool if you could consume compliance while you’re coding? 

      So you’re coding, and it’s automatically looking at configurations and modifications, and logging that appropriately without you having to add another layer. And I think the same with infrastructure. I think that the companies I’ve worked for before really didn’t know what was going on, and I worked for DevOps companies that were trying to work out, “Okay, so the infrastructure was there; now it’s gone,” or “It’s there, and it’s modified.” 

      Every commit that a coder does is, y’know, a point in time. “So what has happened between that commit and the next commit?” And so all these stages of infrastructure, how do you build that in? So at Quali, what we looked at is providing a method with which you can let people use infrastructure as code practices, configuration management tools. Do what you need, but it gets automatically subsumed into a model. And the model is what you rate on. That’s what applies the policies around access and et cetera. 

      But it should allow the coders, the release managers, the testers to do what they need, but also it provides sort of like a control plane on top that enables you to talk and be governed, but without that Big Brother type of thing looking at you. So I think that’s the magic about DevOps, is if you add something in, it’s gotta be something that doesn’t get in the way of speed. It’s gotta not provide another barrier. It’s gotta do these things. It’s a hard problem to solve, but if you approach it in a way that assumes that people are going to do things, they’re gonna break stuff – 

      People do that; that’s what they do. They do their own stuff. They don’t like being told what to do. So if you can provide as subliminal a level of control over infrastructure that says, “I know how you’re doing it. I’m now going to provide that same capabilities that we used in build to help the testers understand what to build.” So now you’re providing some consistency across, using the control plane. So you’re using your models, and you’re moving them across. But it subsumes what they already use, so you can add it in. 

      Your infrastructure stack and the way that you deploy it is implemented in the model that has policy, access control, secrets integration. And so that’s really the philosophy, is to make sure that we can provide the end-to-end capabilities, accelerate the delivery life cycle, but not impede the practitioners from what they do, but provide the business with visibility which they do not have today. So I would just follow up by saying cost isn’t a governance thing, but unless you govern it, and you understand why you’re doing something, then how do you associate a cost of the cloud, if you can’t associate it with a pipeline? 

      At the moment, cost is associated with instances, and you’re having to group that together. What if you could provide the model with a value while you’re doing it, the application that you’re delivering on? And if you could do that, then you’re subtly applying what they do at the practitioner level with an application, with an owner, with a cost. So that is a way that governance actually comes in with the access, the controls, without being intrusive. We can basically provide all the capabilities there, and enable the business to get better visibility control over all sorts of things, including, for the first time, cost on the end to end. 

    O’Hanlon: Yeah, that makes a whole lot of sense. And in my mind – please tell me if I’m wrong – but it also means that the developers don’t really have to think about, how shall I say? They don’t actually have to learn more about something that would – If they’re moving from, say, a regular kind of a developer motion into more of a DevOps motion, it doesn’t require them to kinda add on a level of expertise that they might not need otherwise. This kinda does it for them, and kind of fills in the blanks for them, and also kinda helps other organizations or other teams within the organization understand the flow a little bit better, which makes a whole lot of sense. 

      And I think that may be part of kinda the governance conversation as well, is if we all have a greater level of understanding about what is happening at any particular time, we can better manage it.

    Williams: Yeah. I mean, secrets management, people hear a lot about that. They don’t quite know what it is when you talk to them about secrets management. I mean, something like a password or a username would be used maybe throughout a process, and it’s a shared piece of knowledge that you don’t wanna expose. So you keep that consistent throughout your environment, but keep the access controls consistent throughout whoever uses that access throughout the pipeline. One thing that we’ve learned is that DevOps isn’t a thing; it’s an interpretation of a thing. 

      The objective is very clear for people, but how they implement it can differ, depending on organization, application, design, et cetera. So what we also have to do is make sure that people with lesser skills – And I don’t mean lesser skills in regards to infrastructure; infrastructure can be cloud provisioned. But what if they have to manage all the networking components and the database instances, the things that aren’t inherent within a cloud-provisioning environment? 

      So we’ve gotta make sure that as developers of different skill levels interact with the infrastructure, we can fill the gaps for them, so they can do what they need to do, but if there’s networking, et cetera that needs to be managed, then the model should be something to give them the guardrails with which to implement that in support of the application requirements to that part of the pipeline. So it is something that we’re working on quite heavily, to make sure that we can get those guardrails in place.

    O’Hanlon: That’s great. And I think about all of the things that have kind of expanded DevOps, if you will, beyond the original parameters, including the implementation of DevSecOps, and the adoption –

    Williams: Yeah.

    O’Hanlon: – and DevNetOps, and all of these things that have kinda piled on top of DevOps. 

    Williams: [Chuckling] Yeah.

    O’Hanlon: It can be very, very difficult for the DevSecOps community, or the DevOps community, especially developers, to kinda have to manage all of that knowledge. So having those guardrails in place, I imagine, is going to be very well received by a lot of organizations, simply because we’re also talking about the dearth of talent in the developer community these days.

    Williams: Yeah.

    O’Hanlon: So anything that an organization can do to help ensure that there is a level of governance around the way their DevSecOps tools and practices and culture and everything is run, I think is a huge benefit. David, thanks very much for walking me through what you guys see happening with DevOps and governance. And hopefully this is an issue that is going to get better as we move forward. To your point, we do tend to throw technologies at problems, and hopefully this is one that will stick, if you know what I mean. 

    Williams: Yeah.

    O’Hanlon: So thanks again for your time. I do appreciate it.

    Williams: Right. Thank you, Charlene. It’s been a pleasure. 

    O’Hanlon: All right, everybody. Please stick around. We’ve got lots more Techstrong TV coming up, so stay tuned.

  • A Guide to Cloud Migration Trends and Strategies

    A Guide to Cloud Migration Trends and Strategies

    For several years, we have witnessed widescale cloud adoption as a software infrastructure growth and maintenance strategy. Spending on cloud application services (SaaS) continues to grow as monolithic applications are replaced by microservices-based apps and monoliths become relics of the past. In 2021 alone, companies spent $410.9 billion on public cloud services, and Gartner expects this number to rise by 20.4% in 2022 to reach $494.7 billion. But as this movement continues to evolve, engineering leaders preparing their cloud migration strategy must be up to speed on the latest key trends.

    Is Migrating to the Cloud Cost-Effective?

    For companies migrating to the cloud now, the process is no longer a simple transition from on-premises to remotely hosted servers. Cloud service providers now offer a wide variety of options, and companies run the risk of overspending in the cloud marketplace.

    A lack of clarity of their business and product needs or submission to shiny object syndrome can drive companies to spend on non-essential services, especially during migration. It is easy to fall into this trap; companies only realize it after they have committed and try to undo the damage by reversing the process.

    Although many public cloud providers offer pay-as-you-go pricing, companies often purchase more services than they use. When companies are unclear about their objectives, such expenditure is more extravagant than the utility. This awareness of cloud spending has encouraged companies to take stock of their needs and expenses, and when they do not add up, to consider partly shifting back to on-premises.

    But Isn’t On-Premises a Thing of the Past?

    Not yet, but gone are the days of mindlessly unloading entire infrastructures into the cloud. Increasingly, companies are aware of the costs of pulling their data from the cloud.

    For many, security and control concerns motivate them to keep data on-premises. The public cloud has its benefits, but private storage has its place, too, especially for some components of your software. All in all, on-premises is here to stay, as companies continue to acknowledge and appreciate its value despite their reliance on public cloud environments.

    Why Do Companies Choose Cloud Migration in 2022?

    Cloud technologies can potentially deliver faster and better-performing experiences to end users, while innovations are giving rise to new services that cloud vendors can capitalize on. Various use cases are surfacing as companies understand the cloud better and see its potential. They are pushing the boundaries on how they can use the cloud, and cloud service providers are keeping track of the changing nature of the demand.

    For example, cloud-based gaming platforms like Amazon Luna are seeing significant growth. The cloud is also of great value to artificial intelligence and machine learning efforts, and these are often intertwined.

    The Great Resignation phenomenon also drives some companies towards the cloud. Although some knowledge is needed when moving to the cloud, cloud services offer automation and require less employee supervision in the long term.

    The COVID-19 pandemic has also been a catalyst for accelerating cloud adoption, making resiliency a priority. Companies can stay afloat in an emergency if their workloads are moved to the cloud. It comes at a cost, sure, but allows companies to outsource time-consuming responsibilities and focus resources elsewhere. While this was most significant in 2020 and 2021, this logic remains relevant.

    Cloud computing is also helping companies improve performance, efficiency and scalability. In addition to AI, the cloud will be instrumental in the development of more powerful augmented and virtual reality apps.

    Approaches Defining Cloud Migration Today

    The possibility of these approaches or facilities also motivates companies to migrate to the cloud:

    Containerization: The process of compiling and storing a service, along with its dependencies, in a container. Due to their small size, multiple containerized applications can run on a server, allowing for easier and cheaper shipping of the service. According to CNCF’s 2021 Annual Survey, containerization is now a mainstream practice, and Kubernetes, for instance, is a heavyweight in this domain, with 96% of companies employing or evaluating it.

    Security: In Q1 of 2022 alone, 90% of data breaches had their origins in cyberattacks. Mindful of this alarming trend, companies are factoring security concerns into their cloud adoption strategies. We are also witnessing a shift in businesses’ approach to security via DevSecOps: Incorporating security as a continuous practice from the beginning of the software development life cycle instead of relegating it to the back-burner until the last moment. Additionally, frameworks like secure access service edge (SASE) and customized cloud disaster recovery strategies are becoming essential elements of cloud adoption.

    Sustainability: Sustainability is becoming a key component in most business models. Companies are eager to adopt practices and models that reduce their carbon footprint. This impetus is also emerging from business partners and investors who are beginning to employ sustainability as a critical factor when deciding which companies to work with. When a cloud migration is planned carefully, chances are that you will retire certain services in your infrastructure and make decisions about what to migrate and what to retain. By optimizing your migration strategy, you are likely to only run and pay for the services you need, ensuring that your energy expenditure is under control.

    Edge computing: Edge computing, a $176 billion market in 2022, emphasizes the importance of building data centers locally; i.e., closer to where they are needed for storage and computational purposes. It solves any bandwidth and latency issues. Improved security is another added benefit of this practice. Edge computing has a bright future as it attempts to compensate for some of the pitfalls of using centralized servers in the cloud.

    Cloud-native applications: Cloud-native applications are easier to work with on the cloud, as they are designed and built on the cloud itself, not rearchitected to ensure compatibility with cloud services. When compatibility is not a concern, they can leverage various benefits offered by cloud services. As a result, scalability, flexibility and resilience are built into these applications. With cloud adoption, companies are increasingly in favor of cloud-native applications. This subsequently informs their migration strategy as they decide which services to retire, rearchitect or build anew once the actual migration is complete.

    How Does the Hybrid Cloud Fit In?

    Hybrid computing is on the rise as it leverages the public and the private cloud to deliver maximum benefit to companies with little to no compromise. The public cloud offers scalability, flexibility and speed, while the private cloud provides cost-effectiveness, security and compliance with regulations. While sensitive data can rest on the private cloud, frequently accessed data can be stored in public servers like AWS and Google Cloud. Investing in a hybrid cloud strategy from the point of migration itself can go a long way in giving companies access to the best of both worlds.

    Similar to a hybrid cloud environment, a multi-cloud strategy where companies distribute workloads across multiple clouds put some control into the company’s hands and ensures additional flexibility. By working across cloud services, companies can pick and choose which services are most compatible with their application’s components. These can include services offered by different cloud providers.

    Cloud Providers Seeing Growth, and What They are Doing Differently

    Serverless cloud environments: Cloud vendors like Amazon, Microsoft and IBM are pushing serverless cloud services. Although these continue to rely on servers (despite the name), companies don’t rent servers or pay fixed charges. Instead, the storage and bandwidth infrastructure scales with the software as and when it requires it. Companies do not need to concern themselves with technicalities and can focus on innovation and improving their existing applications. An added advantage is the provision of safe testing environments that encourage developers to tinker with cloud services and develop solutions for potential failures or outages. Serverless cloud solutions are also beneficial for smaller companies that need not invest large amounts of capital to ensure growth.

    Partnerships: To meet the soaring demand for cloud services, partnerships among cloud providers are also on the rise. By partnering with other service providers, companies can offer more services by leveraging the resources and reputations of one another while strengthening their competitive edge in the cloud market.

    Industry optimization: Cloud adoption is not limited to software companies. Businesses across industries like healthcare, finance and retail are migrating their infrastructures to the cloud. This variety in demand has encouraged providers to customize their services to meet the specific requirements of each industry. Often, these include regulatory compliance needs in these sectors.

    The cloud is only getting bigger and better. You can develop a well-curated transition strategy with the knowledge of current cloud migration trends. Additionally, identifying your requirements at an early stage will go a long way in leveraging the cloud to your advantage.

  • More Than Half of DevOps Pros Have Backdoor Access to IT Infrastructure

    More Than Half of DevOps Pros Have Backdoor Access to IT Infrastructure

    A survey of 600 DevOps professionals conducted by strongDM, a platform for managing access to IT infrastructure, found nearly two-thirds (64%) had productivity impacted on a daily or weekly basis because of access issues.

    It’s not surprising that, as a result, DevOps teams created a number of workarounds to gain access, even though those methods don’t comply with cybersecurity best practices, according to the survey. More than half of respondents (55%) maintained backdoor access to systems, and 42% have embraced shadow IT to get access to tools and systems they needed for their roles, the survey found.

    Worse yet, the survey found that 53% of respondents admitted they are sharing credentials across teams.

    Well over half (57%) of respondents said it could take days or weeks for access requests to be approved, with 40% reporting it takes days to add tools to existing access solutions. Nearly three-quarters (73%) of respondents also noted it requires 15 minutes or more to gain access to infrastructure.

    More than half of respondents (52%) said they have missed deadlines due to issues with accessing infrastructure, with 53% of those respondents being held accountable for the missed deadline even if it was caused by their lack of access to infrastructure.

    Justin McCarthy, CTO for strongDM, said despite all the hype surrounding identity management and zero-trust IT, it’s clear many organizations are still struggling with the basics of access control. That issue has been further exacerbated by the need to enable more DevOps professionals to work from home since the start of the COVID-19 pandemic more than two years ago, he added.

    Many of those organizations are still relying on virtual private networks (VPNs) even though it’s been shown that stealing the credentials to access those VPNs is relatively trivial, McCarthy added.

    In place of those approaches to access IT infrastructure, strongDM is making a case for a dedicated infrastructure access platform that manages and audits access to databases, servers, clusters and web apps. It is based on local client software, a gateway intermediary and configuration tools that track roles and permissions.

    The benefits of that approach span everything from lower IT support costs to faster onboarding of new IT staff, noted McCarthy.

    Managing infrastructure access for DevOps teams is, of course, always going to be a challenge. DevOps teams are made up of engineers—among the most challenging types of end users to support. Many of them will use their own expertise to circumvent any process they deem to be inefficient. The trouble is, many of those end-runs would not pass muster with cybersecurity teams responsible for access management. The challenge each of those cybersecurity teams needs to come to terms with is the amount of time they want to spend time looking for these cybersecurity policy workarounds versus making it easier to securely access IT infrastructure.

    The sad truth is both DevOps and cybersecurity professionals are wasting a lot of time and effort trying to outwit each other when that time could be put to better use.